Dolt Roadmap update
Introduction
At DoltHub, we ship features, a lot. We fix bugs in 24 hours, but we also have a lot of long-term projects we're working on at any given time. We keep track of these goals on our roadmap, which just got its periodic update on what's coming next and what we launched recently.
Our next major goal is getting the Postgres version of Dolt, Doltgres, to a production quality Beta release. Doltgres Beta will ship in Q1 2025, with hosted deployment available. This is occupying a lot of our time, but we're also working on a bunch of new features for Dolt as well.
Vector indexes
Vector databases are hot because of AI. Vectors are multi-dimensional arrays of floating-point numbers that you can search for approximate nearest neighbors. Doing this efficiently lets you improve the output of LLMs without retraining them, which is why so many people are interested in them right now. We have written about our plan for vector support a few times now, and work is currently underway. Watch this blog for updates.
Hosted GCP private cloud
Dolt is free and open source, so anyone can run it on their own hardware for free. But many customers prefer to pay us to run their databases for them via our hosting service. The hosted offering started out AWS only, then got support for Google Cloud Platform (GCP) earlier this year. Then we added support for virtual private cloud (VPC) for AWS instances in July. Now we're hard at work on the equivalent functionality for our GCP customers. GCP private cloud will ship later this year.
Doltgres authorization
Doltgres is making rapid progress towards its beta release. One of the last remaining large
must-have features before we can launch is support for authorization, e.g. USER
and GRANT
statements. This lets you create users, assign them passwords, and control which tables they can
query or modify. Doltgres auth features will ship later this year.
Doltgres SQL support
Doltgres can support many customers' schemas now, but there are still some relatively common features that we need to support before we can call it beta quality. We're working on these now and will have them out this year.
- User-defined types
- User-defined functions
- Full function and pg_catalog coverage
Database improvements in 2025
Dolt and Doltgres share an engine, so whenever we improve that engine both products benefit. We have several large improvements to that engine that won't be ready this year, but we'll be working on in 2025. These include:
- Automatic garbage collection. Dolt generates a lot
of garbage when issuing updates or inserts to a database, generating intermediate values that
never make it into a commit graph. The
dolt_gc
stored procedure will clean up this garbage to reclaim disk space, but we're working on automatic garbage collection so this is unnecessary for most customers. - Row-level locking. Dolt doesn't yet support statements with the
SELECT FOR UPDATE
clause, which takes row-level locks on all the rows selected. This is required for certain concurrency workflows. - Transaction isolation levels. Dolt only supports
the
REPEATABLE READ
transaction isolation level, which is the default in most databases. Some customers might need support for other levels though, such asREAD COMMITTED
. Support for those will begin in 2025.
Other cool things, yet unscheduled
We keep a list of things we think would be cool to implement but haven't committed to yet on our roadmap, in case a prospective customer really wants them and can tell us. Paying customers move to the front of the roadmap. Do any of these look interesting?
- Encryption at rest. Dolt is already tamper-evident, but some compliance processes require data also be encyrypted at rest. The best way to do this today is to place Dolt's data on an encrypted partition, but native support might be nice in some contexts.
- History compression. Dolt takes more space than a typical database because it stores snapshots of every commit in history. If you only need data from a certain time in the past, or at a certain granularity, it would be nice if Dolt could compress the data, rewriting the commit history to eliminate all the unneeded commits and their storage.
- Customizable merge rules. Dolt is the only database with merges. But Dolt's merge logic is pretty rigid, and doesn't do quite what some customers would want. There are workarounds, but they require writing a lot of application code. Dolt could instead let you specify merge logic as a stored procedure to make merges work better in your domain without application-layer cleanup.
- Other frontend support. Dolt is a drop-in replacement for MySQL. Doltgres is a drop-in replacement for Postgres. Should we support other database front-ends, like MongoDB or Microsoft SQL Server?
Conclusion
It's an exciting time at DoltHub. Dolt just went 1.0 less than 18 months ago, and we've been amazed at everything our customers have built with it in that time.
But we're just getting started. Do you need something from Dolt that it can't do yet? Let us know! Come by our Discord and we'd be happy to help you out.