cobalt

The modern databases you use every day almost always have the same engine shape underneath. cobalt exists so that shape stops being a black box: it implements the whole LSM write path from the log to compaction, in Zig, a language that hides not a single byte or allocation from you.

This is not another database for production but an engine taken apart into pieces where every decision is visible. Once you write the WAL, memtable, SSTable and compaction yourself, RocksDB and Cassandra stop being magic and become familiar variants of the same idea.

Classic databases write in place, updating B-tree pages, which means random writes across the disk. LSM goes the other way: every write is an append. New data lands in an in-memory structure while older layers sit immutable on disk. That makes writes fast and sequential, because sequential writes are exactly what a disk likes most.

The cost is clear and deliberate: the same data may live in several places at once and has to be merged later. The whole art of LSM is paying that debt in the background without slowing the writes that create it.

The order of these steps is not accidental. The WAL comes first precisely because it guarantees durability before the data reaches volatile memory. If the order were reversed, a power failure between the memory write and the log would mean silently lost data.

Compaction is the heart that separates a working LSM from one that chokes over time. Without it, SSTable files pile up endlessly and a read has to look through more and more layers. Leveled compaction merges them across levels, throwing out versions that were overwritten and reclaiming the space taken by dead data.

The biggest payoff of cobalt is not the store itself but that big databases stop being a mystery. Once you have walked the whole path from appending to the log to reclaiming space in compaction, you know exactly why LSM is fast on writes and where it pays for it on reads.