kestrel

A relational database engine from scratch in Kotlin: a SQL parser, B-tree storage, an executor, WAL transactions and a CLI. It runs real SQL queries and is covered by a full test suite.

TL;DR

A relational database engine written from scratch in Kotlin: its own SQL parser, a query executor, B-tree storage and WAL-backed transactions. Plus a CLI and a full test suite.

Overview

kestrel is a relational database engine written from scratch in Kotlin. It has everything that makes a database a database, not just a place to keep data: its own SQL parser, a query executor, B-tree storage, WAL-backed transactions, plus a CLI and a full test suite. It runs real SQL queries rather than pretending to handle them.

Day to day, a database is one line: run a query, get rows. Writing one from scratch strips that comfort away and shows how much happens underneath - and that journey downward is the whole point of this project.

Behind one innocent SELECT hides a chain of hard questions. How do you keep data on disk so you can find it fast? How do you turn SQL text into an execution plan that knows what to read and in what order? And the hardest: how do you not lose a write when the power vanishes halfway through?

Each of those questions is usually hidden behind the comfort of a ready-made database. Writing an engine from scratch pulls them out into the open one by one and makes you solve them for real, not wave them away. That is why a database is one of the best projects you can set yourself to understand the computer more deeply.

The full path of a query

kestrel takes an ordinary query and pushes it through the full path - from text all the way to bytes on disk. Each stage has one job and hands its result to the next, and you can trace the whole thing with a finger from a line of SQL to where the data lands.

demo.sql · sql

CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT);
INSERT INTO users VALUES (1, 'ada');
SELECT name FROM users WHERE id = 1;

Parser

SQL text turns into a query tree.

Executor

the tree becomes a plan: what to read and in what order.

Storage

data lives in a B-tree so lookups by key are fast.

WAL

every change goes to the log first, only then to its place.

The engine's layers

Layer	Job
Parser	SQL into a query tree
Executor	tree into an execution plan
Storage	data in a B-tree for fast lookups
WAL	a log of intent before the actual write

WAL, or why you do not lose data

The hardest part of a database is invisible until something goes wrong. The WAL is the mechanism that decides whether, after a sudden power-off, the database returns to a consistent state or leaves you with half a written transaction. The rule is simple and inviolable: write the intent to the log first, then the actual change - never the other way round.

Warning

The WAL is not decoration. It is what decides whether, after a sudden power-off, the database returns to a consistent state or leaves you with half a written transaction. Write the intent to the log first, then the actual change - every reversal of that order is a potential data loss.

The result: knowledge that stays

It is a project that, once finished, changes how you look at every other database - because suddenly you know what sits under that one SELECT. kestrel runs real queries, passes a full test suite and has a CLI, so it is not a sketch but a working engine. And the knowledge left behind is worth more than the code itself.

More projects

More work from the same category - see how we tackle similar challenges.

Have a similar project?

Get in touch - a quote is free and comes back within an hour.