Introducing DDB.zig – A simple, embedded database for Zig

Hello Zig community! :waving_hand:

I’ve been working on a small library in my free time and I’m excited to share the first version of DDB.zig with you.

DDB stands for a simple way to manage structured data in your Zig applications. It’s not a full-fledged SQL database, but rather a lightweight, embedded library that provides a basic set of tools to persist and load your Zig structs with minimal boilerplate.

Think of it as a simple, file-based document store for Zig.

:sparkles: Key Features

  • Simple API: Designed to be intuitive for Zig developers. You work directly with your structs.
  • Schema based on Zig types: Define your data once as a Zig struct – the table schema is derived from it.
  • Easy integration: Can be added to your project with a single zig fetch command.
  • Currently supports: Linux and macOS (Zig version 0.15.1).

:package: How to Install

Add it to your project:

zig fetch --save https://github.com/Neon32eeee/DDB.zig/archive/refs/heads/main.tar.gz

Then import the module in your build.zig (a full example is in the README).

:rocket: Quick Example

Here’s a glimpse of how it works:

const std = @import("std");
const ddb = @import("ddb");

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    var db = try ddb.DB().init("my_app_data", allocator);
    defer db.deinit();

    const Player = struct {
        hp: i32,
        name: []const u8,
        score: i32,
    };

    try db.createTable("players", Player);
    const players_table = db.getTable("players").?;

    const player1 = Player{ .hp = 100, .name = "Jon", .score = 100 };
    const element = try ddb.Adapter.toElement(player1, allocator);
    try players_table.append(element);

    try db.save();
}

This creates a database file, defines a “players” table, and appends a new player entry.

:light_bulb: Why DDB?

I wanted something even simpler than SQLite for small tools, game prototypes, or applications where I just need to save and load state without writing a single query. If you’ve ever needed to persist a few structs and found setting up a full database engine to be overkill, DDB might be for you.

:link: Links & Feedback

This is a very early release (my first Zig library!), and I’m looking for feedback, ideas, and contributors. What features would you like to see? (Querying? Indexes? Better error handling?) Feel free to open an issue or start a discussion here!

Thanks for checking it out! :folded_hands:

11 Likes

Hey welcome on Ziggit, I’ve tried to compile your project but I think you might have accidentally left a typo in there :slight_smile: Cool idea otherwise!

~/r/DDB.zig main• 5.7s ❱ zbr
run
└─ run exe DDB
   └─ compile exe DDB Debug native 1 errors
src/Table.zig:22:34: error: no field named 'tmane' in struct 'Types.Element'
        if (!std.mem.eql(u8, row.tmane, self.tname)) {
                                 ^~~~~
src/Types.zig:10:21: note: struct declared here
pub const Element = struct {
                    ^~~~~~
referenced by:
    main: src/main.zig:28:24
    callMain [inlined]: /home/pollivie/.local/share/zigup/0.15.1/files/lib/std/start.zig:627:37
    callMainWithArgs [inlined]: /home/pollivie/.local/share/zigup/0.15.1/files/lib/std/start.zig:587:20
    posixCallMainAndExit: /home/pollivie/.local/share/zigup/0.15.1/files/lib/std/start.zig:542:36
    2 reference(s) hidden; use '-freference-trace=6' to see all references
error: the following command failed with 1 compilation errors:
/home/pollivie/.local/share/zigup/0.15.1/files/zig build-exe -ODebug --dep ddb -Mroot=/home/pollivie/review/DDB.zig/src/main.zig -ODebug -Mddb=/home/pollivie/review/DDB.zig/src/root.zig --cache-dir .zig-cache --global-cache-dir /home/pollivie/.cache/zig --name DDB --zig-lib-dir /home/pollivie/.local/share/zigup/0.15.1/files/lib/ --listen=-

Build Summary: 3/6 steps succeeded; 1 failed
run transitive failure
└─ run exe DDB transitive failure
   └─ compile exe DDB Debug native 1 errors

error: the following build command failed with exit code 1:
.zig-cache/o/5afa9343b1dd41fee14c1fb231500a13/build /home/pollivie/.local/share/zigup/0.15.1/files/zig /home/pollivie/.local/share/zigup/0.15.1/files/lib /home/pollivie/review/DDB.zig .zig-cache /home/pollivie/.cache/zig --seed 0x24b23323 -Z60bfd7cf64a95e7b run
~/r/DDB.zig main• | 1 ❱
1 Like

I’ve already fixed this bug in a new commit. Sorry for the inconvenience.

3 Likes

Don’t worry it happens :slight_smile: Great idea btw I think it’s a great project and a very useful one there are probably a lot of smaller project who as you have said don’t need to drag in a “industry grade” db but could still benefit from a smaller one. Hope you’ll keep maintaining it in a few iteration it might be a really great project.

4 Likes

Welcome to Ziggit @Neon32eeee!

First tip: there are languages, like Julia, where .jl is an expected part of repo names, Zig isn’t one of them. DDB is plenty.

I’d like to see this do “real database” stuff. Your current design pulls everything into memory, and writes it all on save, and it does it in a way which is not nearly paranoid enough imho. It also has per-table metadata files, which is even more interaction with the file system.

This kind of design tends to work reliably (enough) while data is small, but invisibly degrades insofar as that is no longer true. It’s also just a stroke of bad luck away from corrupting everything, at all times.

The standard is to use a B+ tree, and it isn’t as difficult as it might seem. There’s a lot of information online, it’s well-trodden ground, chatbots will probably not say totally insane things if you ask them about it.

Forget queries, joins, all that, the light table-oriented architecture is just fine. What you want is: interaction with the file system proportional to what’s requested from the database, and proportional to what gets changed. You also (this is harder, but well worthwhile) want to ensure that bad things happening cannot put the database into an unrecoverably bad state: this means transactions.

I would put multiple writers out of scope, personally, but do handle it properly: have a write lock on the file, always acquire it before writing, release it when the transaction is committed, etc.

Transactions don’t have to be user-visible, either: it can just mean that when code flushes new contents to the database, that either succeeds completely, or it has no effect.

Atomicity, Consistency, Isolation, and Durability: that’s a database.

3 Likes

DDB is designed as an in-memory database for scenarios where all data fits in RAM and maximum access speed is required. All data is saved to disk for simplicity and reliability. B+ trees are an excellent mechanism for disk-based databases, but they are overkill for my target audience. However, I am open to suggestions for improvements in the world of in-memory databases.

That’s fair, you asked what features we’d like to see, I told you.

Be a lot more paranoid about saving to disk.

3 Likes

Okay, I’ll try to improve saving to disk in the future.

2 Likes

What exactly is overkill there? B+ tree feels like if it was invented for databases. It also fits nicely to the “paranoid saving to disk” because you can write a new page and only then update the header pointing there. So if something goes wrong, you should still have consistent db file.

I suppose I’m oversimplifying a bit but if I were to implement a db, I’d use B+ tree.

1 Like

That’s not the only way to do databases. The trend over the last 10-20 years has been towards layered architectures. Especially if you go to the really large distributed databases, but it applies at any scale. If I was writing an in-memory database, I’d definitely not use B trees of any kind, but have proper WAL and LSM-tree if I want to have consistent storage as well.

The literature is pretty divided on LSM- vs B+ -Trees. I’m more in the latter camp as I find it easier to reason about since the in memory representation is equal to the one on storage and one also gets the paging semantics for free and there is no need for compaction.

There are ways to make B±Trees as fast as LSM Trees for in memory workloads, but again I’m a bit biased: link to paper

2 Likes

The API seems a bit strange. I would expect that the columns can be set and get by a constant column index, like I know from other DBs.

So I could ask the table for the index eg of the “name” column once, assign that to a const u8 and then use this to get the name attribute for every row.

This should be much faster than looking up a string every time, and if I understand correctly, each table has a fixed set of columns like classic SQL databases anyway, so this would be a good match.

On another point, for the DB to be actually useful as a DB, I think it needs at least a feature to find a record somehow, even if it’s only by one particular index column. Like what VSAM KSDS was doing 40 years ago.

Without that, I wouldn’t call it a database, but a serializer/deserializer, which ofc can be useful enough in itself.

2 Likes

Hi! Thanks for the thoughtful suggestions.

About column access by index – you’re absolutely right, that would be a solid performance improvement. I’m going to think about how to best integrate it into the current design and will likely add it in a future update. I appreciate you pointing that out!

About adding an index for record lookup (like VSAM KSDS) – I see where you’re coming from, but I think that’s a bit against the philosophy of DDB. The library is intentionally kept simple: it’s an in-memory store where users have full control over filtering and searching via plain Zig code (like for loops). Adding a built‑in indexing mechanism would introduce complexity and opinionated APIs, which I’d like to avoid. DDB aims to be a lightweight tool, not a full‑fledged database engine.

That said, I’m always open to discussion, so if you have ideas on how to keep it simple while still offering efficient lookups, I’m happy to hear them. Thanks again for the feedback – it’s helping me clarify the direction of the project!

Edited: Also, if you can, please write more details about indexing using DDB termins. Thanks if you do!

1 Like