Stanchion: Column-oriented tables in SQLite

Stanchion is a SQLite extension that allows you to use column-oriented tables in SQLite like you would any other table in SQLite (well, for now just inserting and querying are supported but deleting and updating will come later).

I built stanchion to scratch my own itch. I have personal applications running on SQLite where I want to store log and metric data (basically forever) but don’t want to add complexity by bringing in a new database or whole separate server. Some of these applications are running on a Raspberry Pi, where storage and compute are limited, but I still want data warehouse like capabilities.

I envision stanchion being used in similar scenarios: on phones and resource-limited devices in applications that are already using SQLite. I know that there are alternatives like DuckDB (which is very cool), but I want stanchion to be “good enough” that it is useful without having to add whole new database technology.

This is my first major project in Zig, and I have really been enjoying it. It’s also my first project in a long time in a systems language that doesn’t have a borrow checker, and it’s been a lot of fun remembering what it’s like to have full control of memory. That also means I am sure there are lots of opportunities for improvement.

On that note, feedback on the code, especially with Zig best practices in mind, and optimizations are very welcome (either here, in a github issue, or in a PR if you are really interested). If you also happen to think stanchion may be a good fit for your use case and you are interested in contributing, please test it and provide feedback by opening issues for any bugs, difficulties, or missing features you would need! The CONTRIBUTING doc in the repository has more details. I’m also just happy to answer questions about the project and my experience with Zig.

7 Likes

BTW, column oriented tables are in particular very good for so called anchor_modeling, where an entity is one column table (ID) and it’s properties are two-column tables (FK, value).

That sounds a lot like Datomic’s model. I don’t see Datomic mentioned in the wikipedia article, so I wonder if it is actually the same? Either way, it’s very cool. Adding to the readme

https://en.wikipedia.org/wiki/Datomic is an DBMS, anchor modeling is a data modeling technique, it can be used with any relational DBMS, regardless of whether it’s record-oriented or column oriented.