What’s everybody working on? (August Edition)

Still tinkering with my toy language “Moin” in my free time.

Because I like creating simple games and to see things moving, I am working on integrating raylib (the Zig wrapper). So it will be a Moin wrapper for the Zig wrapper for the C library :smiley:.

As a first step, I created helper functions to unwrap the function args from Moin Values to Zig structs/i32/[]const u8 etc.

Next step will be to auto-generate the wrappers if possible (similar to what SWIG does for several languages).

Not worth showing yet.

3 Likes

I’ve been continuing work on my music player muzika I started last month.
I’m very proud to say that though it’s still far from complete, it’s gotten to the point that I’ve already used it to listen to music while doing other stuff!

It only plays .flac files for now (though that is what my entire library is), has a playlist, and even has some basic controls (next/prev track, 5 seconds forward/back, and go to 0/10/20/…/90% point).
While working on it I considered pulling in a tui library, but after trying some out and not really liking any of them I decided against it. I thought about the api I wanted and realized that I basically just wanted some convenience functions for writing into a framebuffer, and after writing them I can confirm that its a joy to use. I’m hoping to flesh this out at some point and release it as its own thing.

And then I decided to make it integrate with the desktop so it can display what’s being played and so it can be controlled vie media keys. For that I needed dbus integration. I tried libdbus first, but the api felt like it would be a pain to work with in zig so that was a no go. Then I tried goose, the only zig implementation I was able to find in my brief search, and after a bad first impression I decided against it too.

So then, I kinda decided to uh…

Write one myself?

So this month I’ve been slowly reading through the dbus spec and implementing it. Currently I’m able to connect to dbus and immediately disconnect, and I also have a bunch of type marshaling logic done but not wired up to anything yet. I’m hoping to be able to send some messages by next week and to be able to implement the MPRIS interface by the end of the month. Hopefully also to be able to show it off not too long from now!

7 Likes

I’m new to Zig. And after I gave it a go to write the shell of my Rust OS kernel, I wanted to do a standalone project with it.
So I started doing a memory snapshot allocator called znapshot. It’s memfd and PAGEMAP_SCAN-based so the moment snapshot is called, the already-existing memory is marked as PRIVATE. So if those parts of memory is modified after the snapshot, Linux’s internal CoW mechanism kicks in and create a copy of those pages. restore is very simple where I just discard all the dirty pages and you are good to for your next iteration. And for commit, I use PAGEMAP_SCAN ioctl to go through the dirty pages and write them back to the original memfd region.

I also created a uffd-based version of it for testing but it’s substantially worse in the benchmarks. So it will just stay there probably alongside other methods (e.g. sigsegv handling) for a nice blogpost for the future.

Next step is to support nested checkpoints and rollbacks. Maybe some metrics for the future but haven’t really thought of it yet. But in general, I’m enjoying this as if I’m back in college and learning C for the first time. Very optimistic about Zig for now.

11 Likes

Keywisp - A small Wayland keystroke visualizer.

Recently I wrote a small tool called Keywisp in Zig, that displays your keystrokes on Wayland.

and also supports Waybar integration.

image

14 Likes
  • rewrote small utility to set a background message in x11 from c to zig: back-round
  • updated and upgraded, library to restrict which access a programm can have: restricted
  • started with building platform for roc in zig: restricted-roc
5 Likes

Yet another QOI decoder

const std = @import("std");

pub const Channels = enum(u8) {
    rgb = 3,
    rgba = 4,
};

pub const Colorspace = enum(u8) {
    srgb = 0,
    linear = 1,
};

pub const Header = struct {
    w: u32,
    h: u32,
    channels: Channels,
    colorspace: Colorspace,
};

const Op = packed struct(u8) {
    data: u6,
    tag: u2,

    fn rle(n: u6) @This() {
        std.debug.assert(n > 0 and n <= 62);
        return .{ .tag = 0b11, .data = n - 1 };
    }

    fn toInt(self: @This()) u8 {
        return @bitCast(self);
    }
};

const Pixel = packed struct(u32) {
    b: u8,
    g: u8,
    r: u8,
    a: u8,

    const black: @This() = .{ .r = 0, .g = 0, .b = 0, .a = 0xff };
    const transparent: @This() = .{ .r = 0, .g = 0, .b = 0, .a = 0 };

    fn hash(self: @This()) u6 {
        const vec: @Vector(4, u8) = .{ 7, 5, 3, 11 };
        return @truncate(@reduce(.Add, self.toVector() *% vec));
    }

    fn toVector(self: @This()) @Vector(4, u8) {
        return @bitCast(self);
    }

    fn diffRgb(self: @This(), data: u6) @This() {
        const Stream = packed struct(u6) { b: u2, g: u2, r: u2 };
        const diff: Stream = @bitCast(data);
        const bias: @Vector(4, i8) = .{ -2, -2, -2, 0 };
        const vec: @Vector(4, i8) = .{ diff.b, diff.g, diff.r, 0 };
        return @bitCast(self.toVector() +% @as(@Vector(4, u8), @bitCast(vec + bias)));
    }

    fn diffLuma(self: @This(), data: u6, extra: u8) @This() {
        const Stream = packed struct(u16) { g: u6, _: u2, b: u4, r: u4 };
        const payload: [2]u8 = .{ data, extra };
        const diff: Stream = @bitCast(payload);
        const g: @Vector(4, i8) = .{ diff.g, 0, diff.g, 0 };
        const bias: @Vector(4, i8) = .{ -40, -32, -40, 0 };
        const vec: @Vector(4, i8) = .{ diff.b, diff.g, diff.r, 0 };
        return @bitCast(self.toVector() +% @as(@Vector(4, u8), @bitCast(vec + g + bias)));
    }

    fn fromRgb(rgb: [3]u8, a: u8) @This() {
        return fromRgba(.{ rgb[0], rgb[1], rgb[2], a });
    }

    fn fromRgba(rgba: [4]u8) @This() {
        const swizzle: @Vector(4, u8) = .{ 2, 1, 0, 3 };
        return @bitCast(@shuffle(u8, rgba, undefined, swizzle));
    }
};

pub const Error = error{ InvalidHeader, InvalidRleChunk } || std.Io.Reader.Error || std.Io.Writer.Error;

/// Decode QOI image from source
/// Writes pixels in [31:0] A:R:G:B 8:8:8:8 format (BGRA little-endian) to the sink
pub fn decode(noalias source: *std.Io.Reader, noalias sink: *std.Io.Writer) Error!Header {
    var magic: [4]u8 = undefined;
    try source.readSliceAll(&magic);
    if (!std.mem.eql(u8, &magic, "qoif")) return error.InvalidHeader;

    const native_endian = @import("builtin").target.cpu.arch.endian();
    const hdr: Header = .{
        .w = try source.takeInt(u32, .big),
        .h = try source.takeInt(u32, .big),
        .channels = source.takeEnum(Channels, native_endian) catch return error.InvalidHeader,
        .colorspace = source.takeEnum(Colorspace, native_endian) catch return error.InvalidHeader,
    };

    const size = std.math.mul(usize, hdr.w, hdr.h) catch return error.InvalidHeader;
    const raw_size = std.math.mul(usize, size, @sizeOf(Pixel)) catch return error.InvalidHeader;
    const buffer: []Pixel = @alignCast(std.mem.bytesAsSlice(Pixel, (try sink.writableSliceGreedy(raw_size))[0..raw_size]));

    var index: usize = 0;
    var pixel: Pixel = .black;
    var lut: [64]Pixel = undefined;
    memset(Pixel, &lut, .transparent);
    loop: while (index < buffer.len) : (index += 1) {
        const op = try source.takeStruct(Op, native_endian);
        switch (op.toInt()) {
            0b11111110 => pixel = Pixel.fromRgb((try source.takeArray(3)).*, pixel.a),
            0b11111111 => pixel = Pixel.fromRgba((try source.takeArray(4)).*),
            0b00000000...0b00111111 => {
                pixel = lut[op.data];
                buffer[index] = pixel;
                continue :loop;
            },
            0b01000000...0b01111111 => pixel = pixel.diffRgb(op.data),
            0b10000000...0b10111111 => pixel = pixel.diffLuma(op.data, try source.takeByte()),
            Op.rle(1).toInt()...Op.rle(62).toInt() => {
                if (op.data + 1 > buffer.len - index) return error.InvalidRleChunk;
                memset(Pixel, buffer[index..][0 .. op.data + 1], pixel);
                index += op.data;
                continue :loop;
            },
        }
        buffer[index] = pixel;
        lut[pixel.hash()] = pixel;
    }

    sink.advance(index * @sizeOf(Pixel));
    std.debug.assert(sink.end == raw_size);
    return hdr;
}

// @memset is slow
fn memset(T: type, dst: []T, src: T) void {
    if (@import("builtin").mode == .ReleaseSmall) {
        for (dst) |*d| d.* = src;
    } else {
        const chunk: [@max(64 / @sizeOf(T), 1)]T = @splat(src);
        switch (dst.len) {
            0...chunk.len => |len| @memcpy(dst[0..len], chunk[0..len]),
            else => {
                var idx: usize = 0;
                while (idx < dst.len - chunk.len) : (idx += chunk.len) @memcpy(dst[idx..][0..chunk.len], chunk[0..]);
                @memcpy(dst[idx..], chunk[0 .. dst.len - idx]);
            },
        }
    }
}
6 Likes

I’ve been getting ready for SYCL 2026! I’ve been doing more of the boring stuff like logistics and zig master updates, but Spexguy has optimized frame rendering, introduced switched mode for the audio interface (PCM instead of basic square waves), came up with a 3D printed joystick design, and is currently adding Tracy support. Straight from the hardware and into the client!

20 Likes

Since its summer holiday time, I make a break from my s3 zig cli client which is mainly a job thing.

Now, I went back to my hobby project, my tui file manager lui.

When I find some time I’m working on a note management mode for lui to manage plain text notes (mostly markdown): https://codeberg.org/lukeflo/lui/src/branch/notes_management

Its planned as all in one successor of my tui notes management tool I hacked into fff.

Its fun and in the end I’ll have a file manager with integrated notes management all inside the terminal since I never was a fan of much too complicated note apps like e.g. Obsidian etc. I just don’t want to leave the terminal until its absolutely necessary :smile:

3 Likes

BamOS

I’m still writing my Linux-compatible kernel (syscall compatibility) in Zig since 2024. This is a huge lie in the name; it’s not an OS, just a kernel for now =) Currently, it has basic subsystems like VFS (with devfs, tmpfs, initrd, ext2 drivers), memory management, processes/threads, a scheduler, a drivers subsystem, and a bunch of other minor things.

This month I’m implementing syscalls and things needed to run the Xorg server. I recently added signals and epoll calls, and now I’m moving on to Unix sockets.


Since this forum is about Zig, I want to express my major personal pain :slight_smile: : trying to make kernel module development in Zig possible. Unfortunately, Zig essentially lacks infrastructure for Zig-to-Zig dynamic libraries (kernel modules are basically dynamically loaded executables, like .so or .dll). I tried writing a binding generator, but it feels like I just want to strip all non-inline functions and implementations from the kernel, leaving everything else, since that’s exactly what you put in C/C++ header files. Exporting Zig functions is a massive pain - it’s easier to just rewrite everything in C. After long attempts, I realized it’s all just wild hacks. I’ve looked into the C3 language to rewrite the kernel, but I’m too used to Zig and not ready to rewrite everything from scratch yet… so for now, I’ll just leave it as is.

12 Likes

But wouldn’t this marry your kernel into whatever ABI zig decides? Personally I would not like that. I recommend instead creating actual ABI spec and generate bindings from that, this is what I do for one of my projects which allows you to load and run ELF files from any OS (callconv is also part of the ABI contract!). Ashet OS (another zig kernel) does something similar https://github.com/Ashet-Technologies/Ashet-OS/blob/master/src/abi/src/ashet.abi

1 Like

I’m trying to figure out how to get zig build (master) to let me use even an empty header file with translateC from Android Termux. Because that’s currently how I program on the go. And I need to go places, and I need to program Zig. :grinning_face_with_smiling_eyes:

2 Likes

I’m exploring idea of single-threaded zero-allocation reverse proxy. All current state of the art projects (like pingora for example) falls into worker threads and even work stealing, but is it right approach (given that reverse proxy should essentially move bytes)?

So far I get good results, but it is still an experiment and maybe I’m just lacking some security features (did you know about HTTP request smuggling?) or proper TLS handling.

3 Likes

Hey everyone! :waving_hand:

For the August edition, I wanted to share the progress on my Zig personal project: ZVector, a vector database engine built from scratch to be lightweight and fast.

What got done this month

Over the last few weeks, I focused heavily on the compute and memory performance layer:

  • SIMD Vectorization with @Vector: Implemented dot product and cosine distance calculations leveraging Zig’s native @Vector primitives. The performance gains for batch vector operations in memory have been huge.
  • Memory Alignment & Layout Optimization: Re-architected data structure alignments (align(64)) to line up cleanly with L1/L2 cache lines and eliminate allocation overhead.

The current bottleneck: Binary disk persistence

This is where things got tricky. Lately, I’ve hit a wall with serializing and storing the vector database state to disk in binary format.

My goal is to keep memory usage low by persisting the vector index directly to disk, but I’m running into two main challenges:

  1. Binary Layout & Alignment: Designing a compact binary format on disk without violating Zig’s strict memory alignment rules when reloading or reading data directly.
  2. I/O & Memory-Mapping: Finding the sweet spot between simple sequential writing (std.fs.File) and high-performance random reads (like mmap) for fast nearest-neighbor search without blowing up the RAM.

What’s next

If anyone here has worked on custom binary file formats or memory-mapped files (mmap) in Zig, I’d love to pick your brain after the talks!

Thanks for listening, and I can’t wait to hear what the rest of you have been building this month! :raising_hands:

1 Like

Writing Zig modules for a Zig kernel feels natural. Your solution might be fine for syscalls if you want to generate bindings for other languages later. But when it comes to kernel modules, it means that if I have Drive.Cache.Cursor struct in my kernel, I want exactly the same thing available when developing a module.

If you specifically work on nearest-neighbor search, you need to architecture your data model to avoid reading it at all. This is a fairly complex topic. I have implemented something that could be considered a vector database in Zig, but it’s built for a very niche problem. You typically need to use some tricks that utilize patterns in the the data you want to search, if you want to be really fast. For a general purpose solution, use the standard approaches, like LSH/HNSW. In any case, you have picked a hard project. :slight_smile:

1 Like

I’ve been working on a zig-native toolchain for plain text accounting. beancount is great software and fava is especially excellent, but I am unsatisfied with the existing tooling I’ve found in the community for robust transaction importing and categorizing. In particular, the challenge of deduping imported transactions with existing transactions in the ledger in a robust and flexible way, as well as allowing for a human to work with the ledger directly and have the import tool respect that human edits while being able to manage/organize the parts of the transaction that it can still have ownership over (for example if I create a new categorization rule and have it re-categorize existing transactions in the ledger, I don’t want it to clobber the entire transaction, just the postings that need to change).

This is ending up a much bigger project than I initially intended, but I’m excited for when I can actually use it and share it with others. I’ve got a working ofx parser and several of the intermediate representations worked out. Next step is to finish the initial beancount parser and CST/AST representations, and then to get an initial hacky version of getting the parsed transactions through the IR stages and into a correct beancount AST.

3 Likes

Good morning Ziggit! Recently, I’ve taken an interest in serialising PDF documents. I took a quick look at the specification, and it turns out it is a delightfully simple format!

And so, I present my weekend project - a PDF serialisation library called Tocument. It is still a little rough around the edges, but the building blocks it provides are already enough to produce this test document:

That file is entirely generated with Zig, no other dependencies, and no AI used - just me, the PDF specification, and Tocument!

I expected it to take more work, but it does really take only 287 lines (see src/tocument.zig) to generate well-formed PDF documents. The code for the example document is currently quite verbose (and I haven’t played around with all PDF features), so it’s very likely the line count will grow.

I don’t have a concrete plan for this library as it was just a fun little weekend project, but maybe I’ll use it to automatically generate PDF documentation for the Zig standard library.

27 Likes

I am working on a cute little compiler. Right now most of my work has been in parsing expressions (which are almost done) and in creating a flat, typesafe data-oriented AST structure. I am actually very very happy with how it’s come out, though it could use better memory management. Right now it only allocates everything with an arena and forgets about it.

Complicated expressions like:

if x == b then {
    fn.[T, V](1, "two");
    x = Map[Int, String] { 1 => "1", 2 => "2", };
    b = if !cond then 'a' else 'b';
}

are tokenized into:

{ .if_kw, .ident, .double_eq, .ident, .then_kw, .lbrace, .ident, .dot_lbracket, .ident, .comma, .ident, .rbracket, .lparen, .number_lit, .comma, .string_lit, .rparen, .semicolon, .ident, .eq, .ident, .lbracket, .ident, .comma, .ident, .rbracket, .lbrace, .number_lit, .fat_arrow, .string_lit, .comma, .number_lit, .fat_arrow, .string_lit, .comma, .rbrace, .semicolon, .ident, .eq, .if_kw, .bang, .ident, .then_kw, .char_lit, .else_kw, .char_lit, .semicolon, .rbrace, .eof }

and end up in AST form as:

if_else:
  binary(==): "x == b"
    ident: "x"
    ident: "b"
  block:
    call: "fn.[T, V](1, "two")"
      ident: "fn"
      generic_args:
        typename: "T"
        typename: "V"
      number: "1"
      string: ""two""
    binary(=): "x = Map[Int, String] { 1 => "1", 2 => "2", }"
      ident: "x"
      object:
        typename: "Map"
          typename: "Int"
          typename: "String"
        keyval:
          number: "1"
          string: ""1""
        keyval:
          number: "2"
          string: ""2""
    binary(=): "b = if !cond then 'a' else 'b'"
      ident: "b"
      if_else:
        unary(!): "!cond"
          ident: "cond"
        char: "'a'"
        char: "'b'"

I implemented the pretty printing myself too.

Error messages are out there, but things are in place to make them as nice as possible.

Slowly but surely! My plan is to compile to WASM.

8 Likes

I’ve been working on a uxn/varvara emulator, to get more familiar with zig and specifically to experiment with comptime code. The uxn design has 32 main instructions but each of those has 8 versions (eg. u8 or u16 values, the working stack or the return stack, etc.), and the instructions are encoded in a single byte where 3 bits represent the 8 flag combinations and the remaining 5 bits are the instruction. The C reference uxn code uses macros to abstract over the different versions and I wanted to see what that would look like using comptime. The result was not as succinct as the C code but I think it is easier to read.

This code switches on the 5 bit instruction code (I was happy to discover that zig knows when I’ve covered all the cases of a u5 in a switch). And I only have to list the 32 instructions rather than the full 256 instruction variants. It’s not shown here but op is a comptime parameter.

        switch (op) {
            // immediate op codes
            0x20, 0x40, 0x60 => jxi(e, op),
            0x80, 0xa0, 0xc0, 0xe0 => lit(e, flags),

            // trucate to 5 bits == op & 0x1f
            else => switch (@as(u5, @truncate(op))) {
                0x00 => return false, // BRK
                0x01 => inc(e, flags),
                0x02 => pop(e, flags),
                0x03 => nip(e, flags),
                0x04 => swp(e, flags),
                0x05 => rot(e, flags),
                0x06 => dup(e, flags),
                0x07 => ovr(e, flags),
                0x08 => equ(e, flags),
                0x09 => neq(e, flags),
                0x0a => gth(e, flags),
                0x0b => lth(e, flags),
                0x0c => jmp(e, flags),
                0x0d => jcn(e, flags),
                0x0e => jsr(e, flags),
                0x0f => sth(e, flags),
                0x10 => ldz(e, flags),
                0x11 => stz(e, flags),
                0x12 => ldr(e, flags),
                0x13 => str(e, flags),
                0x14 => lda(e, flags),
                0x15 => sta(e, flags),
                0x16 => dei(e, flags),
                0x17 => deo(e, flags),
                0x18 => add(e, flags),
                0x19 => sub(e, flags),
                0x1a => mul(e, flags),
                0x1b => div(e, flags),
                0x1c => andOp(e, flags),
                0x1d => ora(e, flags),
                0x1e => eor(e, flags),
                0x1f => sft(e, flags),
            },
        }

Then each instruction function takes the flags argument as a comptime argument, which means that we get separate compiled versions of each instruction depending on flags. I write one generic version and the compiler creates the 8 specific versions.

The pop and push functions are also comptime, so they push/pop u8 or u16 values depending on flags. This is generic code that I can understand.

    fn ovr(e: *Evaluation, comptime flags: u8) void {
        const b = e.pop(flags);
        const a = e.pop(flags);
        e.push(flags, a);
        e.push(flags, b);
        e.push(flags, a);
    }

    fn jmp(e: *Evaluation, comptime flags: u8) void {
        if (comptime short(flags)) {
            e.pc = e.pop(flags);
        } else {
            e.pc = e.pc +% relative(e.pop(flags));
        }
    }

Initially I executed a sequence of instructions with a while loop and inline else, which means that the switch expands to all 256 branches - that’s how I can treat the op as a comptime known value.

    pub fn step(self: *Uxn) bool {
        const op = self.readNextByte();

        return switch (op) {
            inline else => |o| self.execOp(o),
        };
    }

    pub fn run(self: *Uxn) void {
        while (self.step()) {}
    }

Then later, I read about ‘labeled switch’ and was very happy to realise that I could get a tail-call/threaded-code version for almost no effort.

        eval: switch (e.nextByte()) {
            inline else => |op| {
                if (execOp(&e, op)) continue :eval e.nextByte();
            },
        }

Looking at the assembly output, the evaluation is inlined completely, compiling to one massive jump table, with each of the 256 instructions tail-calling to the next instruction in the sequence - it’s a thing of beauty. Thanks to the neat design of uxn where stacks are 256 bytes and ram is 64k, I can index with u8 or u16 so the compiler removes all bounds checks.

I spent a long time implementing the graphics device using @floooh’s great sokol-zig - the time spent was entirely my fault, graphics APIs have moved on since I was last working with them.

Next should be the audio device but I’ve been putting that off, instead I watched the guide to data oriented design video and started experimenting with breaking up my structs. I found that keeping the machine’s scalar values, like the program counter and stack pointers, separate from the arrays used for stacks and ram made a big difference - almost 2x faster, but then a seemingly small, and I thought unrelated, change resulted in 3x slower. I should have waited until I had it feature complete, before profiling, because now every change I end up spending more time tweaking inline/noinline and looking at struct layouts to keep the performance.

So far, I’ve really enjoyed working with zig - I find it easier to read and write compared to C, and appreciate the friction and bumps that make me stop to consider the details of what I’m doing.

10 Likes

I’ve finally completed the project that started my journey with Zig. Back in 2010, I created a custom search index for my audio fingerprint database AcoustID. It was a c++ project, it’s essentially a persistent inverted index, continually updated at real-time, very much inspired by Lucene, but written for my specific needs. I made a strange decision and used Qt for writing the networking side of the server, as I was familiar with it and it had good async I/O support. I’ve never seen anybody crazy enough to use Qt in a server application.

Anyway, last year, after watching Andrew’s video where he talked about how bad C++ is and used my project Chromaprint as an example, I decided to give Zig a try, it has been on my radar for a while. :slight_smile:

So I decided to rewrite the C++ project in Zig, as I had some ideas how to make it better. It was a nice experience, I fell in love with the language, but I was disappointed in the ecosystem. It was a huge step backwards compared to what I was used to in C++, but I didn’t need to finish the project in time, I used it as a therapy for myself, so I just kept pushing. I created a msgpack library, so that I can serialize structs to disks efficiency and still have an easy way to inspect them using other tools. The I started working on the NATS client, which I didn’t end up using after all, but I did finish. Then came my frustration with the fact that I’m using threads for networking, which is worse architecture than I had before. I started experimenting with using libuv and libxev, but using callbacks in Zig just sucks. So I did another crazy thing and started working on zio, and then http client and server (dusty), and just keep iterating on those projects for a long time.

Yesterday, I finally deployed the project to production and it’s now serving all AcoustID traffic. It has a database of around 100M audio fingerprints, finishes one search in about 10ms, 5x faster than the previous version, stores the data compressed using StreamVByte using a new layout, which means it uses about 20% less RAM and about 50% less CPU thanks to more efficient SIMD decoding. Everything is highly concurrent, there are thousands of active coroutines at any point, using io_uring per thread for network and file I/O. I’m not using std.Io anywhere in the project, besides the Dusty client/server. I’ve given up on gRPC for now and use msgpack-over-HTTP for now. Overall, I’m super happy with it.

LLM disclosure: the first Zig version of the project was completely hand made, but the last migration/rewrite to zio was done mostly by Claude Code using my original code as template, I didn’t want to wait any longer until I have energy to do the migration myself

27 Likes