Msgpack library

As part of a project, on which I’m learning Zig, I implemented a library for working with msgpack. There is one alternative msgpack library for Zig, but that did not fit my needs, as it tries to be dynamic. My library is very static and low level. I need it to be fast and encode low level data structures exactly as I want without overhead of constructing variant objects.

It’s not a standalone project for now, and there are no guarantees on the API yet, but if there is interest, I can turn it into a library that can be easily included into build.zig

Comments on the API are very welcome.

10 Likes

Can’t wait to give this a try in a few days, if you would like help packaging it for the zig package manager I am happy to help.

It’s fairly common for me to use static msgpack formats and redis for effectively interprocess communication / queues.

It’s great to see another Zig developer working on a static, performance-focused msgpack implementation.

The focus on static, low-level implementation that handles data structures without unnecessary overhead sounds exactly like the approach I’m taking, although my project is primarily for learning purposes and I don’t plan to maintain it long-term.

I look forward to checking out your code once my implementation is more mature. I’m curious to see the different approaches we took to solving a similar problem. Feel free to share more details about your library as it progresses!

I’ve updated the main API to match the conventions from std.json and moved the code to a separate repository. API is still not stable, I’m open to comments, as I’m still learning what’s typical in Zig. Code works on 0.13, not the development version of Zig.

2 Likes

Love msgpack format, great to see this project; been using it for a few years with msgspec and litestar in Python land.

Released version 0.5.0 with zig 0.15 support and msgspec-compatible encoding of tagged unions.

3 Likes

Nice! Looking forward to checking this out.

Release version 0.6.0 with support for the new std.Io.Reader/Writer interfaces. The main benefit is smaller binary size, as we it no longer needs to deal with anytype and also for null handling, it’s beneficial that the code can peekByte() on any reader.

1 Like

Great library, have you tried benchmarking it?

I’d like to know a performance comparison between these two.

Oh, your library is static, great, the library zig-msgpack of zigcc is dynamic parsing

I believe dynamic performance is inferior to static performance.

But I still want to know how much the performance will be worse.

1 Like

I’ve not benchmarked my library, but I expect it to be faster than any dynamic solution.

I use it in a very performance sensitive project and it’s fine.

But I’d welcome an external benchmark.