Zig-msgpack: zig implementation of message pack

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it’s faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.

Using the comptime features of zig can help us transfer many runtime things for other languages to the comptime!

Github: GitHub - zigcc/zig-msgpack: zig messagpack implementation

And I wrote an article to introduce and illustrate it, including a brief explanation of the principles and how to use it

9 Likes

The reason why I developed this library is that neovim uses message pack as serialization format and I am currently developing neovim’s remote rpc client

Wow!
I’m also working on a zig msgpack library – as my first non-trivial learning-Zig project. (I would link to it, but it’s sitting in my private catch-all repo at the moment).

I’m building mine on top of getty: GitHub - getty-zig/getty: A (de)serialization framework for Zig

Just wondering, did you look at getty and decide not to use it?

Anyway, once mine is a little closer to completion, I’ll compare the two implementations and probably learn a lot from that.

thanks!

1 Like

I’ve looked at this library but I don’t know there is a message pack implementation

Yep, as far as I know, there’s no getty msgpack implementation, except for the one I’m writing.

What I meant was: did you consider using getty for your own msgpack library?

I just re-read the following getty, it currently only supports json, right?

At present, the functions of zig-msgpack are sufficient. Unless I develop messagepack-rpc, no new libraries will be introduced.

From the getty README:
“Getty is a framework for building robust, optimal , and reusable (de)serializers in Zig.”

So, getty can be (according to that description, at least) used to build any kind of de/serializer for zig. (which is what I’m attempting)

The only library I’ve seen built on getty is getty-json, which I assume is what you are referring to when you say “only supports json”.

But, getty itself is not limited to json only.

At present, the functions of zig-msgpack are sufficient.

Understood. That’s what’s important.

1 Like

I understand what you mean. As a framework this is great. If I develop a new serialization library in the future (such as fast binary protocol), I will try to use getty.

Are you going to support the timestamp and/or user extension types?
I am using extension types extensively and I am not sure how to translate those into Zig comptime (but maybe that will be clear once you publish your code).

I saw that the person you replied to me was not me (I don’t know if it was the wrong reply), but I still want to explain zig-msgpack.
In fact, the ext type of msgpack is a type and corresponding data
I have implemented this in zig-msgpack, the type is EXT
I didn’t implement timestamp because I didn’t know under what circumstances users would use timestamp.
zig-msgpack only provides basic sending and reading of EXT format, and the remaining user-defined types only require a little extra processing.

I didn’t mean to hijack this thread at all, but since you asked.

I had intended to support timestamps, at least, because I want it for my own purposes. But, I hadn’t given the details much thought and assumed it would be fairly straightforward. However, after reviewing getty’s data model documentation again, I’m less sure. It appears as if timestamps don’t have built-in support.

That’s not a show stopper; getty is extensible. But, now I’m guessing it will be more of a challenge than I had expected, especially given my novice status with zig.

I’d still like to attempt it, but probably not in the first release (which will be focused on soliciting feedback).

Interesting… Apple just released documentation about how to write new language bindings for Pkl and it’s based on msgpack. Pkl Binary Encoding :: Pkl Docs

1 Like

Interesting, is this a library for generating configurations?
I saw it exists bindings for go and swift

Yes. It’ll be even more interesting once there’s support for more languages.

Reimplemented payload, standardized naming, and added online documentation