Serde.zig - format-agnostic serialization framework

Showcase
1

Hello,

I’ve been working on a serialization library that gives a single
API across JSON, MessagePack, TOML, YAML, ZON, and CSV. All six
formats support both serialize and deserialize.

The core idea: define a struct, call serde.json.toSlice or
serde.yaml.fromSlice, same pattern everywhere. Swap the format,
keep your code.

const bytes = try serde.toml.toSlice(allocator, config);
const cfg = try serde.toml.fromSlice(Config, arena.allocator(), input);

Uses @typeInfo to walk struct fields at comptime. Supports nested
structs, optionals, tagged unions, enums, slices, StringHashMap,
pointers, tuples, void.

Serde options

You can declare pub const serde_options on your types to
customize behavior. Everything resolved at comptime.

Field renaming and naming conventions:

const User = struct {
    user_id: u64,
    first_name: []const u8,

    pub const serde_options = .{
        .rename = .{ .user_id = "id" },
        .rename_all = serde.NamingConvention.camel_case,
    };
};
// => {"id":1,"firstName":"Alice"}

Available conventions: .camel_case, .snake_case, .pascal_case,.kebab_case, .SCREAMING_SNAKE_CASE

Skip fields:

pub const serde_options = .{
    .skip = .{
        .token = serde.SkipMode.always,     // never serialize
        .email = serde.SkipMode.@"null",     // skip if null
        .tags = serde.SkipMode.empty,        // skip if empty slice
    },
};

Flatten nested structs:

const User = struct {
    name: []const u8,
    meta: Metadata,

    pub const serde_options = .{
        .flatten = &[_][]const u8{"meta"},
    };
};
// {"name":"Alice","created_by":"admin","version":2}
// instead of {"name":"Alice","meta":{"created_by":"admin",...}}

Union tagging styles (external, internal, adjacent, untagged):

const Command = union(enum) {
    ping: void,
    execute: struct { query: []const u8 },

    pub const serde_options = .{
        .tag = serde.UnionTag.internal,
        .tag_field = "type",
    };
};
// .external:  {"execute":{"query":"SELECT 1"}}
// .internal:  {"type":"execute","query":"SELECT 1"}
// .adjacent:  {"type":"execute","content":{"query":"SELECT 1"}}
// .untagged:  {"query":"SELECT 1"}

Enum as integer:

const Status = enum(u8) {
    active = 0, inactive = 1,

    pub const serde_options = .{
        .enum_repr = serde.EnumRepr.integer,
    };
};
// serializes as 0, 1 instead of "active", "inactive"

Per-field custom serialization:

pub const serde_options = .{
    .with = .{
        .created_at = serde.helpers.UnixTimestampMs,
    },
};

Built-in helpers: UnixTimestamp, UnixTimestampMs, Base64

Deny unknown fields:

pub const serde_options = .{
    .deny_unknown_fields = true,
};

For full control you can also declare zerdeSerialize /
zerdeDeserialize methods directly on your type.

There’s also zero-copy JSON deserialization via fromSliceBorrowed

  • when strings have no escape sequences, returned slices point
    directly into the input buffer.

Requires Zig 0.15.0.

Would appreciate feedback on the API, missing type support, or
anything that feels off

14 Likes
2

Is it possible to serder containers you don’t control, or just in case you want to serder single container in multiple ways? Aka define the encoding / decoding out-band instead of in-band. I think my main gripe with std.json and such is that they all are based on in-band serder concept and depend on magic functions implemented in the struct if you want to deser some fields differently than the default.

1 Like
3

yes, right now this is purely in-band. All the
config (pub const serde_options, zerdeSerialize/zerdeDeserialize)
has to live on the type itself. If you don’t own the type, the only
option today is wrapping it:

const MyPoint = struct {
    inner: ThirdPartyPoint,
    pub fn zerdeSerialize(self: @This(), s: anytype) !void {
        // ...
    }
};

Which works but is annoying, especially if you need the same type
serialized differently in two places.

I’ve been thinking about this and I think the right approach is an
optional comptime config you can pass at the call site. Something
along these lines:

const ThirdPartyPoint = struct { x: f64, y: f64, z: f64 };

// config defined outside the type
const point_cfg = serde.Config(ThirdPartyPoint, .{
    .rename = .{ .x = "X", .y = "Y" },
    .skip = .{ .z = .always },
});

// pass it when serializing
const bytes = try serde.json.toSlice(allocator, point, .{
    .config = point_cfg,
});

// same type, no config - default behavior, all fields
const full = try serde.json.toSlice(allocator, point, .{});

Internally this would be a comptime parameter on serialize/deserialize.
If the call site provides overrides for T, use those; otherwise fall
back to the type’s own pub const serde_options like today. Since it’s
all comptime, it compiles to the same specialized code - no runtime
dispatch

This covers both cases: types you can’t modify, and one type with
multiple serialization shapes depending on context.

Not implemented yet, but it’s on my list. The plumbing in
core/serialize.zig and core/deserialize.zig already reads options
through getSerdeOptions(T), so it’s mostly a matter of threading
an extra comptime parameter through that path and merging it with
the type-level options.

If you have opinions on the API shape (config at call site vs
attached to a serializer instance vs something else) I’m all ears

2 Likes
4

I was thinking something like:

const decoder: std.json.Decoder(T) = .schema(.{
  .field1 = .string, // decode a number as string 
  .field2 = .custom(someCustomLogic),
});
const parsed = try decoder.fromSlice(arena.allocator(), slice);
2 Likes
5

TBH, I think I prefer no in band config, if it has its own config it must be passed explicitly. I think it would be a very minor inconvenience with the benefit of being very explicit.

1 Like
6

Yes, I think i ll implement this

7

Interesting to see if you come up with any stumbling blocks with that kind of design. The decoder/encoder can have different kind of initializers to remove boilerplate. .schema is the kind of typical problem that I usually have when I want fields to just be interpreted in certain way.

For example I’m not sure how field renames etc should be expressed here.
Perhaps the coder could have functions that you can call after initialization like rename(.field1, "something") (runtime cost?), or have initializers accept more options for field specific options, just some ideas anyhow.

1 Like
8

I love the idea!

I’m curious about the naming situation, though. I assume it’s inspired by rust’s serde. But is this project just taking the name outright? I noticed ‘zerde’ in one of your examples, and that seems more appropriate.

1 Like
9

It probably shouldn’t harm discoverability too much. I’ve seen a fair amount of people asking for something like Serde in Zig in discord servers and such, so presumably searching for “serde zig” could yield more results when it’s named this way rather than zerde.

P.S. I’m not incredibly knowledgeable on SEO

3 Likes
10

Also serde is pretty common term for serialization/deserialization

5 Likes