`jsonStrinfify` of nested tagged union

Hi,

I’m trying to serialize a tagged union to json. The problem is one of the union fields holds a value to an array of the union itself. And I’m just not sure how to correctly serialize this.

Here is the code snippet. The values are defined by the underlying data structure:

const Filter = union(enum) {
    prefix: []const u8,
    tag: []const u8,
    object_size_greater_than: usize,
    object_size_less_than: usize,
    @"and": []Filter,

    pub fn jsonStringify(self: *const Filter, jws: anytype) !void {
        try jws.beginObject();
        switch (self) {
            .prefix => |val| {
                try jws.objectField("Prefix");
                try jws.write(val);
            },
            .tag => |val| {
                try jws.objectField("Tag");
                try jws.write(val);
            },
            .object_size_greater_than => |int| {
                try jws.objectField("ObjectSizeGreaterThan");
                try jws.write(int);
            },
            .object_size_less_than => |int| {
                try jws.objectField("ObjectSizeLessThan");
                try jws.write(int);
            },
            .@"and" => |val| {
                for (val) |filter| {
                    // Not sure how to get this done
                }
            },
        }
        try jws.endObject();
    }
};

The and field won’t come up again in the nested array of the union again. It is restricted to the top layer.

The simplest way seem to call the jsonStringify() fn again from inside the loop, but I think thats not really possible since its executed by std.json.Stringify directly…

There might be an obvious solution, however, I haven’t found it yet…

Edit: Sorry for the incomplete question posted first, my kids interrupted me and I unintentionally pressed enter:grin:

When you have a custom jsonStringify function, it is entirely up to the function how to stringify the type, there is no reason you cant call it recursively for a recursive type. When that function returns Stringify assumes the type is done and moves on.

But do make sure to [begin/end]Array before and after the loop…

1 Like

Thanks. So you mean something like this:

    pub fn jsonStringify(self: *const Filter, jws: anytype) !void {
        try jws.beginObject();
        switch (self) {
            .prefix => |val| {
                try jws.objectField("Prefix");
                try jws.write(val);
            },
            .tag => |val| {
                try jws.objectField("Tag");
                try jws.write(val);
            },
            .object_size_greater_than => |int| {
                try jws.objectField("ObjectSizeGreaterThan");
                try jws.write(int);
            },
            .object_size_less_than => |int| {
                try jws.objectField("ObjectSizeLessThan");
                try jws.write(int);
            },
            .@"and" => |val| {
                try jws.beginArray();
                for (val) |filter| {
                    try filter.jsonStringify(jws);
                }
                try jws.endArray();
            },
        }
        try jws.endObject();
    }

However, this fails with:

json_serialize.zig:51:14: error: expected type '*const json_serialize.Filter', found '@typeInfo(json_serialize.Filter).@"union".tag_type.?'
            .prefix => |val| {
            ~^~~~~~
json_serialize.zig:31:16: note: enum declared here
const Filter = union(enum) {
               ^~~~~

I think my access of the active union field in the stringify function is wrong. But couldn’t figure it out so far reading through the source code…

In order to switch on a tagged union, you have to dereference it. Then, you can explicitly capture it by a pointer by using |*val| syntax.

2 Likes

Oh, damn it. Sometimes I just overlook the simple things because I think it has to be complicated… :smile:

Thanks, to both @xeondev and @vulpesx I was able to figure it out. Need to use a nested beginObject() (not array). And its simpler with a little helper function:

const Filter = union(enum) {
    prefix: []const u8,
    tag: []const u8,
    object_size_greater_than: usize,
    object_size_less_than: usize,
    @"and": []const Filter,

    pub fn jsonStringify(self: *const Filter, jws: anytype) !void {
        try jws.beginObject();
        switch (self.*) {
            .@"and" => |val| {
                try jws.objectField("Filter");
                try jws.beginObject();
                for (val) |filter| {
                    assert(filter != .@"and");
                    try filter.printJsonField(jws);
                }
                try jws.endObject();
            },
            else => try self.printJsonField(jws),
        }
        try jws.endObject();
    }

    fn printJsonField(self: Filter, jws: anytype) !void {
        switch (self) {
            .prefix => |val| {
                try jws.objectField("Prefix");
                try jws.write(val);
            },
            .tag => |val| {
                try jws.objectField("Tag");
                try jws.write(val);
            },
            .object_size_greater_than => |int| {
                try jws.objectField("ObjectSizeGreaterThan");
                try jws.write(int);
            },
            .object_size_less_than => |int| {
                try jws.objectField("ObjectSizeLessThan");
                try jws.write(int);
            },
            else => unreachable,
        }
    }
};

This code does not handle the situation where @“and” is used inside the @“and” slice:

const breaks = Filter{
    .@"and" = &.{
        .{ .prefix = "prefix" },
        .{ .@"and" = &.{ .{ .tag = "tag" }, .{ .object_size_less_than = 15 } } },
    },
};

try breaks.jsonStringify(jws); // Silently ignores .tag and .object_size_less_than

Possible solutions:

1. Encode the structure in the type system

This introduces complexity and makes the memory layout worse (two discriminated unions):

const SingleFilter = union(enum) {
    prefix: []const u8,
    tag: []const u8,
    object_size_greater_than: usize,
    object_size_less_than: usize,
};

const Filter = union(enum) {
    single: SingleFilter,
    @"and": []const SingleFilter,
};

2. Encode the and as the base state of the filter:

This is I think the correct approach because it removes complexity, and a single filter is just an @“and” filter of length one. This does not then work with @“or” if you want to introduce that later.

const SingleFilter = union(enum) {
    prefix: []const u8,
    tag: []const u8,
    object_size_greater_than: usize,
    object_size_less_than: usize,
};

const Filter = struct {
    items: []const SingleFilter,
};

const single_filter = Filter{ .items = &.{ .{ .prefix = "prefix" } }};

3. Make the recursion top-level:

To allow for nested and structures (that your type allows), you can just remove the helper.

    pub fn jsonStringify(self: *const Filter, jws: anytype) !void {
        try jws.beginObject();
        switch (self.*) {
           .prefix => |val| {
                try jws.objectField("Prefix");
                try jws.write(val);
            },
            .tag => |val| {
                try jws.objectField("Tag");
                try jws.write(val);
            },
            .object_size_greater_than => |int| {
                try jws.objectField("ObjectSizeGreaterThan");
                try jws.write(int);
            },
            .object_size_less_than => |int| {
                try jws.objectField("ObjectSizeLessThan");
                try jws.write(int);
            },
            .@"and" => |val| {
                try jws.objectField("And");
                try jws.beginArray();
                for (val) |filter| {
                    try filter.printStringify(jws); // Now the nested .@"and" structure gets serialized as a nested and structure.
                }
                try jws.endArray();
            },
            else => try self.printJsonField(jws),
        }
        try jws.endObject();
    }

4. Disallow nested .@“and” structures, and introduce type invariance:

  • The else clause should not be {} but unreachable:
    fn printJsonField(self: Filter, jws: anytype) !void {
        switch (self) {
            .prefix => |val| {
                try jws.objectField("Prefix");
                try jws.write(val);
            },
            .tag => |val| {
                try jws.objectField("Tag");
                try jws.write(val);
            },
            .object_size_greater_than => |int| {
                try jws.objectField("ObjectSizeGreaterThan");
                try jws.write(int);
            },
            .object_size_less_than => |int| {
                try jws.objectField("ObjectSizeLessThan");
                try jws.write(int);
            },
            else => unreachable, // or return error.NestedAndFilter
        }
    }
  • Introduce assert() to enforce the type invariance:
const std = @import("std");
const assert = std.debug.assert;

const Filter = union(enum) {
    // ... fields as before


    pub fn jsonStringify(self: *const Filter, jws: anytype) !void {
        try jws.beginObject();
        switch (self.*) {
            .@"and" => |val| {
                try jws.objectField("Filter");
                try jws.beginObject();
                for (val) |filter| {
                    assert(filter != .@"and"); // Make sure the type holds invariance
                    try filter.printJsonField(jws);
                }
                try jws.endObject();
            },
            else => try self.printJsonField(jws),
        }
        try jws.endObject();
    }
};

You would pair the assert() with unreachable, (for type invariance), or return error if you want the Filter state to be a valid state handled as a runtime error if you try to serialize it.

You would also put these asserts in other methods your type provides (where applicable). It’s a bit less robust but it is the minimum you need to change if you want to keep your mental model and layout.

If you only ever need @“and”, I would go with option 2. If you plan to add @“or”, I would go with option 4. If you want to allow nested @“and” structures, option 3 is the only solution for the serialization.

1 Like

Thanks for the enhanced response. Definitely some code snippets which help my learning process.

Of course, its your case 4, as I stated above:

The code is for parsing xml texts which I receive as http response from a server application. The server will never return a nested and structure. However, as you stated, my code simply ignores this case with just {}. The unreachable makes it more clear. Will update the code.

Just a recommendation, don’t shy away from introducing redundant asserts. Even if you ensure the input can not be in the incorrect state, adding the assert everywhere where you iterate the and filter makes the code more robust in isolation (for example with testing).

1 Like