How should I implement a payload abstraction for a server

Help
,
1

Hello!

I’m learning zig by building a server that implements a binary protocol and I would like some thoughts on how I should design the payload abstraction.

To summarize, I need:

  1. structs that represent data that will be sent or received.
  2. functions to serialize and unserialize these structs, as well as some auxiliary ones.
  3. an association of struct type to an opcode.

So, my first attempt to implement this (which I’m very discomfortable about) consists of having an enumerated union where each field is a payload type.

Here is a skeleton of how I made it:

pub const Payload = union(enum(u16)) {
    PayloadType1: struct {
        f1: u16,
        f2: u32,
        // ...
    } = 1,

    PayloadType2: struct {
        f1: f32,
        f2: [8]u8,
        // ...
    } = 5,
    // ... 

    pub fn serialize(self: Payload, allocator: mem.Allocator) ![]u8 {
        switch (self) {
            inline else => |payload| {
                /// The actual implementation here
        }
    }

and it’s used like that:

const payload = Payload{.PayloadType1 = .{
    .f1 = somethig,
    .f2 = something_else,
}};
const serialized_payload = try payload.serialize(allocator);
defer allocator.free(serialized_payload);
send(p); // Just a simplification here

But I’m not really convinced that this is the best way to do that, as there’s some things that are bothering me:

  1. I have to do a static dispatch to do everything I need about the payload. i.e. every method I put behind Payload will need to have that same pattern, which costs 2 indentation blocks for the function logic (or making these public methods just a wrapper, but I don’t like it that much).
  2. I’m not really using any union feature other than the opcode to type association thing.

Given that, I would love to hear suggestions about how I can improve the abstraction of this code.

Thanks!

1 Like
2

Welcome to the forum. Good question. I found that utilizing generic parameter in function and having common fields and common methods in struct types can go pretty far.

The following is a sample pattern using generic type in function parameter for the payload types. You can easily add a new payload type as long as the common field and the two common methods are added for the payload.

const std = @import("std");
const Allocator = std.mem.Allocator;

const OpCode = enum { p1, p2 };

const PayloadType1 = struct {
    op: OpCode = .p1,
    f1: u16,
    f2: u32,

    pub fn serialize(self: *const @This(), writer: *std.Io.Writer) !void {
        try writer.print("{} (f1={}, f2={})", .{self.op, self.f1, self.f2});
    }

    pub fn deserialize(self: *@This(), alloc: Allocator, reader: *std.Io.Reader) !void {
        _=self; _=alloc; _=reader;
    }

    // ... other payload specific methods.
};

const PayloadType2 = struct {
    op: OpCode = .p2,
    f1: f32,
    f2: []const u8,

    pub fn serialize(self: *const @This(), writer: *std.Io.Writer) !void {
        try writer.print("{} (f1={}, f2={s})", .{self.op, self.f1, self.f2});
    }

    pub fn deserialize(self: *@This(), alloc: Allocator, reader: *std.Io.Reader) !void {
        _=self; _=alloc; _=reader;
    }
};


// Access to the common fields or functions of the payloads.
fn getOpCode(payload: anytype) OpCode {
    return payload.op;
}

fn serialize(payload: anytype, writer: *std.Io.Writer) !void {
    try payload.serialize(writer);
}

fn deserialize(payload: anytype, alloc: Allocator, reader: *std.Io.Reader) !void {
    try payload.deserialize(alloc, reader);
}


test {
    var gpa = std.heap.DebugAllocator(.{}){};
    defer _ = gpa.deinit();
    const alloc = gpa.allocator();

    {
        const p1 = PayloadType1 { .f1 = 1, .f2 = 2 };
        std.debug.print("opcode: {}\n", .{getOpCode(p1)});

        var out_buf = std.Io.Writer.Allocating.init(alloc);
        defer out_buf.deinit();
        try serialize(p1, &out_buf.writer);
        std.debug.print("{s}\n", .{out_buf.written()});

        var in_buf = std.Io.Reader.fixed(out_buf.written());
        var p1a: PayloadType1 = undefined;
        try deserialize(&p1a, alloc, &in_buf);
    }

    {
        const p2 = PayloadType2 { .f1 = 1, .f2 = "abc" };
        std.debug.print("opcode: {}\n", .{getOpCode(p2)});

        var out_buf = std.Io.Writer.Allocating.init(alloc);
        defer out_buf.deinit();
        try serialize(p2, &out_buf.writer);
        std.debug.print("{s}\n", .{out_buf.written()});

        var in_buf = std.Io.Reader.fixed(out_buf.written());
        var p2a: PayloadType2 = undefined;
        try deserialize(&p2a, alloc, &in_buf);
    }
}
2 Likes
3

what specifically don’t you like? indentation blocks are not a useful metric at all here.

that is literally the only feature of a tagged union.

For deserialisation a tagged union is great here, it is a type safe way to be agnostic of which particular message is actually received.
And for serialisation, reusing that tagged union is good to reduce type duplication and mental overhead. You can definitely separate the inner types if you prefer.

To provide anything more, I would need to know what this protocol is and what you are using it for.