I’ve got three implementations of a serializer and deserializer for a struct. Which do you think is best and why?
Supports allocation, or no allocation
const std = @import("std");
pub const Packet = struct {
header: Header,
data: []u8,
pub const Header = packed struct(u16) {
length: u11,
reserved: u5 = 0,
};
/// asserts data.len < max u11
/// data must have lifetime beyond returned memory
pub fn init(data: []u8) Packet {
return Packet{
.header = .{ .length = @intCast(data.len) },
.data = data,
};
}
/// asserts data.len < max u11
/// no lifetime requirement on data parameter
/// free returned memory with deinit
pub fn initAlloc(allocator: std.mem.Allocator, data: []u8) !Packet {
return Packet{
.header = .{ .length = @intCast(data.len) },
.data = try allocator.dupe(u8, data),
};
}
pub fn serialize(self: Packet, writer: *std.Io.Writer) !Packet {
try writer.writeStruct(self.header, .little);
try writer.writeAll(self.data);
}
/// buf must have lifetime past returned memory
pub fn deserializeNoAlloc(buf: []const u8) !Packet {
var reader = std.Io.Reader.fixed(buf);
const header = try reader.takeStruct(Header, .little);
const data: []u8 = try reader.take(header.length);
return .{ .header = header, .data = data };
}
/// free using deinit
pub fn deserialzeAlloc(allocator: std.mem.Allocator, reader: *std.Io.Reader) !Packet {
const header = try reader.takeStruct(Header, .little);
const data = try reader.readAlloc(allocator, header.len);
return Packet{ .header = header, .data = data };
}
/// only required if you use deserializeAlloc
pub fn deinit(self: *Packet, allocator: std.mem.Allocator) void {
allocator.free(self.data);
}
};
Notes:
- most complex
- lifetimes are harder to explain / requires more documentation
Supports no allocation
pub const Packet = struct {
header: Header,
data: []u8,
pub const Header = packed struct(u16) {
length: u11,
reserved: u5 = 0,
};
/// asserts data.len < max u11
/// data must have lifetime beyond returned memory
pub fn init(data: []u8) Packet {
return Packet{
.header = .{ .length = @intCast(data.len) },
.data = data,
};
}
pub fn serialize(self: Packet, writer: *std.Io.Writer) !Packet {
try writer.writeStruct(self.header, .little);
try writer.writeAll(self.data);
}
/// buf must have lifetime past returned memory
pub fn deserializeNoAlloc(buf: []const u8) !Packet {
var reader = std.Io.Reader.fixed(buf);
const header = try reader.takeStruct(Header, .little);
const data: []u8 = try reader.take(header.length);
return .{ .header = header, .data = data };
}
};
Notes:
- simpler, less code
- lifetimes might be a bit sketchy
supports allocation
pub const Packet = struct {
header: Header,
data: []u8,
pub const Header = packed struct(u16) {
length: u11,
reserved: u5 = 0,
};
/// asserts data.len < max u11
/// no lifetime requirement on data parameter
/// free returned memory with deinit
pub fn initAlloc(allocator: std.mem.Allocator, data: []u8) !Packet {
return Packet{
.header = .{ .length = @intCast(data.len) },
.data = try allocator.dupe(u8, data),
};
}
pub fn serialize(self: Packet, writer: *std.Io.Writer) !Packet {
try writer.writeStruct(self.header, .little);
try writer.writeAll(self.data);
}
/// free using deinit
pub fn deserializeAlloc(allocator: std.mem.Allocator, reader: *std.Io.Reader) !Packet {
const header = try reader.takeStruct(Header, .little);
const data = try reader.readAlloc(allocator, header.len);
return Packet{ .header = header, .data = data };
}
/// only required if you use deserializeAlloc
pub fn deinit(self: *Packet, allocator: std.mem.Allocator) void {
allocator.free(self.data);
}
};
Notes:
- requires allocator
- lifetimes are clearer (because allocator screams lifetime stuff at me)
What I hate about allocation is that is introduces the possibility of error.OutOfMemory. Generally, I prefer to not use any allocation if possible, so I typically use the “no allocation” approach. Sometime going to great lengths to do so. This I feel helps me better manage error explosion.
Some may say to “just use fixed buffer allocator”. I would respond that, practically, I want infailable deserialization. Or extremely limited failures. The deserializeNoAlloc method can produce only one error: error.ReadFailed. It cannot produce OutOfMemory. If I attempt to use fixedBufferAllocator, I need to guess the size by reading the code in deserializeAlloc. Sometimes this is not practical, there can be many branches in the deserializer. Also, you must fully understand the behavior of the allocator under possible fragmentation, you must know how efficient your allocator is, which can also be difficult. Does the API contract for fixedBufferAllocator include how efficient it is? IDK.
What do you think is the most maintainable, robust implementation? Which would you prefer to come back to and maintain ~5 years from now? Which provides the best balance of complexity / error proneness?
Edit: actually, the take I am using is unsafe under untrusted input because it asserts the size of the buffer has at least header.length. That’s annoying, anyone have a fix for that?