Proper way to auto serializer/deserializer

Help
1

I have this struct, and also 14 similar:

const Config: Type.Struct = @This();

reg: Register = undefined,

reflect_rx_data_received_on_irq_pin: bool = false,
reflect_tx_data_sent_on_irq_pin: bool = false,
reflect_max_retransmit_on_irq_pin: bool = false,
crc: Crc = .@"1byte",
power: Power = .off,
operational_mode: Mode = .tx,

pub fn init() Config {
    return .{};
}

I have two functions with roles of serializer and deserializer

pub fn fromRegister(self: *Config, opt_data: ?[]const u8) !void {
    if (opt_data == null or opt_data.?.len == 0) {
        return error.NoData;
    }

    const data = opt_data.?;

    self.reflect_rx_data_received_on_irq_pin = (data[0] & (1 << 6)) > 0;
    self.reflect_tx_data_sent_on_irq_pin = (data[0] & (1 << 5)) > 0;
    self.reflect_max_retransmit_on_irq_pin = (data[0] & (1 << 4)) > 0;
    self.crc = try Crc.fromInt((data[0] & (0b11 << 2)) >> 2);
    self.power = try Power.formInt(data[0] & (1 << 1));
    self.operational_mode = try Mode.fromInt(data[0] & 1);
}

pub fn toRegister(self: Config) ?[]const u8 {
    var data: u8 = 0;
    data |= @as(u8, @intFromBool(self.reflect_rx_data_received_on_irq_pin)) << 6;
    data |= @as(u8, @intFromBool(self.reflect_tx_data_sent_on_irq_pin)) << 5;
    data |= @as(u8, @intFromBool(self.reflect_max_retransmit_on_irq_pin)) << 4;
    data |= Crc.toInt(self.crc) << 2;
    data |= Power.toInt(self.power) << 1;
    data |= Mode.toInt(self.operational_mode) << 1;

    return &.{data};
}

I want to replace them with functions that gets Type and some extra prepared data with all shifts etc.
Then fn gets fields, and field by field, writes it to self or read it from self.
Something like this

self.write(@typeInfo(Config).fields[0].name, value);

or

self[@typeInfo(Config).fields[0].name] = value;
2

I’ve found @field works exactly, as I need.
In my case

@field(self, "reflect_rx_data_received_on_irq_pin") = 4;

var x = @field(self, "reflect_rx_data_received_on_irq_pin");
3

I think you’ll like this:

const std = @import("std");
const mem = std.mem;

const Crc = enum(u1) {
    @"1byte",
    @"2byte",
};

const Power = enum(u1) {
    on,
    off,
};

const Mode = enum(u1) {
    rx,
    tx,
};

const Config = packed struct {
    const Integer = u8;
    const len = @sizeOf(Integer);

    reflect_rx_data_received_on_irq_pin: bool = false,
    reflect_tx_data_sent_on_irq_pin: bool = false,
    reflect_max_retransmit_on_irq_pin: bool = false,
    crc: Crc = .@"1byte",
    power: Power = .off,
    operational_mode: Mode = .tx,
    // These are padding because `mem.readInt`
    // and `writeInt` only work with multiples
    // of 8 (u8, u16, u24, u32, etc.)
    flag_1: bool = false,
    flag_2: bool = false,

    /// Read and decode a Config from the start of `buf`.
    pub fn read(buf: []const u8) Config {
        return @bitCast(mem.readInt(Integer, buf[0..len], .big));
    }

    /// Encode and write a Config to the start of `buf`.
    pub fn write(self: Config, buf: []u8) void {
        mem.writeInt(Integer, buf[0..len], @bitCast(self), .big);
    }
};

pub fn main() void {
    const c1 = Config{
        .crc = .@"2byte",
        .power = .on,
        .operational_mode = .rx,
    };
    std.debug.print("\n{any}\n", .{c1});
    var buf: [256]u8 = undefined;
    c1.write(&buf);
    const c2 = Config.read(&buf);
    std.debug.print("\n{any}\n", .{c2});
}

❯ zig build run

main.Config{ .reflect_rx_data_received_on_irq_pin = false, 
.reflect_tx_data_sent_on_irq_pin = false, 
.reflect_max_retransmit_on_irq_pin = false, 
.crc = main.Crc.2byte, .power = main.Power.on, 
.operational_mode = main.Mode.rx, 
.flag_1 = false, .flag_2 = false }

main.Config{ .reflect_rx_data_received_on_irq_pin = false, 
.reflect_tx_data_sent_on_irq_pin = false, 
.reflect_max_retransmit_on_irq_pin = false, 
.crc = main.Crc.2byte, .power = main.Power.on, 
.operational_mode = main.Mode.rx, 
.flag_1 = false, .flag_2 = false

So in summary, the combination of packed struct, mem.readInt, mem.writeInt, and @bitCast is quite powerful for these types of use cases.

4 Likes
4

This is the smart way :slight_smile:
I also have some unused bits, which are hardware level blocked to zero. I don’t know if can implement such struct with packed.
I’ve thought it would be easier to do this with bit shifting manually.

If you know, let’s say i have byte with data of such mask for e.g.
00111011
Where 1 is the only bits with data.
Can this be implemented with packed struct?

EDIT: I know, I can just ignore first “zeroes”, but if its in the middle of data?
And also I have situations where the multi-bit data has separated bits by one or two
for e.g.:

00011111
   ^  ^
   |  |
these coresponds to the same type, but not those between them.
5

I think your solution is proper for those, who care. My task is hardware :brick: specific, there’s no logic on it’s registers. Looks like they’re don’t care at all… :melting_face:

6

Yes, you would just have some bool or u1 fields in those positions that would be ignored, i.e.:

const Signals = packed struct {
    ignore_1: u1 = 0,
    ignore_2: u1 = 0,

    signal_1: bool = false,

    ignore_3: u1 = 0,

    signal_2: bool = false,

    ignore_4: u1 = 0,
};

Note that the choice of bool or u1 should reflect what’s most meaningful for your program, they’ll both take up just 1 bit.

This one is really tough. I think in this case you have to go the manual bit shifting route.

2 Likes
7

I’ve created the solution, which a bit messy, but…

const FieldData = struct {
    bit_index: []const u8,
    type: type,
    rejected_values: ?[]const u8 = null,
};

fn FieldDataStruct(comptime struct_type: type) type {
    const struct_info: Type.Struct = @typeInfo(struct_type);
    comptime var fields: [struct_info.fields.len]Type.StructField = undefined;

    for (struct_info.fields, 0..) |field, i| {
        fields[i] = field;
        fields[i].type = ?FieldData;
        fields[i].default_value = null;
        fields[i].alignment = @alignOf(FieldData);
        fields[i].is_comptime = true;
    }

    return @Type(.{ .Struct = .{
        .layout = .auto,
        .fields = struct_info.fields,
        .decls = &.{},
        .is_tuple = false,
        .backing_integer = null,
    } });
}

const field_data: FieldDataStruct(Config) = .{
    .reflect_rx_data_received_on_irq_pin = .{
        .bit_index = &.{6},
        .field_type = bool,
    },
    .reflect_tx_data_sent_on_irq_pin = .{
        .bit_index = &.{5},
        .field_type = bool,
    },
    .reflect_max_retransmit_on_irq_pin = .{
        .bit_index = &.{4},
        .field_type = bool,
    },
    .crc = .{
        .bit_index = &.{ 3, 2 },
        .field_type = Crc,
    },
    .power = .{
        .bit_index = &.{1},
        .field_type = Power,
    },
    .operational_mode = .{
        .bit_index = &.{0},
        .field_type = Mode,
    },
};

pub fn fromRegister(self: *Config, opt_data: ?[]const u8) !void {
    if (opt_data == null or opt_data.?.len == 0) {
        return error.NoData;
    }

    const data = opt_data.?;

    const type_info: Type.Struct = @typeInfo(@TypeOf(self));
    inline for (type_info.fields) |field| {
        if (std.mem.eql(field.type, *Register)) {
            continue;
        }

        var value: u8 = 0;
        const field_properties: FieldData = @field(field_data, field.name);
        inline for (field_properties.bit_index, 0..) |bit_index, i| {
            const bit_shift = field_properties.bit_index.len - i - 1;
            value |= (data[0] & (1 << bit_index)) >> bit_index << bit_shift;
        }

        if (field.type == bool) {
            @field(self, field.name) = value > 0;
            continue;
        }
        switch (@typeInfo(field.type)) {
            .Struct => {
                @field(self, field.name) = try field_properties.type.fromInt(value);
            },
            else => unreachable,
        }
    }
}

pub fn toRegister(self: Config) ?[]const u8 {
    var data: u8 = 0;

    const type_info: Type.Struct = @typeInfo(@TypeOf(self));
    inline for (type_info.fields) |field| {
        if (std.mem.eql(field.type, *Register)) {
            continue;
        }

        const value: u8 = undefined;
        if (field.type == bool) {
            value = @as(u8, @intFromBool(@field(self, field.name)));
        } else {
            switch (@typeInfo(field.type)) {
                .Struct => {
                    value = field.type.toInt(@field(self, field.name));
                },
                else => unreachable,
            }
        }

        const field_properties: FieldData = @field(field_data, field.name);
        inline for (field_properties.bit_index, 0..) |bit_index, i| {
            const bit_shift = field_properties.bit_index.len - i - 1;
            data |= (value & (1 << bit_shift)) >> bit_shift << bit_index;
        }
    }

    return &.{data};
}

By the documentation, that should work, but haven’t tested yet, since this is a part of my mocking system for library

2 Likes
8

a bit messy

facepalm. ugh. Straight to jail.

2 Likes
9

I’m working on it. Don’t take to heart :smiling_face_with_tear:

10

I meant the bad pun lol. “bit” when you’re talking about bit shifting.

2 Likes
11

ba-dum tssss

5 Likes
12

I didn’t even realized :joy:
EDIT: Today or tomorrow will update the solution, had checked it after some university exams and found some mistakes obvious. Plus there were compilation bug Zig related (found on issues when were searching errors), so have done some rework to fix the code to compile.

13

I’ve came to do not write this garbage. Since I don’t want it in every file, that implements interface, I need to do it like comptime from type. This is too much.
Writing all bit shifting stuff manually is nicer to my. This code had broken my brain already :confused:. I want to cry and laugh at the same time, is this called “schizophrenia” :melting_face:?

14

It’s called real programming! But hey, take it one step at a time. Break it up into the simplest steps possible and tackle each step one at a time. In my case sometimes it helps to open a blank new file and start working from nothing in small chunks.

1 Like
15

I’m re-reading your original post here and I don’t know if I like the strategy.

Here’s the thing, you’re building a generic function that can take a field name and write a value - I understand that you have something like 14 structs to do this with, but I think this could be a case of “generic where specific is needed”.

Let’s take a step back here - even if you don’t want to pursue this path, there’s something to be learned here.

What is the situation/circumstance that you are trying to avoid. Are you attempting to avoid writing 14 setter functions? There’s got to be a better solution here but I need to understand your problem better.

2 Likes
16

I see now, that straight forward way is the best now, since I have little experience in Zig (comptime especially). Later I assume, I’ll rewrite this code to be more adaptive, so I’ll easily add to this lib more hardware.
I’m writing low level library, that operates nrf24l01+ (this is transmitter) registers.
Each of my structs represents each of the nrf24 registers. Each register has it’s own special variables (mostly one byte).
To test it without hardware, I’ve created mock for register, that expects my functions to do some specific changes. The idea I’ve get, is to auto-generate testing cases, and also it creates space to generate bit splitting stuff. The first iteration seems to by work, but it must be placed in every struct. Then making it comptime grew it’s size two times at least, and created even more problems, than it solves.
The answer: I’ve tried to avoid writing bit manipulations by hand, cause I always need to compare this stuff with nrf24’s documentation. And if I’ll do this thing, I need to write bit positions only one time.
My thoughts now: This stuff is complicated, and I’ve chosen wrong angle to start with. I’ve tried to rush forward, but I’ve must prepare and check other sides.
Since now I have less time to do this library (working on my bachelor’s thesis, this library is the part of it), need to temporarily abandon this part with auto-generation of test cases.
Hand written tests are also good :smile:
Thank you all, for responses.
P.S.: In addition to my last post, this stuff isn’t as hard, as annoying.

1 Like
17

Gotcha - when working with auto-generated stuff, I ask myself this question: “did I intend to write auto generated stuff from the start, or am I introducing this now to save some other idea?”

I usually find that generating things only works out if it was apart of the plan to begin with. If I start writing it to make something else work (or I am backed into that corner), it’s usually best to step back and look at the problem again.

Thanks for the clarification, and best of luck with your project :slight_smile:

2 Likes
18

Thank you! :heart_hands: