Defining bit-packed binary protocol schemas as types

Help
1

I have some clients that send bit-packed binary data to me (the server).

Each client has a configuration that defines everything I need to know about the client.

How can I deserialize data from these clients given that a little endian packed struct will perfectly describe the layout of the data?

Requirements:

  1. This is for a library, and the user must be able to define their own packed struct data layouts (vtable required?)?
  2. There are multiple kinds of clients, identified by their model number. Clients of the same kind have the same response schema.
  3. I have an array of clients, where each client is initialized using the client configuration.
// this will never work because I cant have runtime type information?
pub const ClientConfiguration = struct {
    model_number: u32,
    response_size_bits: u16,
    response_type: type,
};

const client_a_config = ClientConfiguration{
    .model_number = 45134543,
    .response_size_bits = 16,
    .response_type = packed struct(u16) {
        temperature: u16,
    },
};

clients[34].getResponse()
2

I think I would handle everything as an opaque blob of bytes and then you could have a generic getResponseBytes method for users who just want to deal with the bytes directly and for convenience you could also provide a getResponse(comptime T:type) T method that returns the bytes as the type T (for that type T you would require that it has a known memory layout).

So it is the user that has to know and select the appropriate type, based on whatever field/meta data they are holding on to. This gives you flexibility and avoids having to encode complicated runtime type information schemas into a thing that also deals with transporting the raw data.
Some other thing can deal with the mapping of meta data to type related functionality.

For that mapping type-info thing, if it doesn’t change at runtime I would start with a tagged union for all the supported types.
You map from the meta data to a thing that gives you the operations you need to do on that type, if you have a tagged union (because your set of types is closed and known at comptime) the mapping could just be a switch over the tagged union.

If it is more dynamic you may need to map the meta data to a struct that contains function pointers (basically creating vtables).

I think the easiest thing to do would be to make the type related part be the users problem, I think putting that in the library will make it awkward, but you could provide helpers that are for specific usage scenarios, for example when using a tagged union vs something that is completely dynamic at run time.

Basically the core that communicates shouldn’t care about the type and only about the bytes and the other part needs to care mostly about the operations that need to happen, those operations depend on the type, but it is baked into these operations, so you wouldn’t store the type but instead the operations. Or in the case of being able to use a tagged union you can store the tag of the tagged union.

1 Like