Struct Padding Syntax - Would this work?

Brainstorming
1

Most of us already seen classic reserved0, reverved1, reserved23 or even unknown1 etc

The thing is, that in many scenarios when it comes to extern structs with C-ABI interop we do not implement the struct fully or in some cases we can’t fully implement the struct, so at the end we end-up with struct full of reserved or unknown fields that has numerical sufix, since we already have a discard syntax and we also do have non-exhaustive enum, why not adding padding syntax
to structs, might even work pretty well with bitpacked struct where we can force the packed struct to fill up the backing integer explicitely, anyway

Idea is simple, any unknown or reserved0 field would became general padding using underscore syntax

pub const Interop = extern struct {
      _: *anyopaque, // It might look like a pointer but we don't know what is supposed to point at
      my_well_known_field: SomeType,
      _: [24]u8,     // Let's say we also know there is an offset of 24 unknown bytes, se we can use this padding again
      my_other_well_know: BestType,
};

Since i quote:

why not adding padding syntax to structs

Here are some issues that come with it, and would need proper investigation and discussion about

  • my_struct._ = 5 This is more of personal subjective opionion, but accessing padding would be strictly disallowed as it doesn’t specify what padding it refers to, (What if i want to use the padding value? Then why did u use padding in the first place duh? Also you can always cast the pointer type and access what you want)

  • @typeInfo(Interop).@"struct".fields Would now return slice where padding field name might and might not repeat multiple times, in this case not a big deal. Most of the comptime accessing this information does some kind of serialization, and that not the case for majority of interop structs that are passed between different systems.
    (PS: After more thoughts, i realized even simple formatting in std.debug.print wouldn’t work here properly and would have to skip these padding fields, but also doesn’t seems to be a big issue)

  • my_struct = .{._ = 5} Would also be disallowed, in most cases you do not initialize structure you don’t fully know, so compiler can just report and disallow initialization for this structs (my_struct: MyStruct = undefined would be still allowed)

Tell me what do you think, also compiler would have to emit some unique symbols for each padding, but shouldn’t be big of an issue

Well my point is, lets discuss it rather sooner than later, that’s also reason i choose to bring this up, i seen a GitHub proposal issue where even this kind of proposal was slightly mentioned, but it main topic was discarded params proposal fn tadaa(_: type) and that end up being in lang and then the issue was closed.

So even if its good or bad idea, we can argue about that here now

I didn’t want to open issue on codeberg as this is more controversial and theoretical and myself is not sure if its the right way to go. That’s also why i want to hear your takes, Thanks

7 Likes
2

One thing I would like to mention is that underscore “_” in structs, is valid field name at the moment and i think it shouldn’t be in general

So this is valid syntax btw

my_struct._ = 5;
my_struct = .{._ = 5;}

I would rather explicitly protect the language from this edge case, just by forcing ppl to rename or use raw literal

my_struct.@"_" = 5;
7 Likes
3

I’m gonna be honest, I’m actually pretty surprised this isn’t in the language already as I’ve actually already used “_” as padding in packed structs. Guess I’ve never run into more than one padding block I had to take care of before.

So, very much in favor of this. At the very least to disallow “_” as a raw name, too weird it’s allowed given how it’s used elsewhere.

Regarding the issues, only #2 feels like a real issue, as the other two are only about accessing the padding, which shouldn’t be done anyway. My proposal for solving it would be (keeping in mind that typeinfos will be changed to a struct of arrays format): just omit the padding fields. As with the other two issues, there’s no good reason to be accessing padding (and if there is you should be casting the struct into one with more defined fields).

The only issue with that would be recreating the same struct with @Struct(), but I reckon that adding another field into struct’s typeinfo should be an acceptable solution, though it does feel like it’d probably be unergonomic.
What would its type be? []usize describing the length of padding between every pair of fields plus before the first and after the last field? [][2]usize, an index and a padding length? Would be more a lot better for the case with no padding at least.

1 Like
4

I would love to see this since in the repo I am currently working at, some of the packets have crazy reserved fields…

pub const MtF_Identity = packed struct {
    sw_rev_byte4: u7 = 0,
    _p0: u1 = 0,
    sw_rev_byte3: u7 = 0,
    _p1: u1 = 0,
    sw_rev_byte2: u7 = 0,
    _p2: u1 = 0,
    sw_rev_byte1: u7 = 0,
    _p3: u1 = 0,
    device_family_model_msb: u7 = 0,
    _p4: u1 = 0,
    device_family_model_lsb: u7 = 0,
    _p5: u1 = 0,
    device_family_msb: u7 = 0,
    _p6: u1 = 0,
    device_family_lsb: u7 = 0,
    _p7: u1 = 0,
    manufacture_byte3: u7 = 0,
    _p8: u1 = 0,
    manufacture_byte2: u7 = 0,
    _p9: u1 = 0,
    manufacture_byte1: u7 = 0,
    _pA: u25 = 0,
    status: u10 = 2,
    form: u2 = 0,
    mt: u4 = 0xF,
};

Since they won’t be used in a common scenario, while we don’t care about the differences between the padding beside its length, it would be handy if it have a generic padding syntax for filling the spaces. If people want to access these padding, they should use @bitCast or packed union to transform the type.

2 Likes
5

I like the idea of using “_” to signify that a field is just padding however I think it would be kind of odd to make the name inform how the compiler should treat it. Maybe instead it could be a type? This would allow the field to work with preexisting meta programming that iterates over fields while still being informative that it is just padding. However that solution wouldn’t fix the annoyance of having to name every piece of padding.

7

I think this might work considering how align() works. We may use something like padding() to specify the amount of padding after the field. For example, instead of writing this:

const Something = packed struct {
  a: u6 = 0,
  _p0: u2 = 0,
  b: u5 = 0,
  _p1: u3 = 0,
}

We could replace the padding with the padding() type, to state that the field will have n number of prepending 0s:

const Something = packed struct {
  a: u6 padding(2) = 0, // becomes ppAAAAAA
  b: u5 padding(3) = 0, // becomes pppBBBBB
}
// byte prepresentation after @bitCast: pppBBBBppAAAAAA

We could expend the use case with align() too, and placing it before and after will have different results:

const Something = packed struct {
  a: u6 align(1) padding(2) = 0, // alignment of 1 + 2bits
  b: u6 padding(2) align(1) = 0, // 6bits + 2bits, within alignment of 1
}

Seems like this solution could completely eliminate the padding fields while it is also clear enough to reveal how padding separate each fields. Though, not sure how effective this solution in practice.

8

If they were willing to edit the language, they could take inspiration from 010 Editor or ImHex, what with explicit padding[N] interleaved into the struct.

9

I think making padding an attribute introduces more considerations.

  • Your example makes the order of attributes matter which may not be desirable. The * FieldAttributes of a struct’s field would need to specify if the padding is before or after, or maybe there can be padding both before and after so there are two fields that specify padding in FieldAttributes.
  • If you take the @sizeOf() a field’s type it no longer accurately describes how much space it takes up in a bitpack struct which is problematic. A solution to this would probably be some function in std.meta that needs to know both the bitpack struct’s type and the name of the field.
  • As OP pointed out in the original post, some padding is a temporary limitation in the Zig implementation of a C struct meaning that the padding will be used eventually. Situations like these where padding is just yet to be used happens in real standards too. For example the WebSocket protocol has 2 bits in the header reserved for future use. What I am getting at is that being able to name padding is sometimes useful.
  • Maybe my previous point is wrong if you can pad void like so pad_for_now: void padding(3). However if this is the solution to my gripe then I would argue this is no different than padding just being a type.

Here is what I would imagine a WebSocket frame would look like:

const frame = bitpack strcut {
    fin: bool,
    rsv1: bool,
    rsv2: padding(1),
    rsv3: padding(1),
    opcode: opcodeEnum,
    ...
}

I think this makes it more clear at a glance that rsv2 and rsv3 may potentially be used.

I also think making padding a type rather than an attribute makes more sense when entire bytes are padding.

An example I can think of that would benefit from this are Linux CAN frames.

Original C code taken from linux/can.h (minus some comments and I added some easier to understand types):

struct can_frame {
        __u32 can_id;
        __u8 len;
        __u8 __pad; /* padding */
        __u8 __res0; /* reserved / padding */
        __u8 len8_dlc;
        __u8 data[8] __attribute__((aligned(8)));
};

Zig implementation where padding is a type:

pub const CanFrame = extern struct {
    can_id: u32,
    len: u8,
    pad: padding(8), // padding
    res0: padding(8), // reserved / padding
    len8_dlc: u8,
    data: [8]u8 align(8),
}

Zig implementation where padding is an attribute:

pub const CanFrame = extern struct {
    can_id: u32,
    len: u8 align(1) padding(8),
    res0: void padding(8),
    len8_dlc: u8,
    data: [8]u8 align(8),
}

Or maybe res0 should be padding before len8_dlc or len could have 16 bits of padding after. Having multiple padding options like this breaks the “only one obvious way to do things” philosophy IMO.

I view padding as a meaningful part of a struct’s layout so I would prefer it be an entire field however I understand how that may look ugly at times. I am also concerned that making padding something other than just fields will complicate meta programming a lot when you need to be cognizant of layout and size.

10

This can be fairly trivially achieved in user-space if needed, demo:

const std = @import("std");

/// Construct a struct given the input field names and types
/// Handles a `null` name as padding and anonomises and 0-initialises it
pub fn PackedStructPadding(comptime fields: []const struct { ?[]const u8, type }) type {
    var pad_idx: usize = 0;
    var names: [fields.len][]const u8 = undefined;
    var types: [fields.len]type = undefined;
    var attrs: [fields.len]std.lang.Type.Struct.FieldAttributes = @splat(.{});
    for (fields, 0..) |field, i| {
        types[i] = field[1];
        names[i] = if (field[0]) |name| name else b: {
            defer pad_idx += 1;
            attrs[i] = .{ .default_value_ptr = &@as(field[1], 0) };
            break :b std.fmt.comptimePrint("_p{d}", .{pad_idx});
        };
    }
    return @Struct(.@"packed", null, &names, &types, &attrs);
}

test PackedStructPadding {
    const Example = PackedStructPadding(&.{
        .{ "foo", bool },
        .{ "bar", u2 },
        .{ null, u3 },
        .{ "baz", u10 },
    });

    const r: Example = .{ .foo = true, .bar = 2, .baz = 0x155 };
    try std.testing.expectEqual(@as(u16, 0x5545), @as(u16, @bitCast(r)));
}

Might seem like a bit of comptime abuse though, and ZLS doesn’t handle it too well at the moment, but hey it’s something.

11

There’s no way to have decls on such a struct.

12

My favourite way of naming padding fields is with more and more underscores.

const S = packed struct(u16) {
    _: u3,
    foo: u8,
    __: u5,
};

It usually doesn’t get too bad if there aren’t a lot of padding fields.

5 Likes