Proper way to cast comptime data to `anyopaque`

Help
1

std.builtin.Type.StructField has the default_value field which is of type ?*const anyopaque. I attempted to write a simple convenience function for helping me build these:

fn buildfield(comptime name: [:0]const u8, comptime typ: type, comptime opts: anytype) Type.StructField {
    comptime var default_value: ?*const anyopaque = null;
    comptime var is_comptime: bool = false;
    comptime var alignment: comptime_int = @alignOf(typ);
    inline for (meta.fields(@TypeOf(opts))) |opt| {
        if (mem.eql(u8, opt.name, "default_value")) {
            default_value = @ptrCast(&@field(opts, opt.name));
        } else if (mem.eql(u8, opt.name, "is_comptime")) {
            is_comptime = @field(opts, opt.name);
        } else if (mem.eql(u8, opt.name, "alignment")) {
            alignment = @field(opts, opt.name);
        } else {
            @compileError("got bogus argument for buildfield option");
        }
    }
    return Type.StructField{
        .name = name,
        .type = typ,
        .default_value = default_value,
        .is_comptime = is_comptime,
        .alignment = alignment,
    };
}

This was my best guess at how I should be casting the default_value. This code may even work in some circumstances, however I suspect something is amiss. When I do, for example

const f = buildfield("testfield", i64, .{ .default_value = 1 });

and try to do anything with the result, I get

src/root.zig:62:33: error: runtime value contains reference to comptime var
    std.debug.print("{any}\n", .{f});
                               ~^~~
src/root.zig:62:33: note: comptime var pointers are not available at runtime

Clearly something is happening in my coercion of the type that it thinks may not be knowable at compile time, but I’m unsure how to resolve this. Any suggestions? Thanks!

Update: Expanding on this a bit, I seem to be having a broader problem in not understanding what happens once I start needing pointers to comptime values. Continuing with this example, my next convenience function

fn buildstruct(comptime flds: anytype) Type.Struct {
    const decls: [0]Type.Declaration = .{};
    return Type.Struct{
        .layout = Type.ContainerLayout.auto,
        .backing_integer = null, // I have no idea what this is
        .fields = &flds,
        .decls = &decls,
        .is_tuple = false,
    };
}

(the compiler already suggested that I use the reference operator to coerce these) gives the error

/usr/lib/zig/std/fmt.zig:640:22: error: values of type '[]const builtin.Type.StructField' must be comptime-known, but index value is runtime-known
                for (value, 0..) |elem, i| {
                     ^~~~~
/usr/lib/zig/std/builtin.zig:346:15: note: struct requires comptime because of this field
        type: type,
              ^~~~
/usr/lib/zig/std/builtin.zig:346:15: note: types are not available at runtime
        type: type,
              ^~~~
/usr/lib/zig/std/builtin.zig:349:20: note: struct requires comptime because of this field
        alignment: comptime_int,
                   ^~~~~~~~~~~~
referenced by:
    formatType__anon_8517: /usr/lib/zig/std/fmt.zig:608:31
    format__anon_4842: /usr/lib/zig/std/fmt.zig:185:23
    remaining reference traces hidden; use '-freference-trace' to see all reference traces
1 Like
2

Hi @ExpandingMan welcome to Ziggit!

You can assign your var to a const and then use that const in the return value instead. That way the return value doesn’t directly refer to a comptime var anymore and is seen as static data that doesn’t refer back to comptime.
For a full explanation read this:

So I think this should work:

fn buildfield(comptime name: [:0]const u8, comptime typ: type, comptime opts: anytype) Type.StructField {
    comptime var default_value: ?*const anyopaque = null;
    comptime var is_comptime: bool = false;
    comptime var alignment: comptime_int = @alignOf(typ);
    inline for (meta.fields(@TypeOf(opts))) |opt| {
        if (mem.eql(u8, opt.name, "default_value")) {
            default_value = @ptrCast(&@field(opts, opt.name));
        } else if (mem.eql(u8, opt.name, "is_comptime")) {
            is_comptime = @field(opts, opt.name);
        } else if (mem.eql(u8, opt.name, "alignment")) {
            alignment = @field(opts, opt.name);
        } else {
            @compileError("got bogus argument for buildfield option");
        }
    }
    const frozen_default_value = default_value;
    const frozen_is_comptime = is_comptime;
    const frozen_alignment = alignment;
    return Type.StructField{
        .name = name,
        .type = typ,
        .default_value = frozen_default_value,
        .is_comptime = frozen_is_comptime,
        .alignment = frozen_alignment,
    };
}
3 Likes
3

Thanks, I think that makes sense. Your code did not work as written though (same error). Here’s what I came up with based on your advice that seems to work.

fn buildfield(comptime name: [:0]const u8, comptime typ: type, comptime opts: anytype) Type.StructField {
    comptime var default_value: ?typ = null;
    comptime var is_comptime: bool = false;
    comptime var alignment: comptime_int = @alignOf(typ);
    inline for (meta.fields(@TypeOf(opts))) |opt| {
        if (mem.eql(u8, opt.name, "default_value")) {
            default_value = @field(opts, opt.name);
        } else if (mem.eql(u8, opt.name, "is_comptime")) {
            is_comptime = @field(opts, opt.name);
        } else if (mem.eql(u8, opt.name, "alignment")) {
            alignment = @field(opts, opt.name);
        } else {
            @compileError("got bogus argument for buildfield option");
        }
    }
    var defval: ?*const anyopaque = null;
    if (default_value) |v| {
        defval = &v;
    }
    return Type.StructField{
        .name = name,
        .type = typ,
        .default_value = defval,
        .is_comptime = is_comptime,
        .alignment = alignment,
    };
}

I think the issue was that one cannot coerce the pointer itself into a const, it has to happen to the underlying value, then one can take the pointer.

I think I’m still not entirely understanding the boundary between comptime and runtime values here. Does the code I wrote above make it impossible for the default_value field to be known at compile time? In principle the entire output of this function should be knowable at compile time, I’m not sure I understand what things can happen to cause that to fail.

Update: Something else bad is happening here. With this code I do

const f2 = buildfield("fname2", f32, .{ .default_value = 1.0, .is_comptime = true });
const defval: ?*const f32 = @ptrCast(&f2.default_value);
try testing.expect(defval.?.* == 1.0);

and I find defval has a bogus value. Not sure if this is something wrong with buildfield or if I just don’t understand how to case back out of anyopaque.

1 Like
4

This was sneaky, took me a while to spot it, you are taking the address of the pointer, creating a *?*anyopaque pointer to the pointer and then casting that to a ?*const f32 pointer, effectively reinterpreting the default_value field which is a optional pointer as a f32.

This is it fixed:

const defval: ?*const f32 = @ptrCast(@alignCast(f2.default_value));
5

Yes, always better to actually run the code, instead of trying to simulate it in your head…

On Zig 0.13.0 this works:

fn buildfield(comptime name: [:0]const u8, comptime typ: type, comptime opts: anytype) Type.StructField {
    comptime var default_value: ?*const anyopaque = null;
    comptime var is_comptime: bool = false;
    comptime var alignment: comptime_int = @alignOf(typ);
    inline for (meta.fields(@TypeOf(opts))) |opt| {
        if (mem.eql(u8, opt.name, "default_value")) {
            default_value = @ptrCast(&@as(typ, @field(opts, opt.name)));
        } else if (mem.eql(u8, opt.name, "is_comptime")) {
            is_comptime = @field(opts, opt.name);
        } else if (mem.eql(u8, opt.name, "alignment")) {
            alignment = @field(opts, opt.name);
        } else {
            @compileError("got bogus argument for buildfield option");
        }
    }
    return Type.StructField{
        .name = name,
        .type = typ,
        .default_value = default_value,
        .is_comptime = is_comptime,
        .alignment = alignment,
    };
}

The only change from your original is:

default_value = @ptrCast(&@as(typ, @field(opts, opt.name)));

The added @as(typ, ...) is to make sure that the pointer stored in the default value has the correct type, without that I got this error:

comptimedata.zig:96:32: error: comptime dereference requires 'comptime_float' to have a well-defined layout
    try testing.expect(defval.?.* == 1.0);
                       ~~~~~~~~^~

This happens because the 1.0 literal from the options has type comptime_float without the conversion to typ.

It is hard to test this completely, because I don’t have the full context of how you actually use the buildfield function and I don’t really want to spend extra time, speculating on how you may use it, so it is better if you provide that as a example.

This is just advice that may or may not apply:
Personally I don’t really bother creating functions like buildfield most of the time, I think most of the time it is better to write the code creating all the fields at once just inline within the for loop that creates all the fields, instead of separating the code into tiny functions.

I imagine there may be situations where it makes sense, but I would try to avoid multiple functions and instead create a single bigger functions if that is possible, because IMHO it is better to be able to look in one place at a big switch statement then having to look at many tiny functions (that invent abstractions that aren’t really needed (most of the time)).

2 Likes
6

Use an array with a precomputed length for value instead of a slice.

Something similar to this:

const SoaMode = enum { read, write, component_id };
fn SoaStruct(comptime Struct: type, mode: SoaMode) type {
    const fields = std.meta.fields(Struct);
    var array_fields: [fields.len]std.builtin.Type.StructField = undefined;
    for (fields, 0..) |f, i| {
        const T = switch (mode) {
            .read => []const f.type,
            .write => []f.type,
            .component_id => ComponentId,
        };
        array_fields[i] = .{
            .name = f.name,
            .type = T,
            .default_value = null,
            .alignment = @alignOf(T),
            .is_comptime = false,
        };
    }
    return @Type(.{
        .Struct = .{
            .layout = .auto,
            .fields = &array_fields,
            .decls = &.{},
            .is_tuple = false,
        },
    });
}

I think writing a bunch of helper functions for this sort of code just creates a situation where you invent your own DSL for this stuff, making it harder for everyone else to read the code, when I look at code that others have written, I want to see the implementation, not 3 layers of helper functions that need to be reverse engineered to discover the original code, so it is better to have a little bit of repetition, than having these extra layers that add cognitive burden for the person that needs to read the code.

zig zen:

  • Favor reading code over writing code.
1 Like
7

I don’t yet have a strong opinion of whether or not I’ll need those kinds of helper functions, part of the purpose of this exercise is just to try to better understand the type system.

That said, I am getting the sense that there is a lot of potential to generate a tremendous amount of boilerplate with anything more than very simple types. Most of the examples of zig code I have seen (the main source of which is here) doesn’t do much in the way of generating generics beyond simple templating, even though the language itself seems expressive enough to support an arbitrarily complex type system, so I’m still contemplating what that might look like.