Why does @field work for declarations?

Help
1

[EDITED to try to improve clarity]

I want a poor-man’s static lookup-table, and thought about implementing it like this:

   const E1 = enum {
      a, b, c,
   };
   const E2 = enum {
      w, x, y, z,
   };
   const SS = struct {
      const a: []const E2 = &.{.x, .z};
      const b: []const E2 = &.{.w, .y, .z};
      const c: []const E2 = &.{.y};
   };
   const z: E1 = .b;
   std.debug.print("SS[z]: {any}\n", .{@field(SS, @tagName(z))});

It “works” fine (output is SS[z]: { .w, .y, .z }, which is what I wanted), but I was a little surprised that @field worked on static consts (not technically struct fields)

  1. can you offer insight into @field?
  2. I wonder if there’s any advice to not do this kind of static lookup-table, like this; or if there’s a better/simpler way, if you can see what I’m trying for here.
1 Like
2

@field is foo.bar but both foo and var can be provided by comptime logic, instead of being literal identifiers in the source.

foo.bar syntax is used both for field, declarations (decls), and “methods” (calling functions on an instance).

It could certainly have better name like @access. My guess is either decl support was added later or by accident but kept because it’s just as useful.

please use a better title like @field for declarations?, this has nothing to do with a static hash table other than being how you came across this behaviour.

1 Like
3

As a similar thing, I was doing lookup table. Now I don’t know which is better:

    /// Strategy for creating destination path
    const mk_lookup = [8]MkPathFn{
        MkPathContext.mkNone, // 000: !flat !mk !acl
        MkPathContext.mkAcl, // 001: !flat !mk  acl
        MkPathContext.mkMkdir, // 010: !flat  mk !acl
        MkPathContext.mkMkdirAcl, // 011: !flat  mk  acl
        MkPathContext.mkFlatten, // 100:  flat !mk !acl
        MkPathContext.mkFlattenAcl, // 101:  flat !mk  acl
        MkPathContext.mkMkdir, // 110:  flat  mk !acl (Identical to 010)
        MkPathContext.mkMkdirAcl, // 111:  flat  mk  acl (Identical to 011)
    };

    const MkPathContext = struct {
        io: Io,
        gpa: Allocator,
        dest: []const u8,
        acl_ctx: ?AclContext,
        strat: MkPathFn,

        /// Private function for path creation
        fn mkPath(self: *const MkPathContext, src_path: []const u8, anchor: []const u8) anyerror!void {
            // Pass self as a generic pointer to the strategy function
            return self.strat(self, src_path, anchor);
        }

        /// !flatten + !mkdir + !acls
        fn mkNone(ctx_ptr: *const anyopaque, src_path: []const u8, _: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, src_path });
            defer ctx.gpa.free(dest_path);

            std.log.debug("Creating path: {s}", .{dest_path});
            Io.Dir.cwd().createDirPath(ctx.io, dest_path) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
                return;
            };
        }

        /// !flatten + !mkdir + acls
        fn mkAcl(ctx_ptr: *const anyopaque, src_path: []const u8, _: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            std.debug.assert(ctx.acl_ctx != null);

            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, src_path });
            defer ctx.gpa.free(dest_path);

            std.log.debug("Creating path: {s}", .{dest_path});
            const status = Io.Dir.cwd().createDirPathStatus(ctx.io, dest_path, .default_dir) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
                return;
            };
            if (status == .created) {
                try ctx.acl_ctx.?.exec(&ctx.acl_ctx.?, 0, src_path);
            }
        }

        /// mkdir (standard and flatten variants)
        fn mkMkdir(ctx_ptr: *const anyopaque, src_path: []const u8, anchor: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            const stripped = stripAfterAnchor(src_path, anchor);
            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, anchor, stripped });
            defer ctx.gpa.free(dest_path);

            std.log.debug("Creating path: {s}", .{dest_path});
            Io.Dir.cwd().createDirPath(ctx.io, dest_path) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
            };
        }

        /// mkdir + acls (standard and flatten variants)
        fn mkMkdirAcl(ctx_ptr: *const anyopaque, src_path: []const u8, anchor: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            std.debug.assert(ctx.acl_ctx != null);

            const stripped = stripAfterAnchor(src_path, anchor);
            const dest_root = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, anchor });
            defer ctx.gpa.free(dest_root);
            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ dest_root, stripped });
            defer ctx.gpa.free(dest_path);

            const base_end = src_path.len - stripped.len;
            const end = base_end - @intFromBool(base_end > 0 and std.fs.path.isSep(src_path[base_end - 1]));

            std.log.debug("Creating path: {s}", .{dest_path});
            const status = Io.Dir.cwd().createDirPathStatus(ctx.io, dest_path, .default_dir) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
                return;
            };
            if (status == .created) {
                var local_acl = ctx.acl_ctx.?;
                local_acl.dest_root = dest_root;
                try local_acl.exec(&local_acl, end, src_path);
            }
        }

        /// flatten + !mkdir + !acls
        fn mkFlatten(ctx_ptr: *const anyopaque, src_path: []const u8, anchor: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            const stripped = stripAfterAnchor(src_path, anchor);
            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, stripped });
            defer ctx.gpa.free(dest_path);

            std.log.debug("Creating path: {s}", .{dest_path});
            Io.Dir.cwd().createDirPath(ctx.io, dest_path) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
            };
        }

        /// flatten + !mkdir + acls
        fn mkFlattenAcl(ctx_ptr: *const anyopaque, src_path: []const u8, anchor: []const u8) anyerror!void {
            const ctx: *const MkPathContext = @ptrCast(@alignCast(ctx_ptr));
            std.debug.assert(ctx.acl_ctx != null);

            const stripped = stripAfterAnchor(src_path, anchor);
            const dest_path = try Io.Dir.path.join(ctx.gpa, &.{ ctx.dest, stripped });
            defer ctx.gpa.free(dest_path);

            const base_end = src_path.len - stripped.len;
            const end = base_end - @intFromBool(base_end > 0 and std.fs.path.isSep(src_path[base_end - 1]));

            std.log.debug("Creating path: {s}", .{dest_path});
            const status = Io.Dir.cwd().createDirPathStatus(ctx.io, dest_path, .default_dir) catch |err| {
                std.log.err("Failed to make path: {s}\n{s}\n", .{ dest_path, @errorName(err) });
                return;
            };
            if (status == .created) {
                try ctx.acl_ctx.?.exec(&ctx.acl_ctx.?, end, src_path);
            }
        }
    };

Selecting fn pointers here:

    /// The master build function that sets up and resolves all runtime branching.
    pub fn build(
        io: Io,
        gpa: Allocator,
        buffers: *Buffers,
        buf: []u8,
        dest: []const u8,
        max_chunk: usize,
        acls: bool,
        checksum: bool,
        algo: ChecksumAlgo,
        mkdir: bool,
        flatten: bool,
        falloc_switch: *bool,
    ) CopyEngine {
        // Bitwise Indexing (flatten: bit 2, mkdir: bit 1, acls: bit 0)
        // 0b000 - 0b111
        const idx: u3 = (@as(u3, @intFromBool(flatten)) << 2) |
            (@as(u3, @intFromBool(mkdir)) << 1) |
            (@as(u3, @intFromBool(acls)));

        // ACL Strategy
        const acl_ctx: ?AclContext = if (acls) .{
            .io = io,
            .gpa = gpa,
            .dest_root = dest,
            .acl_list = buffers.acl_list.allocatedSlice(),
            .acl_value = buffers.acl_value.allocatedSlice(),
            // Setup of the ACL execution function
            .exec = acl_exec_lookup[idx],
        } else null;

        // Copy strategy indexing
        // (acls: bit 1, checksum: bit 0)
        const cp_idx: u2 = (@as(u2, @intFromBool(acls)) << 1) | @as(u2, @intFromBool(checksum));

        // Path resolver strategy indexing (2 bits: flatten, mkdir)
        // 00: None, 01: Mkdir, 10: Flatten, 11: FlattenMkdir
        const res_idx: u2 = (@as(u2, @intFromBool(flatten)) << 1) | @as(u2, @intFromBool(mkdir));

        return .{
            .gpa = gpa,
            .buffers = buffers,
            .mkpath_ctx = .{
                .io = io,
                .gpa = gpa,
                .dest = dest,
                .acl_ctx = acl_ctx,
                // Setup of the path creation strategy
                .strat = mk_lookup[idx],
            },
            .copy_strategy = .{
                .io = io,
                .gpa = gpa,
                .buf = buf,
                .acl_list = buffers.acl_list.allocatedSlice(),
                .acl_value = buffers.acl_value.allocatedSlice(),
                .dest = dest,
                .max_chunk = max_chunk,
                .falloc_switch = falloc_switch,
                .algo = algo,
                // Setup of the path strategy
                .path_resolver = cp_path_res_lookup[res_idx],
                // Setup of the file copy strategy
                .copy_logic = cp_lookup[cp_idx],
            },
            .link_strategy = .{
                .io = io,
                .gpa = gpa,
                .buf = buf,
                .acl_list = buffers.acl_list.allocatedSlice(),
                .acl_value = buffers.acl_value.allocatedSlice(),
                .dest = dest,
                // Setup of the path strategy
                .path_resolver = ln_path_res_lookup[res_idx],
                // Setup of the link copy strategy
                .link_logic = if (acls)
                    @as(LnLogicFn, LinkStrategy.linkAcl)
                else
                    @as(LnLogicFn, LinkStrategy.link),
            },
            .arg_path_strategy = .{
                .io = io,
                .gpa = gpa,
                .dest = dest,
                // Setup of the arg path destination strategy
                .strat = arg_path_lookup[res_idx],
            },
            .file_op = if (flatten) copyFlat else copyStandard,
            .link_op = if (flatten) linkFlat else linkStandard,
        };
    }

I think I should rename it from strategy to vtable, since I believe that is a vtable in Zig. I should learn what vtables are.

4

I’m happy to change the title. By that, I mean that I’m “using” it like a hashtable… ish; I have a “key”, in the form of an E1 (my ‘z’, in the example), which I use to access the value that that maps to {.w, .y, .z}. It’s like a const setup where runtime vars/consts can take on key values that map to those … well, in this case, sets, like the set of {.w, .y, .z} (I realize that’s a slice, but, I’ll probably be using it like a set, ultimately).

With that in mind, does it still make sense to change the title? I’m up for suggestions, for sure.

5

I interpreted the first, vague, question in your post to be the same as your second more specific question. Hence, why I thought the title wasn’t useful.

But if you are wanting more general help with static tables then it’s fine. But you could still be more clear about that in your post.

6

Thank you. I edited to improve, I hope. You’ve already treated #1 for me, so thanks.

7

I’m not sure I’m following it completely, on first study, but yes, looks like you have function pointers, which is vtable-ish… but your implementation looks different than the simple vtables I’ve seen, e.g., in Io (0.16). You’re definitely doing much more, here, than my simpleton case, where I just want a plain map of declared identifiers to other declared identifiers (or sets of them, in my case), without using HashMap.

1 Like
8

since you want a set, and assuming the values in the set is an enum like your example or something just as simple; you could model the set as a struct matching fields of type bool, you can make it packed for reduced space, but that might sacrifice speed.

you could generate the set struct from the enum easily with
const Set = std.enums.EnumFieldStruct(E2, bool, false);

1 Like
9

It’s better if you describe what you are trying to do concretely since what you are doing here is fine-ish in certain cases.

2 Likes
10

Got it. Concrete / simplified… not always easy to choose. I actually have two different uses for this. Here’s one, in a learner project. I’ll shorten, but at least it’s more concrete:

//html.zig
pub const Tags = enum {
   article, aside, footer, header, main, nav, section, // semantic
   b, big, i, small, strong, em, mark, code, samp, kbd, var_, cite, dfn, abbr, time, ruby, rt, rp, q, blockquote, pre, address, ins, del, // text formatting
   div, span, p, h1, h2, h3, h4, h5, h6, hr, br, wbr, // structural and layout
//...
}

pub const Attributes = enum {
   accept, accept_charset, accesskey, action, align_, allow, alpha, alt, as,
//...
};

const ValidAttributes = struct {
   const accept: []const Tags = &.{ .form, .input };
   const accept_charset: []const Tags = &.{ .form };
   const accesskey: []const Tags = &.{ .article, .aside, .footer, }; // ... GLOBAL attribute
   const action: []const Tags = &.{ .form };
   const align_: []const Tags = &.{ .caption, .col, .colgroup, .hr, .iframe, .img, .table, .tbody, .td, .tfoot, .th, .thead, .tr }; // DEPRECATED
//...
};

The map would be used to validate that code which looks to assign an attribute to an element does so according to the rules; if the attribute is not allowed to be assigned to the given element (lookup-table used here), compile error is issued.

11

yeah…, I think using StaticStringMap is better here. You can still do things as much as possible in comptime with .initComptime but also keep operating on the string level to avoid bloating the binary with all the inline for (explicitly or implicitly) when you have to translate from runtime values to enum values.

1 Like
12

Ah, very interesting, thank you. And if I’m reading you correctly, the @tagName() string conversion is the one you’re talking about, right?

So, if I could summarize options:

  • The way I proposed, which may “pack” nicely, and define a lot at comptime and on the stack, but suffers from a bloated binary with lots of string conversions (wherever the @tagName() pops up, commonly in inline for loops.
  • Using StaticStringMap - probably the more typical go-to, where the identities are always strings, never clever bits, but the resulting binary is likely to be slimmer anyway. Definitions can remain comptime-defined, with .initComptime … and perhaps runtime performance, otherwise (e.g., for a “lookup”), would be comparable?
  • In either of these cases, the “set” could be more performant as something other than a plain array. With my original idea, enums.EnumFieldStruct could be handy for generating the structs of bools from enums… In the case of using strings and StaticStringMap, values could be plain BufSet or Treap or anything set-like on strings as elements.
  • EnumMap could be used, with EnumSets as values; I haven’t looked into them, but presumably they don’t use @tagName, so don’t suffer the consequence of string conversions as my scheme did.

Other advice? I lean toward keeping enums, rather than strings, unless it just has too many negatives (thank you for the enlightenment, @hachanuy - I did not anticipate that). In my case, only at the very end of all things, and, in fact, only after compact serialization and delivery to client-side, will WASM code convert the enum (for a tag or attribute name) into an actual string for real HTML. So, if I can avoid @tagName() and whatnot, and keep from using strings, I do think I’d prefer that. Happy to be educated, though, and change my mind.

13

It looks like @tagName and @field are used in these, but they’re front-loaded to .init(), to build the bitmap that is used in lookup / set-membership testing, to keep that hotpath code sleek. That seems “good”.

14

So here’s a version with EnumMap and EnumSet; is this the “better way” that it seems to be?

   const E1 = enum {
      a, b, c,
   };
   const E2 = enum {
      w, x, y, z,
   };

   const ES = std.enums.EnumSet(E2);
   const map = std.enums.EnumMap(E1, ES).init(.{
      .a = .initMany(&[_]E2{ .x, .y }),
      .b = .initMany(&[_]E2{ .w, .z }),
      .c = .initMany(&[_]E2{ .w, .x, .y }),
   });
   try std.testing.expect(map.get(.a).?.contains(.y));
   try std.testing.expect(!map.get(.b).?.contains(.y));
   try std.testing.expect(map.get(.c).?.contains(.y));

It’s in terms of my simplified vague E1 and E2, sorry; imagine E1 as HTML “Tag”, and E2 as attributes, and the map as “allowed attributes for given tags” (or vice versa, if you prefer). That’s the gist.

So this gives me my lookup table, and my value sets for membership/containment tests, and, by the looks of the implementations of EnumSet and EnumMap, provide a slim and performant solution. True? Anything I’m missing?

15

The only thing you’re really missing is that while EnumMap is indeed allocation free, it appears to defeat the optimiser and still perform a memory access even on ReleaseFast with entirely comptime-known arguments.
This is likely actually because of the optional unwrap.

You could optimise this even further by having a table struct type, which for each field of E1 has its own field which is a struct with a void field for each valid E2.
This would allow access to be optimised to a true comptime-known boolean:

pub fn e2_valid_for_e1(comptime e2: E2, comptime e1: E1) bool {
	return comptime @hasField(@FieldType(Table, @tagName(e1)), @tagName(e2));
}

The only problem, and the reason why I’m not sharing the Table code I wrote, is that it’s a very long declaration.
Meaning that it doesn’t fit the “slim” criterion.

EDIT: So, I apparently overlooked that it’s actually trivial to turn the EnumMap access into a true comptime-known boolean as well. All you need to do is put the comptime keyword before it!

1 Like
16

I think I would instead pass ValidAttributes to a comptime function that generates subset enums for every accept, accept_charset, …, while generating those subsets I would give both the original Tags and the subset the same backing int and make sure that during generation of the subset each enum-value uses the same value it had in the Tags-enum.

Then if you ever need to convert from any enum to any other you can use @enumFromInt(@intFromEnum()) if you know that they match, or use std.enums.fromInt if you don’t know whether the target enum contains the value.

Then the code will automatically create an error if you try to use an enum value that doesn’t exist in the generated type.

2 Likes
17

This is great, @tholmes , thanks. It’ll take me a bit to unpack, but can I ask right off the top: is the comment,

suggesting a fault in the implementation of EnumMap? Or just a necessary tradeoff for the implementation, and you’re saying that if I want even better performance, then a table struct type will afford me that. (I’m guessing the latter is true either way, but I’m curious about whether there’s a fault with EnumMap, or just a fault in expecting it to work as efficiently as one might naively expect.)

Also, in your proposed e2_valid_for_e1() function, I see a couple of @tagName() calls, as in my first poor-man’s attempt - would these not suffer the binary bloating mentioned by @hachanuy?

Thank you!

18

There are some missing details here, but I’m inferring that the enums somehow start as a string, from a config file for example.

Ideally, those are converted once, as soon as possible, and the enum used natively from that point forward. No way around it: code which maps from the string to the enum must be emitted, but it’s not that expensive.

What you don’t want to be doing is carrying around those strings as a source of truth.

1 Like
19

No, actually; what I mentioned about that probably got lost in the stream. In fact, NO string starts, and NO need for string representations in the runtime on one host at ALL. So I’d be delighted to stay stringless the whole way for … let’s call it, the “server”. On a client host, one of the significant steps (almost the only real step) will involve @tagName()ing to render the enum tag names as strings. That’s a main reason for keeping the names exactly (as possible) matching with the spec names.

I mentioned that there were TWO projects I’m stewing. One is this HTML toy project for learning. The other, bigger, involves a mapping of (12-tone) note names to frequencies possibilities corresponding to equal/just/etc. temperaments (intonation) - in that case, too, there is no string need at all.

So the focus is on enum space. My OP used @tagName() for the sole purpose of mapping to struct member names, via @field(), but this seems unnecessary, except at an init() time, the way EnumMap and EnumSet do it, for setup. After that, it’d be great to capitalize on stringless primitives to represent the mapping. The important part is the mapping, and, in this case, mapping to a set, so I can do set containment testing efficiently (in terms of memory and compute time, in balance).

So, no, no config file to start things off. No original source of strings. Just the enums according to spec (in the HTML example).

Agreed. Which is the explanation for my lean-away from the StaticStringMap suggestion, which otherwise does have the stated advantage over my original idea (stated in OP).

20

I thought I’d point this out in particular, since the title of this thread refers more to a side-comment about my surprise that @field() works on static consts, which @vulpesx offered clear insight on. But the title got changed. My opening line, “I want a poor-man’s static lookup-table, and thought about implementing it like this…“ is still representative of my deeper interest.