Adventures in meta programming: Compile time generic "interfaces"

Brainstorming
1

Good Afternoon all, I was looking for some feedback on this particular pattern, after moving through the patterns post I couldn’t really find what I was looking for. And even in the Ghostty post I wasn’t really satisfied with what I found. After doing a bit of hacking this is what I came up with and I’d like to get some feedback on how to improve this by some people who have a bit more experience.

My goal in this pattern is to have concrete types that implement a number of different methods, and at compile time pass in the type for each implementation, and then from an enum tag pick the implementation. Also at compile time enforce that all implementations at minimum must implement a method with a particular name and signature. The end result looks something like this:
ConnectionManager(.static, 1024).init(io)
const connection = try self.connectionsManager.interface.add(stream);

I understand that I haven’t created anything unique here, I just can’t find any examples to compare my implementation of the pattern against to see if there is anyway I can clean this up more to use in other places.

const ConnectionManagerInterface = enum { static };

pub fn ConnectionManager(comptime interface: ConnectionManagerInterface, comptime max: usize) type {
return struct {
interface: genUnion(Stream, max),
io: Io,

    pub fn init(io: Io) @This() {
        return .{ .interface = @unionInit(genUnion(Stream, max), @tagName(interface), .init()), .io = io };
    }

    pub fn deinit() void {}

    fn genUnion(comptime T: type, comptime size: ?usize) type {
        const capacity: usize = size orelse 1024;
        return union(enum) {
            static: BoundedArr(capacity, T),
            //dynaimc: std.array_list.Aligned(T, std.mem.Alignment.of(T)),

            pub fn add(this: *@This(), value: T) !*T {
                switch (this.*) {
                    inline else => |case| {
                        return try @constCast(&case).*.add(value);
                    },
                }
            }

            pub fn remove(this: *@This(), value: T) !usize {
                switch (this.*) {
                    inline else => |case| {
                        return try @constCast(&case).*.remove(value);
                    },
                }
            }

            pub fn findIndexFromPointer(this: *@This(), target: *T) !usize {
                switch (this.*) {
                    inline else => |case| {
                        return try @constCast(&case).*.findIndexFromPointer(target);
                    },
                }
            }
        };
    }
};

}

Thank you for any help you can provide!

edit: You can test the failure to compile by adding in the dynamic enum and uncommenting the dynamic portion because array list doesn’t impliment those methods.

2 
            pub fn add(this: *@This(), value: T) !*T {
                switch (this.*) {
                    inline else => |*case| {
                        return try case.add(value);
                    },
                }
            }

For functions return a type, the naming convention requires that the first letter be capitalized and the name directly indicate the type being generated. If there is really no meaningful name, then use something like ‘Dispatcher’.

A tagged union can specify an enumeration, and it looks like the ConnectionManagerInterface you defined earlier applies here. The naming can probably still be debated.

return union(ConnectionManagerInterface) {

I guess Zig would probably be more inclined to encourage you to perform dependency injection of Io at the point of the API call, so that users clearly understand the use of Io each time they execute an API that depends on it. A similar situation has already occurred with Allocator.

I understand the intuitive need to manage them, but you could consider not doing so and directly using the Union as your Connection type.

2 Likes
3

I dont think this does what you think it does, your code cant have both static and dynamic versions available to the user, the moment you try you get a compile error, regardless of which the user actually requests.

Which begs the question, what is this even supposed to do?

If your intent is for the interface implementation to be known at compile time, you shouldn’t be using a tagged union. Their purpose is for dynamic types, when you seem to want static types.

This is what I would expect, given my understanding of what you want:

const ConnectionManagerInterface = enum { static, dynamic };

pub fn ConnectionManager(comptime interface: ConnectionManagerInterface, comptime max: usize) type {
    return struct {
        const Static = BoundedArr(max, Stream);
        const Dynamic = std.ArrayList(Stream);

        inner: switch (interface) {
            .dynamic => Dynamic,
            .static => Static,
        },

        pub fn init() @This() {
            return .{.inner = switch (interface) {
                .dynamic => .empty,
                .static => .init(),
            }};
        }

        pub fn deinit() void {}

        pub const add = switch (interface) {
            .dynamic => addDynamic,
            .static => addStatic,
        };

        fn addStatic(cm: *@This(), stream: Stream) !*Stream {
            return cm.inner.add(stream);
        }

        fn addDynamic(cm: *@This(), allocator: Allocator, stream: Stream) !*Stream {
            try cm.inner.append(allocator, stream);
            return &cm.inner.items[cm.inner.items.len - 1];
        }

    };
}

At which point it’s just a wrapper for either DynamicArr or std.ArrayList.
This is only useful if you want to configure which you use with a build option, and want a consistent API.

If you’re not doing that, then just use either directly.

If you want it to be runtime configurable, just use a tagged union, the above is a different type depending on the interface argument, so you can’t pass it around dynamically.

Neither the above nor what you had originally, would I call an interface. For it to be an interface, I argue it has to be able to be implemented by a third part/consumer of your library. Otherwise, it’s just an abstraction.

3 Likes
4

My intent was specifically to achieve: “This is only useful if you want to configure which you use with a build option, and want a consistent API.” so that I can have the same top level methods to work with while being able to swap out the underlying implementations.

”Neither the above nor what you had originally, would I call an interface. For it to be an interface, I argue it has to be able to be implemented by a third part/consumer of your library.”, and I guess we will just have to disagree with what the definition of an interface is. I don’t think I’ve ever seen that to as part of the definition of what an interface is, from everything that I can find it is defined as a datatype that enforces classes or objects implement certain methods. I could be wrong, but from what I looked up I felt that this met that definition. While my post isn’t about the definition of an interface, and I really don’t want this thread to spin off into something unrelated. Bringing that around to say you bringing up your point did nerd snipe me into realizing that while this implementation checks at compile time for method names on the wrapper, it doesn’t check return types, so I’m going to look into that.

I would like to add that a worthy difference between our two examples are that you have to implement on your struct all the different types add function, in my example we are more asserting that the members of the union implement those functions (By name only atm, but I’m going to see about also enforcing the function signature). This allows any struct to be added to this one spot in the code and be switched to and optimized for compile time, or at least that is my goal here.

”your code cant have both static and dynamic versions available to the user” I’m not sure what you mean by this, the code does compile for both versions (If I proxy the dynamic, I haven’t written that implementation yet). So I’m interested to know what you specifically mean by this.

5

Thank you for all of these suggestions, I’m going to try and implement them and retest everything. And the io part, I do agree, that I think it would be much better to pass it in where it needs to be, but given the context that I am going to be using this in it may be better not too, I’m not sure, maybe I need to accommodate the conventions even if it isn’t as convenient

6

Using a tagged union doesn’t exclude the unused implementation, because at runtime you could switch to the other. So the optimiser cant assume which is being used. And both will be compiled.

You also get assertions on the existence and signature of functions for free just by using them, you are only changing where that happens. Which is certainly valuable to do.

That was my point, what you have written requires the dynamic implementation to be proxied, otherwise it doesn’t compile. The alternative is to ‘inline’ the proxy and deal with the differences in ConnectionManager, I don’t see why you wouldn’t want to do that.

That is an accurate techinical definition but it is also quite vague. This is mostly pedantic, but thats important for clear communication; what I said was not clear, sorry.

‘Interface’ is usually used to refer to a runtime known implementation and/or being able to provide an implementation outside a static selection. I was trying to encourage different terminology.

But a ‘compile time interface’ is an accurate description of what you are trying to do. ‘Generic’ on the other hand is the inverse, where you have a single implementation that handles multiple kinds of things.

Programming terminology is a mess :slight_smile:

1 Like