Discussion: A potential solution to the anytype problem

I have been programming a long time in the languages delphi, c#, rust and zig.

In my feeling there is no easy solution for this (I encountered the thing myself a few times in Zig).
In delphi and c# I do not like at all the interface mechanisms at all.
The traits in rust even less.
The question: “Where is the f*ng implementation of this thing” will always be there, regardless what construct the language uses.
Adding sugar for “does this thing support this or that” does not necessarily make things more readable.

The extreme flexibility and shortness of anytype is up until now still the best thing I encountered.
Not 100% ideal but the best :slight_smile:

(I also think interfaces should be used as little as possible. Often we can read the phrase “always talk to interfaces, not classes”. Disagree!).

1 Like

It was an attempt to reduce the number of fresh Brainstorming topics created by very new users, which often haven’t read any existing topics, suggesting things that have been discussed a lot already.

This topic especially seems like another iteration of parts of these topics (ordered in reverse chronological order):


And here some quotes I agree with:

Reading between the lines from prior communication from the core team on this subject, my understanding is that the reason why most proposals for interface- and trait-like features have been rejected (and why validating the “shape” of an anytype value using something like comptime implementsInterface(@TypeOf(foo)) is frowned upon) is because such features would introduce more complexity to the language but not actually help the user write better, safer or more efficient code. The pitfalls of runtime duck typing don’t apply for anytype; the code either compiles or it doesn’t.

Eagerly validating the shape of an anytype parameter can arguably be detrimental. You might think you want to verify that a polymorphic argument foo implements both of the methods foo.getName() and foo.getValue(), but if your code only ever calls foo.getName(), you’re only wasting your user’s time with needless busywork by forcing them to always export the unused foo.getValue(). It’s better to avoid eagerly validating the shape and instead let compile errors guide the user into exporting precisely what is needed.

The only actual problem with anytype is that it doesn’t solve the problem of communicating the expected contract to the user at all. Compile errors can only get you so far and can often be cryptic and unhelpful, and the trial-and-error loop of resolving one error at a time can become frustrating. And documenting expectations by hand can be difficult and introduce a risk of the documentation not matching the actual implementation.

The core team doesn’t think interfaces or traits is a good enough fix for the documentation problem because such a fix would be a local maximum and not the universally best/most useful solution. A better (and long-term planned) and more thinking-outside-the-box solution is to have the compiler implement a language server and serve information about the exact required shape of an anytype parameter based on the surrounding context.


Edit: I should also add that whether or not anytype’s “documentation problem” is an actual problem in practice depends a lot on the user’s mindset and habits when it comes to reading code. Zig is a very explicit and readable language to us humans and you are strongly encouraged to read the code of the libraries/dependencies you use, not just the API surfaces. If a function has an anytype parameter and you dig into the function implementation, give the code a cursory glance and still can’t make heads or tails on the expected shape, it’s a good suggestion that it might be an improper use of anytype and that the function/its API could use a bit of a do-over.

10 Likes

Why not something like this?

fn feed(animal: anytype) void {
    if (requires {
        animal.eat();
    }) {
        @compileError("Type '" ++ @typeName(@TypeOf(animal)) ++ "' does not satisfy the function requirements of 'feed'");
    }
    animal.eat();
}

The requires expression would just return if the block is able to compile or not (via a boolean).

That could even work for something like this:

const Vector = struct {
    x: i32,
    y: i32,
    z: i32,
};

fn move(object: anytype, vector: Vector) void {
    if (requires {
        object.*.x += vector.x;
        object.*.y += vector.y;
        object.*.z += vector.z;
    }) {
        @compileError("Type '" ++ @typeName(@TypeOf(object)) ++ "' is not a pointer to a mutable object with x, y and z coordinates");
    }
    object.*.x += vector.x;
    object.*.y += vector.y;
    object.*.z += vector.z;
}

(To prevent code duplication one could put the actual function implementation into another function.)

At its core, anytype is really only nice syntax over fn func(T: type, parameter: T) void, at least as far as a Zig programmer is concerned. Because of DX that’s imo even a necessary thing (think of if you would need to use the long syntax for print calls).

And this is really just unconstrained C++ templates with a more uniform syntax:

template<typename T>
void func(T parameter);

And because of that I do think that Zig needs something to constrain things too, because this has the same problems unconstrained C++ templates have (over constrained ones). And being able to chose compile time over runtime polymorphism is needed in some problem domains (even if it’s at the sacrifice of a bigger binary). Besides the fact that well implemented tooling gives imo a better DX for compile time polymorphism than for runtime polymorphism.
Chasing down a bug into a runtime interface is just a lot more annoying than if you are dealing with monomorphism, independent of the language.

And what I threw into the room here (which is essentially just requires expressions from C++20) would make it at the very least easier for programmers to chose different implementations depending on the parameters. So essentially explicit compile time overloading.

But I guess we’ll see what the core team will do about it (which hopefully will be something instead of nothing, even if they currently seem to prefer runtime polymorphism, judging from the last few releases and the last few accepted proposals).

I had suggested something like

const Animal = interface { Dog, Cat };

fn animalEats(animal: interface.Animal) void {
    animal.eats();
}

That is, a named, type-restricted anytype. Similar to how error unions look like.

With the Interface segregation principle an interface should be split if it is expected that some users might need only part of it.

In this regard the “Animal” example in this thread for things with an “eat” method is bad; the interface should be something like “Eater”.

An important purpose of named interfaces/traits (instead of duck typing) is that they don’t just validate the name/shape but also the meaning of methods. Both the use and the implementation of an interface need to have the same interface definition in view; this ensures that a called method does not just happen to have the expected name and parameter list, but also actually does what the caller expects.

This is why I think that this part of OP’s proposal is not helpful:

fn feed(animal: anytype satisfies { fn eat void, age: u8 }) void {
    animal.eat();
}

This still just requires that there is a method called eat, it does not ensure that this method does what the feed function expects it to do.

2 Likes

That is, a named, type-restricted anytype. Similar to how error unions look like.

FWIW this strongly-typed style is already fairly trivially possible already:

const Animal = union(enum) {
    dog: Dog,
    cat: Cat,

    pub fn eat(self: Animal) void {
        switch (self) {
            inline .cat, .dog => |s| s.eat(),
        }
    }
};

fn animalEats(animal: Animal) void {
    animal.eat();
}

test animalEats {
    const cat: Cat = .{ .age = 10 };
    animalEats(.{ .cat = cat });
}
1 Like

Yes but switching happens at runtime so it’s not exactly the same. Or I misunderstood?

2 Likes

The most enjoyable of Zig’s ways to me is I find it extremely opinionated in practice, but doesn’t really care about theoretical principles.

This, typically, I would argue is obvious in theory but hard to do in practice. It leads to focus on categorisation and it’s easy to get it wrong and create abstractions that are more harmful (complexity wise) than useful for the benefits it provides.

I certainly fall for that a lot in general and in particular for anytype I used to think (maybe a couple of years ago) strongly that it was nice but kinda not “finished” feature and missing ways to express what was expected from such values to conform to.

Interestingly I didn’t thought about that a single time in the past 6 months and writing much more zig than prior. Most of the time I find the expectations logical from context and when not I just jump in, have a quick look in the function and that alongside compile errors is satisfactory.
Note that the extremely fast feedback of incremental builds makes this argument even stronger. Definitely faster to go through 10 well known compile errors than wrapping your head against a conceptual category.

But granted, it requires practice with the language and established practices. And granted as well than you can totally come up with very bad APIs that abuse comptime magic via anytype. But you can come up with very bad traits categorisations as well.

This makes me think of a comment Andrew wrote once about what goes and doesn’t go in Io interface that was in substance (and sorry for not finding the quote) You have to let go of this OOP bull**** and just think all that can block in async context goes in and it will all become clear. I personally found that enlightening.

[Sorry, came out longer than expected but wanted to share the opinion of a happy and convinced user :smiley:]

1 Like

(post deleted by author)

I like this proposal. It’s simple and satisfies the main requirements for traits: it offers actionable information to both the user (via function signature) and LSP (enabling auto-complete).

With the recently accepted syntax chqnge, this would look like this:

fn feed(animal: |T| satisfies { fn eat(T) void }) void

Which seems quite elegant to my eyes.

1 Like

With |T| syntax maybe also this would be possible:

fn IEater(T: type) type {
    if (!@hasDecl(T, "eat") or @TypeOf(T.eat) != fn (T) void)
        @compileError(@typeName(T) ++ " lacks `eat` method.");
    return T;
}

fn feed(animal: IEater(|T|)) void {
    animal.eat();
}
2 Likes

I think therein lies much of the challenge. The “Eater” (rather than “Animal”) example seems sensible enough as we’ve analyzed it (and brings back memories of conversations like this in C++ back in the GoF 90s), but can/must we anticipate or constantly adapt to our API users’ every twist with constraint evolution? There is a power to the anytype motif in freeing the API to evolve new functions without requiring existing code to suddenly support them UNLESS you actually run into a compile-error (as currently happens with anytype, as already noted), even if that error cannot always be as clear as we might wish. Is the cost of clearer verbosity worth the extra code? That’s what differentiates languages, and I like that, in zig, you can manufacture clarity like this now, as @naxlavun and @tholmes have shown. For me, it would be important that we not be forced to define interfaces clearly in order to implement polymorphic solutions; that is, that something like the current anytype would always be an available option.

(And I’m not implying that the OP suggests this be forced; I’m supposing that (foo: anytype would continue to work as it does, and that (foo: anytype satisfies ... would be optional sugar for clarification. That would be fine in my opinion, if it added enough value to justify supporting the extension to the language. On that question, I think it could, but would have to play with it more to feel convinced.)

FYI: I wrote this like four hours ago, but lost cell service half way through, so I’m posting it now.

Wow, that’s a lot of responses (I guess this is quite the hot topic). This idea is certainly being pulled in many conflicting directions (which is important for discussion), but I do want to bring it back to the core idea: a ‘syntax sugar’-only shorthand for manually writing comptime checks. To that end, I’ve defined a few constraints for this feature:

  1. Syntax sugar only: any satisfies block should be directly convertible to comptime checks, and it should not be able to do anything that is not already possible now.
  2. No new builtins. Builtins (like @TypeOf()) can be part of the intended use-case for the feature, but no new builtins should be added and the existing behavior of all builtins should remain the same.
  3. Supporting every possible use-case is not the goal. The idea is to provide a shorthand for the most common use-cases. The following are what satisfies should be capable of doing:
  • Specifying whether an anytype/type should be struct, union, enum, etc.
  • Specifying the required names of an anytype/type’s members and fields.
  • Specifying the required return type of an anytype/type method.
  • Specifying the required types of the parameters of an anytype/type method.
  • Specifying the required types of an anytype/type’s fields.
  • Specifying that an anytype/type’s method parameters/fields should be its own type.
  • Splitting its declaration and usage.
  1. This feature should not be able to:
  • Be used outside of a function declaration.
  • Be referenced as a type.
  • Exist after compilation.

Here is the syntax that I’ve come up with:

// Separated declaration, 'struct' specifies that it should be a struct. This might conflict with regular struct declarations though. Any ideas?
const Animal = satisfies struct {
    age: _, // Any type (different from specifying that it should be 'anytype').
    height: u8,
    fn eat(@This(), u8) void, // @This() breaks constraints 2 and 4b a bit, which is why I'm looking for an alternative. Any ideas?
    // I propose that the '_' syntax can also be used in place of 'void' to specify 'any return type'.
};

fn foo(animal: anytype satisfies Animal) void {}

// Inline version
fn bar(animal: anytype satisfies struct { age: _, height: u8, fn eat(@TypeOf(animal), u8) void }) void {}

This is honestly quite messy and I don’t claim to have the answers to everything. At the very least, I found the thought process and discussion to be quite valuable in growing my understanding of the problem.

Know that I have read every message in this thread, but I am unable to address every concern that has been raised, many of which are way past my skill level and understanding. Maybe this is naive (feel free to call me out on that), but I am not trying to debate the philosophy of whether or not interfaces as a concept are a good practice, nor am I capable of doing so; I simply wish to create a discussion around whether or not some syntax sugar could enable readability and tooling to an existing zig feature.

1 Like

Oh sorry, I thought this was implied; current usage of anytype would still be completely valid, and satisfies is only an optional shorthand for those who want it. Like I said, it’s meant to be syntax sugar as an alternative to a bunch of comptime checks in the function body, not some sort of enforced system that everyone must adhere to. In this way, if someone wanted to check the type in a more complex way, they can just emit the satisfies and use comptime checks as normal. This also means that satisfies does not have to cover every use-case, and can therefore drop some complexity that would otherwise be required.

1 Like

The through-line that I’m getting from these and historical suggestions is a better way to communicate constraints of an unknown/generic type.

Perhaps a compromise is to improve the clarity of the error message when a value passed through any generic function argument is “misused”. For example:

const empty = .{};

fn printdata(thing: anytype) void {
    std.debug.print("{any}", .{thing.data});
}

Already gives the error (in 0.15.2) of error: no field named 'data' in tuple '@TypeOf(.{})' std.debug.print("{any}", .{thing.data});

If the compiler also noticed that the offending value passed through an anytype parameter in the call stack, then it could also say:

function printdata requires the value of 'thing: anytype' to contain the field 'data'

This communicates the link between the generic function itself and the implicit requirement, not just at the line where the error occurred, and not just buried in the call stack.

Still not ideal, since it relies on compilation errors, but it would be an improvement in clarity.

2 Likes

I had mentioned previously in this thread sharing an implementation for interfaces, I haven’t used this code in quite sometime and I am quite certain that there could be improvements made to it, but sense I do not personally really use this in my code anymore I haven’t made those appropriate changes. But if others are looking to do some kind of compile time checking of a group of methods on a struct here is the launching off point you are likely looking for. This will throw a compile time error if a struct that does not implement the interface is passed into the check. The pattern is essentially to create a function that returns a type that wraps the implementor and its functions. IE you are actually interacting with the wrapper which is proxying the request down into the actual implementer and hiding other available method on the implementor. As part of the doing that you take that ‘result’ in the code block below and use it to define the methods that constitute the ‘interface’ than it asserts onto the passed implementor that it does indeed implement the group of methods and their signatures exist returning the newly constructed wrapper that proxies those calls.

Additionally if there is sufficient interest in this I do not mind moving it to its own repository and setting it up as a library.

(I have not built this past 15.2 so some changes maybe required I am unsure)
https://codeberg.org/0mn1a/Juniper/src/commit/f1e5d375f0492e1ec48c74aa21a43f3d3ad74252/src/utility/Interface.zig#
Usage:

pub inline fn Container(comptime implimentor: fn (comptime type, comptime usize) type, comptime T: type, comptime U: usize) type {
    const impl = implimentor(T, U);

    const result = struct {
        implimentation: impl,

        pub inline fn init() @This() {
            return .{ .implimentation = impl.init() };
        }

        pub inline fn add(this: *@This(), x: T) AddErrors!*T {
            return this.implimentation.add(x);
        }

        pub inline fn remove(this: *@This(), target: *T) RemoveErrors!usize {
            return this.implimentation.remove(target);
        }

        pub inline fn findIndexFromPointer(this: *@This(), element: *T) FindIndexFromPointerErrors!usize {
            return this.implimentation.findIndexFromPointer(element);
        }
    };

    _ = Interface(result).Implimentor(impl);
    return result;
}

pub const FindIndexFromPointerErrors = error{OutOfBounds};
pub const RemoveErrors = error{ElementNotFound};
pub const AddErrors = error{CapacityError};

pub const ContainerErrors = error{ CapacityError, ElementNotFound, OutOfBounds };

const Interface = @import("Interface.zig").Interface;

The actual implementation of the Interface type:

const std = @import("std");

const Testing = std.testing;
const expect = Testing.expect;
const expectError = Testing.expectError;

pub fn Interface(comptime T: type) type {
    return struct {
        pub fn Implimentor(comptime U: type) type {
            const interface = @typeInfo(T);
            const implimentation = @typeInfo(U);

            const interfaceDecls = switch (interface) {
                .@"struct" => |s| s.decls,
                else => @compileError("Interface must be a struct"),
            };
            const implimentationDecls = switch (implimentation) {
                .@"struct" => |s| s.decls,
                else => @compileError("Implementation must be a struct"),
            };

            declLoop: inline for (interfaceDecls) |interfaceDecl| {
                const interface_fn = @TypeOf(@field(T, interfaceDecl.name));
                const interface_fn_info = @typeInfo(interface_fn);
                const interface_fn_fn = switch (interface_fn_info) {
                    .@"fn" => |s| s,
                    else => @compileError("Interface should only hahve fn Declarations"),
                };

                inline for (implimentationDecls) |implimentationDecl| {
                    _ = implimentationDecl;
                    const impl_fn = @TypeOf(@field(U, interfaceDecl.name));
                    const impl_fn_info = @typeInfo(impl_fn);
                    const impl_fn_fn = switch (impl_fn_info) {
                        .@"fn" => |s| s,
                        else => @compileError("Interface should only "),
                    };

                    if (interface_fn_fn.params.len != impl_fn_fn.params.len) {
                        @compileError("Mismatched number of parameters" ++ interfaceDecl.name);
                    }

                    paramsLoop: inline for (interface_fn_fn.params, impl_fn_fn.params) |expParam, implParam| {
                        if (expParam.type == *T) continue :paramsLoop;
                        if (expParam.type != implParam.type) {
                            @compileError("Parameter signature doesn't match " ++ interfaceDecl.name);
                        }
                    }

                    const interface_fn_return = @typeInfo(interface_fn_fn.return_type.?);
                    const impl_fn_return = @typeInfo(impl_fn_fn.return_type.?);

                    switch (interface_fn_return) {
                        .error_union => |K| {
                            const set = K.error_set;
                            const returnTypes = K.payload;

                            const implSet, const implReturnTypes = blk: {
                                switch (impl_fn_return) {
                                    .error_union => |S| {
                                        break :blk .{ S.error_set, S.payload };
                                    },
                                    else => {
                                        @compileError("Implimentor is not expected error union on decl: " ++ interfaceDecl.name);
                                    },
                                }
                            };

                            if (set != implSet) {
                                @compileError("Implimentor does not impliment the correct errorset on decl: " ++ interfaceDecl.name);
                            }

                            if (returnTypes != implReturnTypes and returnTypes != T) {
                                @compileError("Implimentor does not satisfy return type on decl: " ++ interfaceDecl.name);
                            }
                        },
                        else => {
                            if (interface_fn_fn.return_type.? != impl_fn_fn.return_type.? and interface_fn_fn.return_type.? != T) {
                                @compileError("Return Type Rejection " ++ interfaceDecl.name);
                            }
                        },
                    }

                    continue :declLoop;
                }
                @compileError("Failed to satisfy interface: missing " ++ interfaceDecl.name);
            }

            return U;
        }
    };
}

Just throwing in the idea, how about this:

In zig types are first class values and generics are implemented via functions Like so:


fn Foo(comptime bar: baz) type {
    // ...
}

This is a very powerful pattern that allows for metaprogramming without a separate macro language/syntax. Unfortunately this means that types cannot be understood by the compiler as easily as in languages with traits or templates.

How about leveraging zig’s existing strengths (comptime evaluation, use of zig for metaprogramming) and making the satisfies mechanic also a function? The satisfies function would have to have a return type of !void and would make use of errors to report any missing info


fn Foo(bar: anytype satisfies myInterface(@TypeOf(bar))) void {
    // ...
}


fn myInterface(comptime T: type) !void {
    try std.interface.ensureField(T, "baz", u8);
    try std.interface.ensureSelfFn(T, "qux", *const fn (a: T, b: u8));
}

// std/interface.zig

fn ensureField(comptime T: type, comptime name: []const u8, comptime V: type) !void {
    if (does_not_have_field) {
        @compileLog(@typeName(T) ++ " is missing field " ++ name);
        return error.InterfaceFieldMissing;
    }
}

This way you can express arbitrary constraints (including an interface dependant on multiple anytype arguments), tooling can leverage existing comptime evaluation logic, and no additional syntax or concepts need to be introduced apart from the keyword. Interfaces can also be easily combined together (since they are just functions), (fuzz) tested by the existing testing utilities, ect.

It would be more difficult to implement good type hinting for a language server, but I think it’s a fitting choice for the language.

3 Likes

Rejected.
Contrary to @Tallis-Larsen 's idea, this is not actionable neither by the user or the LSP. The user would still ned to read the function body to know what the constraints are, just instead of reading this function, the user would have to find the constraint function and read it. Also, an LSP can’t provide type completion based on arbitrary code inside a function.

2 Likes