Traits for Static Dispatch

I decided to write up a quick proposal for static-dispatched traits, I think there are several problems with anytype and I’m hoping that this explains some of my problems with it as well as how they can possibly be solved.

I’ll apologize ahead of time for the 80-character linebreaks which might be a bit frustrating to read on web, I had written this in a text editor ahead-of-time :sweat_smile:

Problems with Current Solutions

There are two main ways to do static dispatching in Zig:

  1. somevar: anytype
  2. fn myFn(comptime T: type, somevar: T)

Both of these suffer from the same issue: there is no named type
associated with the identifier being defined.

Because there is no named type constraint for the identifier,
a beginner would never know what to pass in to writer: anytype,
but if it could be like writer: std.io.Writer, it’s much more clear (as well
as there now being documentation for what std.io.Writer is). This also
helps in large codebases where there may be an important difference between
different kinds of writers, and there’s no way to separate
special_writer: anytype and io_writer: anytype (where with traits,
this could be writer: mylib.Writer and writer: std.io.Writer).

In addition to poor documentation, anytype also runs into the issue of having
pretty awful error messages. Accidentally passing a BufferedWriter
or File to a library which takes a writer: anytype
may give an error message that relates to a deeply-nested file within
the library’s code. It would be great to have a solution which tells the caller
that they are simply calling the function incorrectly.

Static Dispatch: Trait Types

The goal of Trait Types is to allow an interface-like type which
can have reusable types, documentation, and better error messages.

Trait types may only be used in the same places as anytype.
For instance, fn (x: SomeTrait) void is legal, but
const x: SomeTrait = 5 is not.

A type is assignable to a given trait type
if it contains the same fields and public declarations that the trait has.

Defining and using Traits

For instance, we could have a std.io.Writer interface defined as:

/// A standard writer interface which allows for statically-dispatched writers.
/// See `GenericWriter` for a way to implement this trait using only a `write` function.
const Writer = trait {
    pub const Context;
    pub const WriteError;
    pub fn write(self: Context, buf: []const u8) WriteError!usize;
    pub fn writeAll(self: Context, buf: []const u8) WriteError!void;
    // ...
};

(I’m aware that std.io.Writer is going to be entirely
reworked for std.Io, but I’m using it as an example since
it’s one of the most widely used cases for static dispatch anytype)

Aside - I have also taken a liking to naming this anytype
(ie: const Writer = anytype { pub const ... }).
This makes it much more clear that this is essentially just a
restricted form of anytype (and the feature could
be called “anytype constraints” instead of “traits”)

Implementing this trait would then look like:

// this is implementing `Writer` from scratch; in reality you'd use `GenericWriter`.
const MyThingWriter = struct {
    pub const Context = *MyThingWriter;
    pub const WriteError = error{ SomeError };
    pub fn write(self: *MyThingWriter, buf: []const u8) WriteError!usize {
        // ...
    }
    pub fn writeAll(self: *MyThingWriter, buf: []const u8) WriteError!void {
        // ...
    }
};

// and GenericWriter could look like:
pub fn GenericWriter(
    comptime ContextT: type,
    comptime WriteErrorT: type,
    comptime writeFn: fn (context: Context, bytes: []const u8) WriteError!usize,
) type {
    return struct {
        pub const Context = ContextT;
        pub const WriteError = WriteErrorT;

        pub inline fn write(context: Context, bytes: []const u8) WriteError!usize {
            return writeFn(self.context, bytes);
        }
        // ...
    };
}

This makes it so that now functions which used to be defined
as fn writeMyThing(myThing: MyThing, writer: anytype) !void can become
fn writeMyThing(myThing: MyThing, writer: std.io.Writer) !void,
and it would work the exact same.

This would also restrict type bounds at the call-site of writeMything,
rather than inside of a deeply-nested file in library code.

One thing to take note of is that traits are implicitly implemented
(really, traits aren’t “implemented” at all - they’re
just type-bounds for functions). However, it may be worth adding a
way to explicitly implement an interface with a function like
try std.testing.expectImplements(MyType, SomeTrait), or even
an explicit implementing keyword on struct definitions.

9 Likes

Have you seen any of the GitHub issue discussions related to this? The four big threads I remember are 1268, 1669, 6615, and 17198, but there were more.

7 Likes

I believe that there is real benefit to the traits as defined, for documentation and editors code completion. A similar design might omit the self arguments in traits.

Andrew response to a similar proposal for static dispatch interfaces was: comptime interfaces · Issue #1268 · ziglang/zig · GitHub

I’m not saying there won’t be interfaces of any kind, ever, but there are no current plans for adding them, …
I suggest if you want to use zig, you accept the fact that it may never gain an interface-like feature, and also trust the core team to add it in a satisfactory manner if it does get added at all.

2 Likes

I think one can accept that a change might never happen while still lobbying for that change. I think OP is 100% correct that anytype is too generic and having a way to put constraints on the type would be a great improvement for the readability of the language.

5 Likes

While agreeing something maybe should change regarding anytype / interfaces, I also think flexibility should be preserved. Currently we can literally just put anything in an anytype and the compiler figures it out.
What if anytype should be a u8 or u16? Should we have a trait for that?

I think it is a very very difficult problem. Before we know we would get into Rust-like unreadable stuff. I cannot visualize any good + simple + readable syntactical solution better than the current one with anytype.

6 Likes

If #32099 is accepted, I think the following options are worth considering.

Allow |T: trait| syntax. When the inferred T does not satisfy trait, it leads to compilation errors. I think this approach is more intuitive than simply writing trait in the type position.

I have thought of using many keywords to define the keywords for this container: interface/anytype(from Zig)/trait(from Rust)/concept(from C++)…I’ll call it trait for now.

trait should be able to perform type merge operations like error. That is to say, in |T: trait1 || trait2|, T satisfies both requirements trait1 and trait2. But, semantically speaking, && may be more suitable.

type can be regarded as an unconstrained trait. Or, trait is a subset of type. Just like the relationship between error and anyerror. trait is more like types of type.

I think clear boundaries are more important than false freedom. anytype allows you to pass any type to parameters, but this does not mean that the API can accept all types.

5 Likes

I would personally really like use something like that, as my code has a lot ducktyped anytype code unfortunately.
I could add safety checks with reflection but what ends up happening is that I decide it would take too much time and avoid it.
I think something like a generic that returns a trait would be useful. (Example, Iterator(T) would be a function that returns a trait, and that trait has a next() function that returns ?T etc. On that topic, I tried making a userland implementation of comptime interfaces and it ended up being pretty bad).

It would also solve the Io regressions (Binary size + mandatory VTable indirection). However it could be the case that this feature is redundant when the core team solves these regressions (with potentially a different mechanism)

Personally, as a user of zig I am really concerned with trying to write optimal machine code (And I chose zig because more so than any other language it cares about that), and things like that would really reduce friction when trying to write optimal machine code. We make fun of OOP for putting VTables everywhere for things that are comptime known.

2 Likes

I guess the replacement of anytype would be a good opportunity to bring those traits.

I’ve seen a few different kinds of proposals though.

  1. They could be a sort of type-matching for structural types:
generic_pointer: *|T|,
  1. They could also be about the definition location of non-structural types, to work well with generic types:
array_list: std.ArrayList(Dummy |Item|),

The Dummy is a concrete type that indicates that the definition of std.ArrayList(Item) should be that of std.ArrayList(Dummy).

  1. Or we could use a new trait {} construct.

  2. Or define traits as an interface in std.builtin like this:

pub const Trait = struct {
    result: fn (comptime type) Result,

    pub fn assert(comptime trait: Trait, comptime T: type) void {
        comptime if (trait.result(T).fail()) |failure|
            @compileError(std.fmt.comptimePrint("{f}", .{failure});
    }
};

And then generic: |T: trait| would just be sugar for generic: |T| and inserting trait.assert(T).

2 Likes

I would caution against interpreting #32099 as opening the door to parametric polymorphism. The syntax might make it seem a relatively small leap, but it’s incompatible with how the type system currently works and other parametric polymorphism proposals have already been rejected.

8 Likes

Yeah, but as things are (and with how Zig tries to position itself), the request for something like this will be a constant in Zig’s lifetime until something like this is either added or Zig is barely used anymore.

People like compile time polymorphism while idiomatic Zig seems to prefer runtime polymorphism these days for more complex stuff (at least if we go by the last two releases), but with anytype (or the pattern of std.mem.eql) you have unconstrained duck-typing which isn’t really liked in compiled languages by many people.

To make a comparison with C++:

anytype is like

auto add(auto a, decltype(a) b) {
    return a + b;
}

but people would like to have something like

auto add(Arithmetic auto a, decltype(a) b) {
    return a + b;
}

And I can understand it. Error messages from this without constraints are just garbage if the program becomes complicated.

And before C++ concepts I have seen people write custom DSLs because of that since they were forced to decide between garbage error message when they do something wrong, suboptimal compilation output, or code duplication.

Maybe some construct which receives an expression (before type checking) and returns if it would pass type checking would work? So something like this:

fn add(T: type, a: T, b: T) T {
    if (@requires(a + b)) {
        return a + b;
    }
    else if (@requires(@as(T, a.add(b)))) {
        return a.add(b);
    }
    else {
        @compileError(std.fmt.comptimePrint("{s} doesn't support normal addition without errors", .{@typeName(T)}));
    }
}

Sure, one would need to write the usages out manually, but the same would happen if you would need to do that in a comment.

7 Likes

You can view it as extension of the type system, but you can also view it as constraining anytype. Traits would really just become comptime asserts that are standardized so that tools like ZLS can read them. That would be a huge benefit to the language. And the concrete type system can stay basically unchanged. Currently, restricted function pointers are being implemented, for something that could easily be comptime known, if we had better DX for anytype.

7 Likes

I skimmed through the text about |T|.
If I understand correctly fn (x: |T|, y: |T|) void would mean x and y are of one and the same type?
That is already a better distinction than anytype

well that in today’s zig would be one of:

  • fn (T: type, x: T, y: T) void
  • fn (x: anytype, y: @TypeOf(x)) void
1 Like

I don’t think so, because you’d be capturing the type T twice, which would be a name collision. Once the type is captured, you’re free to use it. I think what you were looking for is fn (x: |T|, y: T) void

3 Likes

Ok… understood

To be precise, this is not parametric polymorphism, but rather a restriction on the parameters to indicate what is acceptable for anytype .

I think clear boundaries are more important than false freedom. anytype allows you to pass any type to parameters, but this does not mean that the API can accept all types.

2 Likes

The new grammar is unnecessary. Just:

fn (arg: |T|) R: {
    if (!satisfySomeTrait(T)) comptime unreachable;
    break :R T;
} {
    ...
}
2 Likes

In my view, this is similar to type hints in Python, where there are concrete types, but also typing concepts, that are about constraining duck typing. Except that in the case of Zig traits, they would be enforced by the compiler, not an external tool. Both “type” systems can coexists in an single language.

1 Like

If it’s not codified by the language, different projects will use different rules, and we can’t have proper DX, because ZLS can’t parse all the variations. It would be great if it evaluated comptime, but it does not and it’s unlikely it will.

3 Likes

Syntax conventions that only target individual parameters can fall into local optima and cannot express compile-time constraint relationships among all parameters.

ZLS is still very primitive at the moment, and we cannot assume that it will always remain in its current state in the future. #615 may bring some changes in the future.