More thoughts on unusable structs

The idea of unusable structs came up during an earlier discussion about conditionally including functions in a namespace.

The problem

Sometimes a function is not available due to build settings:

pub const foo = switch (builtin.single_threaded) {
    true => @compileError("Not available in single-threaded environment"),
    false => fooAsync,
};

Using @compileError() in this manner is fairly common. It has the drawback of making it impossible to introspect the code. The moment you try to determine what a particular decl is, compilation dies on you. So another way of dealing with unavailability is also used:

pub const foo = switch (builtin.single_threaded) {
    true => {},
    false => fooAsync,
};

This allows for introspection but at the expense of error reporting:

error: type 'void' not a function

Here we have no idea why something we expect to be a fn is a void instead.

The solution

During our discussion, someone came up with the idea of using a comptime defined struct. It takes advantage of the fact that arguments passed to a function defining a type show up in the type name:

pub const foo = switch (builtin.single_threaded) {
    true => unavailable("Not available in single-threaded environment"),
    false => fooAsync,
};

fn Unavailable(comptime msg: []const u8) type {
    return struct {
        comptime reason: []const u8 = msg,
        comptime kind: @TypeOf(.enum_literal) = .unavailable,
    };
}

fn unavailable(comptime msg: []const u8) Unavailable(msg) {
    return .{};
}

Now when we try to call foo(), we get the following error message:

error: type 'unusable.Unavailable("Not available in single-threaded environment"[0..44])' not a function

We can add other fields to the struct to help explain the circumstance. Their presence can be inferred based on kind during introspection.

Usage scenarios

Scenarios where something is expected to be there but is in fact not there are actually common. After thinking about this for a while, I came up with the following types of “unusables”:

Unavailable - Function (or type) is not available due to build target or other settings.

Todo - Function isn’t yet implemented for the target. Struct should contain an issue number so programmers can track the progress of its implementation (or volunteer to do it themselves).

Removed - Function has been removed. An issue number here would be helpful too.

Moved - Function has been moved to somewhere else in the namespace. Struct should provide new address. Docgen can then use it to create a link.

Deprecated - Function is slated to be removed or significantly altered. Struct should provide the old version in a field called @"FIXME!". This allows programmers to very quickly get their programs working again after a Zig version bump. After making such temporary fixes, they can then deal with deprecated features one at a time.

If a new version is supposed to take over that same namespace location, it will be held in the field @"NEW!" for the time being.

Untranslatable - A C macro cannot be translated into a Zig function. Struct should provide path of C file in question plus line number.

Uncertain - Used in RPC implementation to indicate that there’s no guarantee of execution. A return value of Uncertain doesn’t imply success. The call is probably sitting in a queue and can fail in the future (unbeknownst to the caller).

Implementation

A key question is how to determine whether a struct represents an unusable value. We can infer from the presence of reason and kind. Or we can stick in a third field holding a private enum. Another possibility is to add a item to std.builtin.Type.ContainerLayout. Having support at the language level is justifiable, I think, since usuable values are distinct from regular structs. We can do without that initially, of course.

7 Likes

I feel like it would be better to have some kind of “comptime conditional declaration” syntax, of sorts. E.g.:

const x = comptime {
if (!build.single_threaded) {
@unavailable("Not available in single-threaded mode");
} else {
struct {
// ...
};
}
};

Idk how this would work exactly, but my idea would be: you lazily define the struct/enum/whatnot. The compiler only executes the comptime block if you actually reference the type somewhere else. In that way, we can change error messages so that the compiler provides context as a part of the error message, instead of giving you a code sample string literal mix.

1 Like

I just thought of a way to explain my idea–with the help of Leo Tolstoy.

All happy families are alike; each unhappy family is unhappy in its own way.

So begins Anna Karenina. Instead of families, what’s at issue here are different sorts of nothingness. The Zig type void represents the “happy nothingness”. When a function returns it, it means things are happening in accordance with plans. It’s always the same.

Meanwhile, unusable structs are “unhappy nothingness”. Some unhappy circumstances have led to nothing existing at their particular locations. They are unhappily missing in their own ways. Comptime fields contained within them allow us to learn more about their unhappy circumstances–like Tolstoy’s novel.

2 Likes

I still feel that defining a custom struct just to work around not being able to reflect on decls initialized to @compileError() is a local maximum.

I think the proper fix is to make @compileError() errors (and possibly more/all classes of semantic compile errors) lazy, only surface once the value is actually used and provide some kind of means of detecting that use of a decl will be a compile error (think of something similar to @hasDecl()). The compiler already tracks if comptime-known values are undefined and fails with

use of undefined value here causes illegal behavior

so while I could be wrong, from what I’ve seen when digging through the compiler code it looks like the foundational infrastructure required to handle something like this already exists.

As an aside, some form of “lazy” compile errors is more or less a prerequisite for implementing @deprecated().

2 Likes

Why would we want to use @compileError() when no actual error occurred? Calling a function that’s been moved isn’t an error made by the programmer. The programmer who moved the function didn’t err in doing so either. The function is not there anymore because circumstances prevented the function from being at the right place in the namespace from the start.

As I tried to explain, what we’re dealing with here is unhappy nothingness. Like going to your local Chipotle only to find an empty store front. The priority in this situation is not to diagnose what has led you to go to that particular location. Figuring out why you had made your way to a defunct Chipotle helps absolutely no one. The priority is get you to the nearest Chipotle so you don’t starve. A note on the door would be helpful.

When the programmer actually commits an error, then it makes sense to point that out using @compileError(). If somehow we can unify error handling at comptime and and runtime, that’s great. But there’s really no point in overcoming a local maximum only to end up with something conceptually dubious.

I think the simplest way would be to have a concrete comptime type like Unavailable below, defined as part of the standard library.

Unavailable can have a message and it uses ComptimeAny to hold an arbitrary comptime value for further information (based on what makes sense to represent the reason), you can print that value or write comptime code that reflects on its fields.

ComptimeAny works in a similar way like the default value of a StructField, pairing a type and a *const anyopaque.

With that we have a type that works at comptime, however we need unavailable.ctx() which creates an instance of Unavailable.Msg(<msg>) to get the compile error when somebody tries to use the value at runtime where something else was expected (to have the message appear in the compile error), for that we comptimePrint the Unavailable-instance to a string literal.

But now our Unavailable.check function needs to return true for those Msg instances (so that we can check for unavailable things at comptime), for that we check whether it looks like a Msg and if so, we access and reconstruct Msg and then compare against the reconstructed generic type.

So here is the basic sketch, using the scenarios described by @chung-leong:

pub fn main() !void {
    // _ = &foo(); // with -fsingle-threaded results in:
    // unavailable.zig:2:10: error: type 'unavailable.Unavailable.Msg("Unavailable: Not available in single-threaded environment\n\rValue:\n\r    .{ .reason = .needs_concurrency }\n\r"[0..106])' not a function
    //     _ = &foo();
    //          ^~~
    if (comptime Unavailable.check(&foo)) {
        std.debug.print("functionality is not available\n", .{});
        std.debug.print("{f}\n", .{&foo});
    } else {
        std.debug.print("{*}\n", .{&foo});
    }
    example(.valMsg(TODO, "Add support for color escape sequences"));
    example(.val(Removed{ .issue = 1234 }));
    example(.val(Moved{ .to = "std.new_location.Here" }));
    example(.val(Deprecated(std.array_list.Managed, std.ArrayListUnmanaged)));
    example(.val(untranslatable("some_header.h", 25, 15)));
    example(.valMsg(Uncertain, "could take between 1 and 122 business days"));
}

pub fn example(comptime unavailable: Unavailable) void {
    std.debug.print("{f}\n", .{unavailable.ctx()});
}

pub const foo = switch (builtin.single_threaded) {
    // true => Unavailable.msg("Not available in single-threaded environment"),
    true => Unavailable.msgVal(
        "Not available in single-threaded environment",
        NotAvailableInSingleThreaded{ .reason = .needs_concurrency },
    ).ctx(),
    false => fooAsync,
};

pub const TODO = Unavailable.Prefix("TO" ++ "DO "){};

pub const Removed = struct {
    pub const mode: Unavailable.Mode = .prefix_value;
    pub const prefix = "Removed ";

    issue: u32,
};
pub const Moved = struct {
    pub const mode: Unavailable.Mode = .prefix_value;
    pub const prefix = "Moved ";

    to: []const u8,

    pub fn format(
        self: Moved,
        writer: *std.Io.Writer,
    ) std.Io.Writer.Error!void {
        try writer.print("to: {[to]s}", self);
    }
};

pub fn Deprecated(comptime old: anytype, comptime new: anytype) type {
    return struct {
        pub const @"FIXME!" = old;
        pub const @"NEW!" = new;
    };
}

pub const Untranslatable = struct {
    pub const mode: Unavailable.Mode = .prefix_value;
    pub const prefix = "Untranslatable ";

    pub const Location = struct {
        file: [:0]const u8,
        line: u32,
        column: u32,
    };
    loc: Location,

    pub fn format(
        self: Untranslatable,
        writer: *std.Io.Writer,
    ) std.Io.Writer.Error!void {
        try writer.print("{[file]s}:{[line]}:{[column]}", self.loc);
    }
};
pub fn untranslatable(file: [:0]const u8, line: u32, column: u32) Untranslatable {
    return .{ .loc = .{ .file = file, .line = line, .column = column } };
}

pub const Uncertain = Unavailable.Prefix("Uncertain: "){};

pub const NotAvailableInSingleThreaded = struct {
    pub const Reason = enum { needs_concurrency, not_implemented };
    reason: Reason,
};

pub fn fooAsync() !void {}

//////////////////////////////
// impl below
pub const Unavailable = struct {
    pub fn Msg(comptime reason_: []const u8) type {
        return struct {
            pub const reason = reason_;
            unavailable: Unavailable,

            pub fn format(
                self: @This(),
                writer: *std.Io.Writer,
            ) std.Io.Writer.Error!void {
                try writer.print("{f}", .{self.unavailable});
            }
        };
    }
    pub fn ctx(comptime self: Unavailable) Msg(self.str()) {
        return .{ .unavailable = self };
    }

    pub const Mode = enum {
        value,
        prefix,
        prefix_value,

        pub fn showPrefix(m: Mode) bool {
            return switch (m) {
                .prefix, .prefix_value => true,
                else => false,
            };
        }
        pub fn showValue(m: Mode) bool {
            return switch (m) {
                .value, .prefix_value => true,
                else => false,
            };
        }
    };

    message: []const u8,
    any: ComptimeAny,

    pub fn valMsg(comptime value: anytype, msg_: []const u8) Unavailable {
        return .msgVal(msg_, value);
    }
    pub fn msgVal(msg_: []const u8, comptime value: anytype) Unavailable {
        return .{ .message = msg_, .any = .init(value) };
    }

    pub fn msg(msg_: []const u8) Unavailable {
        return .msgVal(msg_, {});
    }

    pub fn val(comptime value: anytype) Unavailable {
        return .msgVal("", value);
    }

    pub fn format(
        self: Unavailable,
        writer: *std.Io.Writer,
    ) std.Io.Writer.Error!void {
        const mode: Mode = comptime if (hasDecl(self.any.type, "mode")) self.any.type.mode else .value;
        const multiline = std.mem.count(u8, self.message, "\n") > 0;

        try writer.writeAll("Unavailable: ");
        if (comptime mode.showPrefix()) {
            if (comptime !hasDecl(self.any.type, "prefix"))
                @compileError("expected type '" ++ @typeName(self.any.type) ++ "' to have a declaration 'prefix' of type []const u8");
            try writer.print("{s}", .{self.any.type.prefix});
        }
        if (multiline) {
            var it = std.mem.splitScalar(u8, self.message, '\n');
            while (it.next()) |line| {
                try writer.print("\n\r    {s}", .{line});
            }
        } else {
            try writer.print("{s}", .{self.message});
        }
        if (comptime mode.showValue()) {
            // try writer.print("{s}\n\r", .{@typeName(self.any.type)});
            try writer.writeAll("\n\rValue:\n\r    ");
            try writer.print("{f}\n\r", .{self.any});
        }
    }
    pub fn str(comptime self: Unavailable) *const [std.fmt.count("{f}", .{self}):0]u8 {
        return std.fmt.comptimePrint("{f}", .{self});
    }

    pub fn Prefix(comptime prefix_: []const u8) type {
        const Impl = struct {
            pub const mode: Unavailable.Mode = .prefix;
            pub const prefix = prefix_;
        };
        return Impl;
    }

    pub fn check(v: anytype) bool {
        const T = @TypeOf(v);
        return switch (@typeInfo(T)) {
            .type => v == Unavailable,
            .pointer => |p| if (p.child == Unavailable) true else (if (p.size == .one) check(v.*) else check(p.child)),
            // zig fmt: off
            .@"struct" => T == Unavailable or
                (hasField(T, "unavailable") and
                 @FieldType(T, "unavailable") == Unavailable and
                 T == @TypeOf(v.unavailable.ctx())),
            // zig fmt: on
            else => false,
        };
    }
};

pub const ComptimeAny = struct {
    type: type,
    ptr: *const anyopaque,

    pub fn init(comptime val: anytype) ComptimeAny {
        return .{ .type = @TypeOf(val), .ptr = @ptrCast(&val) };
    }

    pub fn value(self: ComptimeAny) self.type {
        const val: *const self.type = @ptrCast(@alignCast(self.ptr));
        return val.*;
    }

    pub fn format(
        self: ComptimeAny,
        writer: *std.Io.Writer,
    ) std.Io.Writer.Error!void {
        if (comptime hasDecl(self.type, "format")) {
            try writer.print("{f}", .{self.value()});
        } else {
            try writer.print("{any}", .{self.value()});
        }
    }
};

pub fn hasField(comptime T: type, comptime name: []const u8) bool {
    return switch (@typeInfo(T)) {
        .@"struct", .@"enum", .@"union" => @hasField(T, name),
        else => false,
    };
}
pub fn hasDecl(comptime T: type, comptime name: []const u8) bool {
    return switch (@typeInfo(T)) {
        .@"struct", .@"enum", .@"union", .@"opaque" => @hasDecl(T, name),
        else => false,
    };
}

const std = @import("std");
const builtin = @import("builtin");

And this is the output:

fn () @typeInfo(@typeInfo(@TypeOf(unavailable.fooAsync)).@"fn".return_type.?).error_union.error_set!void@113f8b0
Unavailable: TODO Add support for color escape sequences
Unavailable: Removed 
Value:
    .{ .issue = 1234 }

Unavailable: Moved 
Value:
    to: std.new_location.Here

Unavailable: 
Value:
    unavailable.Deprecated((function 'Managed'),(function 'ArrayList'))

Unavailable: Untranslatable 
Value:
    some_header.h:25:15

Unavailable: Uncertain: could take between 1 and 122 business days

When run with zig run unavailable.zig -fsingle-threaded the first line is:

functionality is not available

and the rest is the same, because those unconditionally print the different Unavailable values.


I think the specific protocol for how Unavailable prints values and what kinds of modes it supports could still be changed a lot, it is just what I came up with on the spot.

3 Likes

Someone on the discord had the same problem.

We decided on a pub const NotDefined = struct {}, a zero sized type, distinct from void, and with a clear name.

Your solution has a big issue of different types per msg, this makes it very unusable as you can’t just compare the type of the field/decl, you have to introspect the type deeper.

I prefer a simpler solution of just separate types for each use case

const Unavailable = struct{};
const Todo = struct{};
const Removed = struct{};
const Moved = struct{};
const Deprecated = struct{};
const Untranslatable = struct{};

I dont see why Uncertain should be in std, seems very use case dependent.

1 Like

It doesn’t really have to be.

You can leave out the Uncertain.Msg part and just use Uncertain which is a single type, that just means that the error message doesn’t show up in compile errors anymore.


I am wondering what if the compiler looked for a compileError function that somehow can add additional context when the type is involved in a compile error? Basically like a custom format function protocol, but instead an add compile error context protocol. Would have to think deeper about it…

Can we draw on the ‘namespaces are good, actually’ zen?

MyStruct.zig

const category1Methods = if (condition1) struct {
    fn firstMethod(…) … { … }
    fn secondMethod(…) … { … }
    …
} else struct {};

No need to put our own compile errors here. No problem with walking the decls.

deprecated namespace sounds pretty sick actually. The way this would work is when the maintainer deprecates a method out of one namespace, they replace its body with @compileError("deprecated, but still available here…") and then move its implementation to deprecated namespace. Then users can upgrade by just following their nose with the compile errors, switching over to the deprecated namespace, and have a nice record in their own code that they’re using deprecated functions. I’d sure prefer that to runtime warning logs, lol

3 Likes

Do you mean my solution with that? The formatting happens at comptime because both Uncertain and Msg are comptime only types, if they are used in a runtime context, you get a compile error.

const Unavailable = @import("unavailable.zig").Unavailable;

const Vec2 = switch (true) {
    true => Unavailable.msg("Not available").ctx(),
    false => struct {
        x: i32,
        y: i32,
    },
};

pub fn main() !void {
    var point: Vec2 = .{ .x = 4, .y = 7 };
    point.x += 1;
}

// unavailable_user.zig:12:16: error: expected type 'type', found 'unavailable.Unavailable.Msg("Unavailable: Not available\n\rValue:\n\r    void\n\r"[0..46])'
//     var point: Vec2 = .{ .x = 4, .y = 7 };
//                ^~~~
// unavailable.zig:97:16: note: struct declared here
//         return struct {
//                ^~~~~~

Why would we want to use @compileError() when no actual error occurred? Calling a function that’s been moved isn’t an error made by the programmer. The programmer who moved the function didn’t err in doing so either. The function is not there anymore because circumstances prevented the function from being at the right place in the namespace from the start.

This is a nonsensical statement to me. If the user tries to call foo.bar(), and foo.bar is not a function, it produces a compile error whether or not it is declared as pub const bar = Removed("use the new foo.baz API instead"); or pub const bar = @compileError("removed: use the new foo.baz API instead");. Both provide the same amount of information to the user, and the main distinction between them is:

  • You can reflect on Removed() values but not @compileError().
  • Removed() does not provide a compile error trace leading back to the line where the value was assigned to a decl, only where its value is used.

So if your goal is to help guide the user back to the spot the error originated from, which could be dozens of calls and references deep, @compileError() is actually better than any other alternative. Once again the main flaw with it is that you can’t know if accessing a decl will result in a compile error without actually accessing it and producing the error.

As I tried to explain, what we’re dealing with here is unhappy nothingness.

@compileError(), or more specifically compile-time “values” of the noreturn type, is already the “unhappy nothingness” you speak of.

2 Likes

No, not at all. Not laughing at anybody else’s solution here, just unrelated libraries I contend with in my zigless dayjob

It sounds nonsensical to you prob ably because you’re so used to the practice in computing where we equate failures with errors. We do this because it’s true most of the time. If the computer is unable to perform something, usually it’s due to human error. Either the end user made a mistake or the programmer did. But “failure” and “error” aren’t synonymous. We have agency over whether an error or not.

Let us consider a real-world example, std.posix.symlink:

pub fn symlink(target_path: []const u8, sym_link_path: []const u8) SymLinkError!void {
    if (native_os == .windows) {
        @compileError("symlink is not supported on Windows; use std.os.windows.CreateSymbolicLink instead");
    } else if (native_os == .wasi and !builtin.link_libc) {
        return symlinkat(target_path, AT.FDCWD, sym_link_path);
    }
    const target_path_c = try toPosixPath(target_path);
    const sym_link_path_c = try toPosixPath(sym_link_path);
    return symlinkZ(&target_path_c, &sym_link_path_c);
}

As a programmer I have zero agency over this situation. I have no say in how Windows works and I’m not the author of this code. Guiding me to this spot accomplishes nothing because it’s not an error on my part. It’s not really an error at all. It’s a limitation caused by circumstances beyond everyone’s control (including possibly programmers at Microsoft).

And I can’t avoid this @compileError() even if I try. The following will not compile for Windows:

const std = @import("std");

pub fn main() !void {
    if (@hasDecl(std.posix, "symlink") and @typeInfo(@TypeOf(std.posix.symlink)) == .@"fn") {
        _ = &std.posix.symlink;
    }
}

Introspection is telling me that the function exists even though it cannot in fact exist. Should we reengineer Zig’s error system to allow for recoverable compile errors just so this sort of inaccurate representation can persist? I don’t think so.

Would you rather that no such decl is reported to exist? The code could be restructured in that way using namespacing. On the downside, you would lose the current quite nice behavior where if the developer does a first pass without regard for windows, then tries to compile for windows, they’ll get some useful error messages that will help them make the necessary windows-specific adjustments more quickly.

1 Like

In the past, we were able to use usingnamespace to exclude functions. That’s really not a great solution, because even though a function does not in the reality of the language, it could still exist in the mind of programmers. In the case of a function that’s been removed or renamed, it would still exist in tutorials floating around on the Internet. Pure nothingness is not desirable. It’s as bewildering as your favorite Chipotle vanishing into thin air with no explanation.

Depending on the situation, a decl should be kept between versions or indefinitely. Its value should be something that represents nothing and is capable of explaining why it is so. “A void with a personal sob story,” if you will.

I don’t think I understand the abstract point you’re trying to make about failures vs errors and “no actual errors occurring”. To me it almost reads like you’re afraid that the user will feel emotionally injured if their code doesn’t compile and they are faced with compile errors.

Maybe we’re misunderstanding and talking past each other. The think I want, and which I suspect you and most people want, is for the language to support all of the following:

1. There should be a way to iterate over every decl in a container and reflect on the type of each decl. A basic loop like this which prints the name and type of each decl should always work:

inline for (@typeInfo(std.posix).@"struct".decls) |decl| {
    const decl_type = @TypeOf(@field(std.posix, decl.name));
    std.debug.print("{s}: {s}\n", .{ decl.name, @typeName(decl_type) });
}

Without this, this pattern of type introspection just becomes one huge mine field. People are working around this problem by using void or dummy types is an enormous hack and shows that the tools the language gives use are insufficient.

2. If a function is implemented like

fn foo() i32{
    if (builtin.os.tag == .windows) @compileError("not supported on Windows");
    return 123;
}

and my code tries to call foo() when compiling for Windows, it should obviously fail the build with a compile error, and the compile error and trace should be useful enough that I can follow the breadcrumbs to see why foo() is called from my code and why it resulted in a compile error in the library author’s code.

3. As an extension to the above, there should be a way to programmatically detect if using (resolving the value of) any particular decl/expression will result in a compile error. This way, given a function like

fn getCurrentThreadId() u32 {
    if (builtin.single_threaded) @compileError("not supported in single-threaded builds");
    if (builtin.os.tag == .windows) @compileError("not supported on Windows");
    if (!builtin.link_libc) @compileError("not supported unless linking with libc");
    return impl();
}

the user won’t need to know and replicate every minute implementation detail about under which circumstances the function works

const thread_id: u32 =
    if (!single_threaded and target.os.tag != .windows and link_libc)
        getCurrentThreadId()
    else
        1;

, they could just do something like this (using a fictional syntax)

const thread_id: u32 = getCurrentThreadId() catch_semantic_error 1;

which is clearly better and automatically more future proof, as the path taken will automatically change as the implementation of getCurrentThreadId() changes in future versions.

Which is my entire point about lazy compile errors. If @compileError() was implemented lazily and only bubbled up at use site, and there was a way to reflect on whether or not using any particular usage is a compile error like @isCompileError(&std.posix.symlink), you would have the tools necessary to support all of the above points.

6 Likes

On the subject of the ‘unusable’ decl - I wonder if this isn’t just a better pattern -

fn Unusable(msg: []const u8) fn () noreturn {
    return (struct {
        fn do() noreturn {
            @compileError(msg);
        }
    }).do;
}

const mydecl = Unusable("removed");

The nice thing about this is -

  1. there’s only one type, namely, fn () noreturn
  2. the instance exists, touching the decl isn’t a compile error, but analyzing its body would be, so you can’t do anything with it (call it, construct a pointer to it…) without getting the msg payload
  3. unless I’m missing something, 2. is a consequence of 1. So, your generic reflection code can check whether the return type of any functions it finds is noreturn and understand that it’s not supposed to proceed, avoiding a compile error.
2 Likes

Using @compileError() in this manner is fairly common. It has the drawback of making it impossible to introspect the code.

Perhaps you’d like @compileLog()

It’s technically not made for generating compile errors, it’s more for debugging comptime code, and the compile-time error is just a side-effect. But it accomplishes what you want here :slight_smile:

@compileLog() won’t help. Analyzing @compileLog() as well as @compileError() prevents code from been compiled. The only difference is what @compileLog() doesn’t prevent compiler from printing other logs and errors.

2 Likes