Proposal : builtin @assert() One assert to rule them all

Hi, so recently I’ve been using a lot more assertions in my code, and realized that the reason I didn’t use more of them before, was because they are in the debug namespace.

But it’s a poor argument to propose a new builtin to be part of the language, so I’ve been thinking about it more and I think it would make a lot of sense to offer the @assert() as a builtin.

I’m going to list all the reasons why I think it makes sense in no particular order.

  • assertions are important for encoding invariant, intent, and to communicates to both the reader/maintainer of the code, and also to talk to the optimizer.

  • assertions are important in many situations. To enforce correctness, to protect ourselves when refactoring. For testing purposes etc.

  • having a builtin would reduce the friction compared to using std.debug.assert (albeit there isn’t much friction in status quo but you get the gist it’s always there if it’s a builtin).

  • as a builtin it would be trivial to implement / maintain.

  • as a builtin it stands out more, which improves the readability IMHO

  • as a builtin it could lead to the feature that I’m going to explain below too.

So as you can see I think I’ve made a pretty strong case that assert is useful on it’s own but one of it’s main features is also one of it’s main drawback. As explained in the doc comment.

/// Invokes detectable illegal behavior when `ok` is `false`.
///
/// In Debug and ReleaseSafe modes, calls to this function are always
/// generated, and the `unreachable` statement triggers a panic.
///
/// In ReleaseFast and ReleaseSmall modes, calls to this function are optimized
/// away, and in fact the optimizer is able to use the assertion in its
/// heuristics.   <------ THIS RIGHT HERE IS WHAT A BUILTIN COULD FIX.
///
/// Inside a test block, it is best to use the `std.testing` module rather than
/// this function, because this function may not detect a test failure in
/// ReleaseFast and ReleaseSmall mode. Outside of a test block, this assert
/// function is the correct function to use.
pub fn assert(ok: bool) void {
    if (!ok) unreachable; // assertion failure
}

because assert is in status quo just a regular function it therefore is constrained by the semantic of regular functions. As such while very helpful in the context of all the things that I’ve mentioned above it can lead to some issues.

Because it’s being used by the optimizer as a way to say “I guarantee this invariant will not be violated, so optimize from that assumption”. It means that you can’t necessarily observe the assertions in Release mode.

take this very small example.

const std = @import("std");
const assert = std.debug.assert;

const MyUnion = union(enum) {
    foo: u8,
    bar: u8,
};

fn undefinedBehavior(foo: MyUnion) void {
    assert(std.meta.activeTag(foo) == .foo);
    std.debug.print("Hello World!\n", .{});
}

pub fn main() !void {
    undefinedBehavior(.{ .bar = 'a' });
}

Here is the current status quo for the different release mode when running this code.

In Debug :

v0 ) ./proposal_base
thread 58503 panic: reached unreachable code
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/debug.zig:546:14: 0x104911d in assert (proposal)
    if (!ok) unreachable; // assertion failure
             ^
/home/pollivie/workspace/github/proposal/src/main.zig:10:11: 0x10df73e in undefinedBehavior (proposal)
    assert(std.meta.activeTag(foo) == .foo);
          ^
/home/pollivie/workspace/github/proposal/src/main.zig:15:22: 0x10df712 in main (proposal)
    undefinedBehavior(.{ .bar = 'a' });
                     ^
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/start.zig:671:37: 0x10df6d0 in posixCallMainAndExit (proposal)
            const result = root.main() catch |err| {
                                    ^
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/start.zig:282:5: 0x10df29d in _start (proposal)
    asm volatile (switch (native_arch) {
    ^
???:?:?: 0x0 in ??? (???)

Pretty nice :slight_smile:

in ReleaseSafe

v0 ) ./proposal_safe
thread 58587 panic: reached unreachable code
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/debug.zig:546:14: 0x1046fd8 in main (proposal)
    if (!ok) unreachable; // assertion failure
             ^
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/start.zig:671:37: 0x103bd62 in posixCallMainAndExit (proposal)
            const result = root.main() catch |err| {
                                    ^
/home/pollivie/zig/0.15.0-dev.369+1a2ceb36c/files/lib/std/start.zig:282:5: 0x103b87d in _start (proposal)
    asm volatile (switch (native_arch) {
    ^
???:?:?: 0x0 in ??? (???)

Still usefull :slight_smile:

But if you use ReleaseSmall or ReleaseFast

v0 ) ./proposal_fast

It hangs, and you have no idea what the problem is. But if @assert() was a builtin we could probably control it’s semantic through the build system or through the use of std.options, or directly within the builtin itself with an enum like

pub const AssertOptions = enum {
    optimize, // still the default
    debug,    // use this in the optimizer, but don't remove the function.
    maybe,    // use for testing triggering assertions randomly helps fuzzer
    throw,    // throw an error instead of triggering unreachable (idk)
};

Obviously I’m not sure if it’s entirely feasible to feed the optimizer the assertion, without removing it, or if making sure it stays is entirely possible. But the idea would be to have a better assertion. One that isn’t constrained by the semantic of regular function,

This builtin could also be a way to unify assertion between comptime/runtime. It could be a way for the compiler to insert instrumentation, it could help with tooling, static analysis tools, etc. And it’s also just more convenient and visible.

Also the maybe mode could be used by the fuzzer to bypass the assertions to reach deeper.

Anyway TLDR; assert is cool but @assert() would be even cooler in my opinion, and I’d love to hear what everyone thinks about this idea ?

ps : I’m making this post as to not propose on github.

5 Likes

std.debug.assert is not the only way to assert, but there are other ways like this:

switch(value) {
    0 => {
        // do sth
    },
    1 => {
        // do sth

        const foo = slice[0]; // asserts that slice.len > 0
        const bar = foo - 1; // asserts that foo > 0
    },
    else => unreachable, // asserts, that value is 0 or 1
}

And unreachable is a language keyword so in my opinion there is no need for a new builtin.

This seems very similar to the option between ReleaseFast and ReleaseSafe (or @setRuntimeSafety).

2 Likes

Exactly, std.debug.assert is not the only way we can assert, but that’s why a builtin would make sense, on top of that the current issue I have with assert being a function, is that it’s only a function.

When I think I’ve made a strong argument that it could be more if it wasn’t simply a function, but instead a builtin. At least that’s how I see it. It would be a fairly simple builtin. It’s very easy to understand and it could potentially help in a lot of fronts (testing/tooling/debugging/convenience/instrumentation) etc.

Also I feel like it follows Zig zen pretty well, “one obvious way to do each things”. “readable”, “maintainable”. It seems like a free win for me

1 Like

I would have rather have fewer builtins than more.

As for assert functionality, for ‘real world projects’ assert shouldn’t be as hardwired as it is now, ideally I’d want to override the standard assert implementation at the top-level of a project to implement things like opening a UI dialog box, or sending crash information to a remote tracker like Sentry). A builtin might make that harder than a std implementation.

Also ideally there should be multiple assert variants controlled by the call site, like an assert_release() which stays active in release mode (and not replaced with unreachable, and there should be an assert_fmt() variant with a human readable error message.

All the optimization-hint stuff is in the unreachable keyword and not directly related to assert.

In C projects, asserts are also useful as hint for the static analyzer (e.g. in a static analyzer run, assert() is replaced with __builtin_assume (or other C compiler’s equivalent). Not sure if the unreachable in Zig is also ‘detected’ by static analyzers.

In the end ‘unreachable-in-release’ mode is the least desired feature of assert for me. I want most asserts to also be active in release mode, since the most “interesting” bugs don’t show up during inhouse testing, but out in the wild. And asserts that are active in release mode are absolutely critical for catching and investigating such bugs. OTH I do want enough control over asserts to be turned into an unreachable optimization hint in some specific places in the code (inner loops and stuff).

6 Likes

Exactly I completely agree with you, but that’s why a builtin would make sense in my opinion. for example instead of an enum the builtin could take a condition and a tagged union, and the tagged union could have the variant you want, and for example the assert_fmt could simply be a variant. witch takes a pointer to a function and or something like that.

The point begin that a builtin in my opinion is a better way than a std function, because it abstract away the multiple ways we could use assert into a single builtin.

Because as I see it, a builtin could make assertion more versatile, more explicit, it could help with fuzzing, testing, debugging, I mean the cost/benefit seems really good, and a builtin is cheap, it’s not like a tremendous amount of change, and you can still choose to ignore it if you want to.

pub const AssertOptions = union(enum) {
    optimize: void, // still the default
    debug: void, // use this in the optimizer, but don't remove the function.
    maybe: bool, // use for testing triggering assertions randomly helps fuzzer
    throw: error{AssertionFailure}, // throw an error instead of triggering unreachable (idk)
    instrument: struct {
        ctx: *anyopaque,
        callback: *const fn (ctx: *anyopaque) void,
    },
};

// @assert(cond : bool, opts : AssertOptions);
fn builtinAssert(cond: bool, opts: AssertOptions) !void {
    if (!cond) {
        switch (opts) {
            .throw => return error.AssertionFailure,
            .maybe => |value| if (value) unreachable else return,
            .instrument => |s| {
                s.callback(s.ctx);
            },
        }
    }
}

Probably not the abstraction you want, but you can overwrite the panic functions including the reachedUnreachable function, that gets called in Debug / ReleaseSafe / @setRuntimeSafety(true).

6 Likes

Hmm, the flexibility I’d want could probably be both implemented as builtin and as std function. I just don’t like the code littered with @ noise, there’s already too much just for integer type casts :wink:

E.g. even if the ‘core assert’ is a builtin I’d probably still want std wrapper functions around it.

The way I see most builtins is that they should be low-level building blocks that sit directly on top of LLVM builtins and basically translate to an LLVM IR instruction (e.g. very low level abstractions over CPU specific features).

I would also prefer @as, @intCast etc… to be keywords tbh, because these are clearly ‘language features’, while something like @popcount or @cmpxchgWeak are ‘typical builtins’

4 Likes

I understand and that’s why a builtin would make sense, because my argument is that assert should be more than a regular function because there are cases where it would make sense for it to be treated differently by the compiler’s internal

const std = @import("std");
const builtin = @import("builtin");

pub const AssertOptions = union(enum) {
    optimize, // still the default
    debug, // use this in the optimizer, but don't remove the function.
    maybe, // use for testing triggering assertions randomly helps fuzzer
    instrument: struct {
        ctx: *anyopaque,
        callback: *const fn (ctx: *anyopaque) void,
    },
};

// @assert(cond : bool, opts : AssertOptions);
fn assert(cond: bool, opts: anyerror!AssertOptions) !void {
    if (!cond) {
        switch (try opts) {
            .optimize => unreachable,
            .maybe => if (builtin.is_test) return else unreachable,
            .instrument => |s| {
                s.callback(s.ctx);
            },
            .debug => @panic("assertion failed"),
        }
    }
}

pub const Logger = struct {
    stdout: std.fs.File = std.io.getStdOut(),

    fn logAssert(ctx: *anyopaque) void {
        var self: *Logger = @ptrCast(@alignCast(ctx));
        _ = self.stdout.write("Assertion Failed\n") catch unreachable;
    }
};

pub const BarError = error{BarNotSeven};

pub fn main() !void {
    var ctx: Logger = .{ .stdout = std.io.getStdOut() };
    const foo: usize = 4;
    const bar: usize = 9;

    const instrument: AssertOptions = .{
        .instrument = .{
            .callback = Logger.logAssert,
            .ctx = &ctx,
        },
    };

    try assert(bar == 7, BarError.BarNotSeven);
    try assert(bar > 10, instrument);
    try assert(bar > 10, .maybe);
    try assert(foo == 4, .optimize);
    try assert(foo > 5, .debug);
}

Yes you can already achieve most of what I’ve said in user land but for some features, you can’t really do it without having the compiler using a builtin to translate your intent.

All these special features seem weird to put into the builtin, especially given that the main motivation was to make it less verbose to write. E.g. compare

@assert(someCondition, error.SomeError);
if(!someCondition) return error.SomeError;

@assert(someCondition, .{.instrument = .{. ctx = ctx..., .callback = function...}});
if(!someCondition) function...(ctx...);

@assert(someCondition, .maybe);
myCustomTestingAssert(someCondition);
@assert(someCondition, .debug);
myCustomReleaseAssert(someCondition);

I do however agree on the root issue. Assertions are relatively hard to access, but in my opinion that is mostly a tooling issue (since we don’t have automatic imports in ZLS).

In my opinion a better short-term solution would to just put it in std.assert to avoid some of the friction while not adding yet another builtin.

A big advantage of having it in std is that you can always just edit the source code yourself. If you just want to temporarily enable all assertions in fast builds you can do that with a small change the standard library and revert it afterwards. As a builtin you lose that power.

7 Likes

I mean with that system std.debug.assert would become

pub fn assert(ok : bool) void {
    @assert(ok, .optimize);
}

or it could stay the same, I don’t think you loose much, by allowing it to be a builtin, And it doesn’t have to be a builtin that takes a tagged union, it could be something else, I’m just sharing features, that it could help implement.

But it could also just be like the math functions as builtin, maybe some platform as a specific way to optimize assertions, maybe some backends too.

I’m not arguing for the necessity of assert being necessarily in the form of a condition and a tagged union or enum. All I’m saying is that from a compiler and user standpoint assertions are not in my opinion regular functions.

Take for example this enhance the fuzzing algorithm to be competitive with other mainstream fuzzers #20804

One can easily imagine that the assert as a builtin could help to guide/target/discard the fuzzer.

In the world of embedded, assertions on device are not necessarily useful, and being able to intercept them and wire them to uart would be nice. Not saying you can’t in some shape of form do that already, but a builtin could help depending on the form it takes.

On a distant process, assertions could be used to trigger a callback, to log informations, or do XYZ.

The PGO could use the assertions to insert instrumentations, etc.

Tools could see in the ast and special case the builtin assert to infer some stuff about the program.

I can’t say what’s the best way to implement it as a builtin, but there are a ton of thing that special casing assert could offer, that isn’t necessarily available or doable in user land

Or it could simply be @assert(ok : bool) I still think it would be useful like this simply because it’s more visible, and could help internally.

I think the use cases you describe here should be handled by the error path rather than assertions, but maybe I am misunderstanding the purpose of assertions?

My understanding of assert is that it should always evaluate to true. If it does not, something that should not happen happened and your program is in an undecidable state. So it should just crash - and that’s what it does.

The crash might trigger a report to Sentry or whatever, but that’s outside of your program’s duty.

On the other hand, if your assert failure is common enough that it makes sense to open a UI dialog box, it means it is part of the normal behaviour of your application and should be handled by the error path somewhere.

15 Likes

Yes, errors are for situations that you expect to happen, while asserts are for “can’t happen” cases. But in the real world, “can’t happen” happens surprisingly often, especially after release when thousands of users are hammering your application. And for those situations a ‘controlled crash’ via an assert and getting some information out for post-mortem debugging is incredibly helpful (and if it’s just a callstack or minidump) - and such a controlled ‘assert-crash’ is much more useful than a regular crash, because you are most likely much closer to the cause of the bug.

E.g. the usual ‘action’ after you receive such an assert crash report is to fix the bug that caused the assert (and keep the assert in so that the bug doesn’t regress), or turn it into a ‘proper’ handled error (when it turns out that what was supposed to be a “can’t happen” case can actually happen for a valid program).

Tbf, this could be restricted to ‘release-safe’ mode, while ‘release-fast’ turns triggered asserts into undefined behaviour via unreachable. But it would be nice to control this case by case, not via a global compiler option.

So it should just crash - and that’s what it does.

…I think this isn’t the case in release mode? If the unreachable behaves like it does in C, literally anything can happen, since the compiler uses this as optimization hint (e.g. the compiler may assume that the unreachble is indeed never reached and use this for various ‘dangerous’ optimizations). If you’re lucky the compiler might insert an ud instruction which would crash the process, but at least in C this isn’t guaranteed (because in that case the compiler would need to check if the unreachable is actually reached, which would defeat the purpose of enabling optimizations, since the whole point of those optimizations is to skip those checks).

3 Likes

You can control this case by case with @setRuntimeSafety(safety_on: bool).

2 Likes

You can control this case by case with @setRuntimeSafety(safety_on: bool) .

…which would be a good reason to have different wrapper functions around the core-assert (although it’s debatable whether those should be in the stdlib, or whether I just write my own assert-wrappers).

In any case, it’s tricky to “inject” such wrappers into external dependencies. Ideally I could inject assert implementations similar to allocators.

2 Likes

Some examples of why I’m fine with std.debug.assert instead of an @assert:

pub fn assertAlways(ok: bool) void {
    if (!ok) {
        @branchHint(.cold); // might not need this:
        @panic("Assertion failed");
    } 
}

pub fn extraAssert(ok: bool) void {
    if (config.extra_assertions) assertAlways(ok);
}

I’ve thought about putting an assert library together which would bulk out this kind of thing. Maybe there should be a std.assertion library which does so, and std.debug.assert could just be std.assert. Although now we’re saving six keystrokes…

Really the built-in assert is one line of code. It’s literally if (!ok) unreachable;, and without adding it to the namespace we save:

std.debug.assert(5 == v);
if (5 != v) unreachable;

…negative one keystrokes, or plus one/two for the general case, which can need an extra set of parentheses.

There are just a lot of ways to tackle this problem, all of them pretty easy to just write when you need them. I think a complex builtin with various ways to tune what it does is not the way forward.

9 Likes

I couldn’t disagree more. Something has gone wrong if people are writing their own wrapper functions for assert.

With the combination of optimization mode, which is independently configurable per module, as well as in any scope, and the ability to provide panic handlers, there’s absolutely no reason to avoid std.debug.assert. If you avoid that function then you also should avoid unreachable, which is not possible, as it’s baked into the language all over the place, such as in slice element access. If you want to handle assertion failures robustly then you must use a panic handler.

unreachable is core to the language. It communicates something very precisely, and the behavior of it must be controlled by settings for what to do in any given site when an unreachable code path is reached. To do otherwise is to fight an uphill battle with the language that may as well be a vertical wall.

Zig is designed to be a reusable language. This means that the idiomatic way to write code is independent of whether it is a “real world project”. unreachable, and thereby std.debug.assert, have been carefully designed to uphold this principle.

Think about it this way: if you’re writing a package for applications to use, then you would not override std.debug.assert, so that applications can provide the panic handler and do error reporting. It would be inappropriate for the package to do that on the application’s behalf. So then, you can also use std.debug.assert in the application code, and take advantage of the same code path.

Related

14 Likes

Maybe you are right, I was convinced It was a cool idea, one that could potentially be used to solve or offer opportunities for improvements. I really do believe that assertions are specials. And like I’ve mentioned I do believe there is an opportunity to improve it. Not from a writing perspective that I don’t really care about, saving keystrokes is not important to me.

But It would be cool to have a way to interact with assertions and use them in more ways than currently available.

1 Like

My bias here is that, while I’ve definitely considered making fancy asserts real (you can’t even print a message! etc.), in practice I’ve only ever added one custom assertion: debugAssert. Which just checks specifically for .debug, rather than asserting in both safe modes. I use it for assertions I expect would make things too slow for users, and furthermore I only expect could be triggered by changes to the code rather than unexpected inputs.

I use even that sparingly, because like @setRuntimeSafety, it alters the contract with user code, which is acceptable but which should always be done judiciously.

I do think a rich assertions library would be interesting, and stdlib has incorporated libraries before once they’ve proved themselves in the wild. I haven’t written it myself, ultimately, because I don’t think I would bother importing it into my own projects, or at least not most of them.

The thing I’m surest about is that assert doesn’t need to be a builtin, the way @typeInfo simply has to be a builtin. So you would be able to fulfill your vision of how it should work by writing a module / library.

3 Likes