I know it's wrong to use `defer` like this, but I can't resist the temptation to use it

When I output JSON, I intuitively use defer and code blocks to execute stringifier.endObject() and stringifier.endArray().
But these api themselves report errors, and I’m arbitrarily catching them and causing them to crash. This is clearly a code smell.
After some thought, I realized that using defer on them was a mistake: defer should only be used for code that needs to be executed when exiting the code block under any circumstances, such as resource release. Here, the conditions for using defer are clearly not met: if the code block exits due to an error, then endObject() and endArray() should not be executed at all.
Nevertheless, I find it hard to give up my current way of using it; it’s just too tempting because it’s so ergonomic.

json_output: {
    try stringifier.beginObject();
    defer stringifier.endObject() catch main.dumpAndCrash(@src());
    try stringifier.objectField("array_a");
    array_a: {
        try stringifier.beginArray();
        defer stringifier.endArray() catch main.dumpAndCrash(@src());
        for (a.items, 0..) |*item, idx| {
            try stringifier.beginObject();
            defer stringifier.endObject() catch main.dumpAndCrash(@src());
            try stringifier.objectField("idx");
            try stringifier.write(idx);
            try stringifier.objectField("content");
            try stringifier.write(item.content);
        break :array_a;
    }
    break :json_output;
}
2 Likes

it is generally not that big of an issue, but an okdefer would be nice.

5 Likes

Yea previously Andrew said:

so these two concepts defer errdefer they prevent any kind of that spaghetti cleanup code that that you would otherwise have to do and in practice I’ve observed that there’s no other cases these handle all of the all of the possible situations

And it seems like we come up with a new pattern which he didn’t think of then implementing defer, errdefer. This pattern is now especially common with Writer flushing.

4 Likes

I’m tempted to say that you might be misattributing the ergonomics to defer when it actually might just be the blocks that are making this nice. Here’s a version without defer that still seems pretty nice to me:

{
    try stringifier.beginObject();
    try stringifier.objectField("array_a");
    {
        try stringifier.beginArray();
        for (a.items, 0..) |*item, idx| {
            try stringifier.beginObject();

            try stringifier.objectField("idx");
            try stringifier.write(idx);
            try stringifier.objectField("content");
            try stringifier.write(item.content);

            try stringifier.endObject();
        }
        try stringifier.endArray();
    }
    try stringifier.endObject();
}
6 Likes

to be fair statement in quotes is still correct. okdefer would be only used for correctness and not for cleanup.

2 Likes

I see two potential problems with okdefer:

  • If it does not allow errors, then using it for stuff like flush is not a good idea
  • If it does allow errors, then control flow gets rather confusing

Consider something like:

okdefer try out.flush();

// ...

errdefer foo.deinit();

if (something) {
    return;
}

errdefer bar.deinit();

If flush fails, then control flow would need to jump back down to the relevant return point and restart evaluating the errdefers in reverse order.

Or, even worse:

okdefer try out.flush(); // if this fails, foo() already ran!

// ...

okdefer foo();

I think this is the most relevant proposal (now closed):

4 Likes

That is an issue with all defers, and it has a good reason to be, as you pointed out.

In such cases, it is far more readable to do the control flow normally. Though it is more tedious.

I dont think it’s such a serious issue to warrent the addition of okdefer to the language.
It’s just a thought I have sometimes, which stems from the desire to make out jobs as programmers easier. ‘but at what cost :face_with_open_eyes_and_hand_over_mouth::stuck_out_tongue:

3 Likes

To a large extent, my fondness for defer stems from the fact that its use forces me to use code blocks to make my code clearer. However, pure code blocks are not common in C, which is accustomed to a flatter style, making it feel slightly odd to me; I suppose this is a stylistic flaw in C.

Nevertheless, using defer and code blocks to notify in advance that symmetric operations will execute at the end of the block does give me peace of mind when checking my code, especially when the block is long. Sometimes I wonder if all instances of defer not intended to ensure resource release are actually being abused.

I wholeheartedly agree that okdefer inherently contains pitfalls if it allows the use of try statements, especially when there is already a return value:

okdefer try w.flush();
...
return success_value;

In this situation, there is already a return value. What does it mean if okdefer encounters an error? Does it overwrite the original return value? This is an implicit behavior that is not well defined.
Suppose there is a breakdefer that allows it to exit with a break value.

const x = block_outside: {
    const y = block_inside: {
        // breakdefer:block_inside break :block_inside y_value; // it is always wrong!
        breakdefer:block_outside break :block_outside  x_value;
        ...
        if (condition1)  break :block_outside x_value; // breakdefer not execute
        if (condition2) return ret; // breakdefer not execute
        break :block_inside y_value; // breakdefer execute
    }
    break :block_outside x_value;
}

I’ve reconsidered the relationship between breakdefer and the statement that causes a code block to exit.
Each breakdefer statement must pre-specify a code block (or the entire function). If the statement that causes the code block to exit reaches or exceeds the content outside the code block, the breakdefer statement cannot be executed.
breakdefer should never break value to the current code block; this is always wrong.
If breakdefer breaks a value, such as x, to an outer code block, then if the statement that causes the code block to exit breaks value to a more inner code block(y), then breakdefer will execute successfully. However, if the statement that causes the code block to exit breaks into the same code block as breakdefer, or into a more outer code block (e.g., a direct return), then the breakdefer statement will not execute.

This is the only well-defined defer that I can think of that allows returning error values. It just doesn’t look nice.

What errors are expected to happen and how else are you going to handle an error, if not by crashing?
Without knowing this, it’s impossible to tell the “best” way to writing things. You code looks correct to me. If something goes wrong, end everything with a stack trace. That seems acceptable to me.

I don’t see how this is more correct than OP’s version. If something goes wrong, you don’t end the objects. Is that acceptable? I don’t know what begin and end are doing here, but it looks like not ending it is a leak. Wouldn’t it be better to just crash?

I contend that much of what is wrong with modern software development is due to too much value being placed on developer ergonomics and comfort. I’m slowly cooking up a talk or blog post about it.

2 Likes

You can simulate okdefer like this I guess:

var success: bool = true;
defer if (success) …;
errdefer success = false;
<business logic>
4 Likes

That would be a bug in stringifier. For context, I’m assuming that stringifier in this example is std.json.Stringify, but in general “is that acceptable?” is a question that has an answer (and hopefully the code will document what is and isn’t acceptable, but looking at the implementation and figuring it out yourself is always on the table).

That’s not always for you to decide. If you’re writing a library, for example, making the choice to crash may make your library ineligible for use by someone wanting to use a different strategy (to give an example, take the “web server that handles error.OutOfMemory by responding with an error code for that particular request” idea that Andrew has mentioned before; see also this section of the language reference).

1 Like

There is one exception to that and that is if you detected memory corruption (through whatever means).

In this case you should go and throw a panic.

defer and errdefer are the devil

defer increases the cognitive load for the reader since it violates the typical expectation that code is executed top down. defer and errdefer only exist in zig as a necessary evil to prevent a worse problem: error-prone cleanup code duplication.

For example, observe this allocating code that sorts and filters u32 values based on whether they’re even or odd:

const std = @import("std");
const Allocator = std.mem.Allocator;
const EvenOdd = @This();

even: []u32,
odd: []u32,

pub fn filter(alloc: Allocator, numbers: []u32) !EvenOdd {
    // Local allocation
    // Free every time
    const sorted_numbers = try alloc.dupe(u32, numbers);
    defer alloc.free(sorted_numbers);

    std.mem.sort(u32, sorted_numbers, {}, std.sort.asc(u32));

    // Normally free'd by .toOwnedSlice()
    // Free manually on error
    // (note: defer would be safe here since .deinit() 
    //  can safely be called after .toOwnedSlice)
    var even_list: std.ArrayList(u32) = .{};
    errdefer even_list.deinit(alloc);
    var odd_list: std.ArrayList(u32) = .{};
    errdefer odd_list.deinit(alloc);

    for (sorted_numbers) |val| {
        if (val & 1 == 0) {
            try even_list.append(alloc, val);
        } else {
            try odd_list.append(alloc, val);
        }
    }

    // Caller owned
    // Free on error only
    const even = try even_list.toOwnedSlice(alloc);
    errdefer alloc.free(even);
    const odd = try odd_list.toOwnedSlice(alloc);
    // Note: this is redundant as there is no error returns (try) after this
    errdefer alloc.free(odd);

    return .{ .even = even, .odd = odd }; 
}

pub fn deinit(self: *EvenOdd, alloc: Allocator) void {
    alloc.free(self.even);
    alloc.free(self.odd);
}

test "filter works" {
    const alloc = std.testing.allocator;
    var vals = [_]u32{ 9, 7, 6, 5, 4, 2, 1 };

    var even_odd = try EvenOdd.filter(alloc, &vals); 
    defer even_odd.deinit(alloc);
    try std.testing.expectEqualSlices(u32, &.{ 2, 4, 6 }, even_odd.even);
    try std.testing.expectEqualSlices(u32, &.{ 1, 5, 7, 9 }, even_odd.odd);
}

fn testHarness(alloc: Allocator) !void {
    var vals = [_]u32{ 9, 7, 6, 5, 4, 2, 1 };
    var even_odd = try EvenOdd.filter(alloc, &vals);
    defer even_odd.deinit(alloc);
}

test "No leaks on failed alloc" {
    const alloc = std.testing.allocator;
    try std.testing.checkAllAllocationFailures(alloc, testHarness, .{});
}

When test is ran, they both pass:

$ zig test EvenOdd.zig
All 2 tests passed.

Now, imagine if defer and errdefer no longer existed in zig. In order to pass the same tests, we have to use catch instead of try on every line we early return:

pub fn filter(alloc: Allocator, numbers: []u32) !EvenOdd {
    // Local allocation
    // Free every time
    const sorted_numbers = try alloc.dupe(u32, numbers);

    std.mem.sort(u32, sorted_numbers, {}, std.sort.asc(u32));

    // Normally free'd by .toOwnedSlice()
    // Free manually on error
    var even_list: std.ArrayList(u32) = .{};
    var odd_list: std.ArrayList(u32) = .{};

    for (sorted_numbers) |val| {
        if (val & 1 == 0) {
            even_list.append(alloc, val) catch |err| {
                odd_list.deinit(alloc);
                even_list.deinit(alloc);
                alloc.free(sorted_numbers);
                return err;
            };
        } else {
            odd_list.append(alloc, val) catch |err| {
                odd_list.deinit(alloc);
                even_list.deinit(alloc);
                alloc.free(sorted_numbers);
                return err;
            };
        }
    }

    // Caller owned
    // Free on error only
    const even = even_list.toOwnedSlice(alloc) catch |err| {
        odd_list.deinit(alloc);
        even_list.deinit(alloc);
        alloc.free(sorted_numbers);
        return err;
    };
    const odd = odd_list.toOwnedSlice(alloc) catch |err| {
        alloc.free(even);
        odd_list.deinit(alloc);
        even_list.deinit(alloc);
        alloc.free(sorted_numbers);
        return err;
    };

    alloc.free(sorted_numbers);
    return .{ .even = even, .odd = odd }; 
}

This is what defer prevents. It gives you a way to register cleanup code to run upon scope exit. If all you need is code that runs before a block ends (Writer.flush()), you should simply invoke the code directly. That’s easiest to read. Do not indulge the devil.

We should especially not allow the devil to modify return values. Let’s imagine defer was allowed to return for a moment:

errdefer foo();
defer try bar(); // could return error
errdefer baz();

If bar() errors, should foo() be called? What about baz()? See how hard you’re thinking right now? Even if we define that behavior, reasoning about this code is a nightmare. It goes directly against the zen of zig, which says to favor reading code over writing code.

In conclusion, defer/errdefer is the devil. We made a deal with the devil to clean up our messes. We keep the devil from taking over our lives by limiting what it can do. We should not indulge the devil unless necessary.

7 Likes

I cant tell if this is satire or not?

Surely you understand that this is subjective.

While your experience is not without value, I do dislike you expressing it vehemently as though it’s the only valid experience.

Hypothetical, ‘what if defer could return values’. I assume was referring to @squeek502 addition to my okdefer naive wish.
They were not advocating for such behaviour, but stating how okdefer wouldnt help op, and if it did then it would be bad language design.

I think OP is right to identify it as a smell. But his problem is real, ensuring that your output is valid is a real concern. I think what is needed is similar to zig’s debug allocator. A debug stringifier, which stores the call stack on all begin statements so it can complain if they’re not closed properly.

Or allow defer to capture an optional error as I’ve suggested before.

1 Like

The idea of defer was invented by Andrei Alexandrescu who convinced Walter Bright to add it to the D language when he developed D 2.0. In D it comes in the form
scope(exit) – execute when leaving a scope
scope(failure) – execute when leaving a scope abnormally (in D, that’s via an exception)
scope(success) – execute when leaving a scope normally.

Go has defer, possibly influenced by D, though they did as poor a job of it as possible so probably not. Other languages followed suit. Zig picked up defer from Go but did it right, and added errdefer. So we’re almost back to where we were with D, but still missing okdefer–there are many valid use cases for it.

4 Likes