0.16.0 Released

[tholmes]

It has been removed. You need to replace all use of @Type() from your 0.15.2 code with the new builtins. This is frustrating at first, but provides a lot of opportunity to rewrite the same code more succintly and expressively e.g using @splat()

Upgrade example:

const Storage: type = blk: {
	var role_handles_fields: []const std.builtin.Type.StructField = &.{};
	for(Roles) |Role| {
		role_handles_fields = role_handles_fields ++ &.{.{
			.name = @typeName(Role),
			.type = std.AutoArrayHashMapUnmanaged(ActorId, Role),
			.default_value_ptr = null,
			.is_comptime = false,
			.alignment = @alignOf(std.AutoArrayHashMapUnmanaged(ActorId, Role)),
		}};
	}
	
	break :blk @Type(.{.@"struct" = .{
		.layout = .auto,
		.fields = role_handles_fields,
		.decls = &.{},
		.is_tuple = false,
	}});
};

to

pub const Storage: type = blk: {
	var field_names: []const []const u8 = &.{};
	var field_types: []const type = &.{};
	
	for(Roles) |Role| {
		field_names = field_names ++ &[1][]const u8{@typeName(Role)};
		field_types = field_types ++ &[1]type{std.AutoArrayHashMapUnmanaged(ActorId, Role)};
	}
	
	break :blk @Struct(
		.auto,
		null,
		field_names,
		field_types[0..],
		&@splat(.{}),
	);
};

(Both of the above examples could, in fairness, be made slightly more succint using arrays instead of comptime-concatenated slices)

In more trivial cases, where all you need to do is make e.g. a packed struct with a bit for each value of an enum, code that previously required a block can be achieved using only the @Struct() builtin, std.meta.fieldNames(Enum), &@splat(bool) and &@splat(.{}).

[andrewrk]

From my perspective, with the Zig project I take a lot of steps based on a hunch without fully thinking things through to the very end before taking said step - particularly on the issue tracker, the land of imagination and vaporware. I’ve always said, I’m not particularly smart. My primary skill as a language designer is trying a lot of things quickly plus being self-critical and willing to let go of bad ideas and try something else. The infamous LLVM dependency issue is no different. In other words it was simply poorly thought out, just like half of the issues I open, to be refined or discarded over time. The only difference was all the scrutiny from the peanut gallery (and by that I mean people outside Zig communities who still want to share their opinion on the issue tracker for some reason).

[ww520]

IMO the early LLVM dependency was the correct decision. Time to market was a huge benefit. You want to concentrate your time and energy on areas that make the most impact, and offloading code gen to an off the shelf, battle proven backend just make perfect sense. I’m not sure why people are having an issue now for transitioning away from it.

[shimekukuri]

Congrats to everyone involved.

My only critique is really I wish that the release notes provided a better examples of some of the new features or API’s, and or that as well as the release notes there would be relevant additions to the language docs. For example there is no shown examples of how to use Io.select in either the release notes or in the language docs and I really think for the Io interface to be ‘gleefully’ implemented there really needs to be a good set of documentation on how to do simple but reoccurring tasks and full examples of what Io currently offers and how to reason about it.

[vulpesx]

Io.Select is a std library thing, not a language thing. Lang ref rarely shows off std and when it does, it doesn’t explain beyond the language feature it was focused on.

The only exception to that would be allocators, and I agree that Io should get the same treatment. But it is also still very WIP so maybe they are just waiting until it settles to not edit such a section every release.
And even then, allocators are not explained in great detail.

As std things, it falls to std docs to provide more detail, which they do. Besides the api is very similar to Group which does get an example in the release notes.

And don’t forget zig is pre 1.0 unstable, wip software. It is unreasonable to expect good docs on such a new and unstable feature.

Regardless, it has been talked about for a while before the release, there are examples, and explanations, though maybe a little outdated on specifics like names and exact parameters etc.

[andrewrk]

I like @kristoff’s idea of “guides”. Would be nice to make guides for:

• various build system recipes
• async I/O stuff - select, group, tasks, etc
• implementing a reader
• implementing a writer
• probably more stuff too

[WillAvudim]

Our main pain points with Zig 0.16 remain to be the lack of syntax sugar for closures, and generators (automatic inversion of code to implement a Zig’s iterator).

Closures let us abstract away dependencies, and layer code for independent development of algorithms and data (feature) suppliers. With the current syntax, we end up copying the same boilerplate around a lot, and it hinders both readability and maintainability. Trivial changes become hard multi-page adventures, both for human and AI. Tracking all captured variables is not an easy task, when those variables scattered across the entire codebase.

Generators are must-have for any RL/ML (reinforcement learning/machine learning) system because we are dealing with very complex exploration patterns, e.g. I’ve got 7 loops in a function, all of various complexity, and I’m not talking about trivial for (0..total_actions) {…} loops. Those loops need to be re-coded in Zig’s iterator pattern, to make it async and theadsafe. This can be achieved once, but constantly evolving loop logic leads to time-demanding refactoring, that are extremely hard to code review to boot. It’s currently very hard to take a function consisting of many nested loops with many intermediate variables, and convert it into a struct that iterates over each value while correctly releasing resources.

[vulpesx]

Generators/closures would make the code more concise, but will not do anything about the complexity of the logic nor resource cleanup.

There is an argument that more concise code has fewer opportunities for developer mistakes, but at the same time it hides complexity with implicit behaviour and assumptions that could just as easily lead to a mistake.

zig clearly prefers to make behaviour and assumptions explicit with safety checks to catch mistakes.

[WillAvudim]

@vulpesx

will not do anything about the complexity of the logic nor resource cleanup

This is simply not true. We just recently rewrote that logic in a language supporting both closures and generators, and ease of both writing and maintenance should not be taken lightly. The code became much simpler, easier to read, and easier to update.

You should try it out first, before blindly rejecting the whole notion. E.g. rewrite parts of your code in Rust or Swift. There’s no comparison. The clarity of intent is simply not there in Zig’s version, all is lost behind lines and lines of boilerplate.

[andrewrk]

@WillAvudim

reveals your lack of experience and expertise in the subject matter

This is very aggressive. Criticizing zig is useful, go as hard as you want. However, it’s not okay to level personal insults at users like this.

[vulpesx]

I am quite familiar with rust, maybe I am just weird, but I disagree.

There are certainly cases where generators/closures map best to what you are doing, but they are not a catch-all improvement.

Your argument seems to only be for the conciseness they can provide, which I already addressed. If you find that to be an improvement, you are free to advocate for it.

I am not saying generators/closures bad, I am not saying zig shouldn’t have something to fill their use case.

I only stated that they don’t reduce the complexity of the logic, they are just a different mechanism and as a result may, or may not, express that logic more succinctly. But the logic is the still the same complexity.

I am not strongly in either camp, I understand the motivations, benefits, and issues on both sides.

[lalinsky]

@WillAvudim

This is simply not true. We just recently rewrote that logic in a language supporting both closures and generators, and ease of both writing and maintenance should not be taken lightly. The code became much simpler, easier to read, and easier to update.

Zig is a language with manual resource management and both closures and generators are concepts that require resource management. Allocations, and some state management in the case of generators. You can provide a nice syntax in a language that handles the resources for you, but not in Zig.

[WillAvudim]

@lalinsky

You can provide a nice syntax in a language that handles the resources for you, but not in Zig

Well, I can. In Zig. It’s just currently I have to do it all manually. This is a transpilation problem, not a resource management problem.

Imagine you have just two for loops:

pub fn evalAllTwoNodeTrees(init_tree: *const DecisionTree, exploration: *const ExplorationSetup, feature_recorder: TreeToFeatureRecordingClosure, tree_eval: Arc(EvalTreeClosure), tree_to_bin: ComputeSimActionBinClosure) !void {
    // ...Lots of code removed for brevity...
    const one_split_trees = try enumerateValidTreeSubsplits(init_tree, noop_rec.value, exploration);
    defer alloc.free(one_split_trees);
    for (one_split_trees, 0..) |one_split_tree, outer_i| {
        defer one_split_tree.release();

        const two_split_trees = try enumerateValidTreeSubsplits(one_split_tree.value, one_split_rec.value, exploration);
        defer alloc.free(two_split_trees);

        for (two_split_trees) |two_split_tree| {
            defer two_split_tree.release();

            *yield* two_split_tree;
...

There are just two loops now, and to convert it into an iterator, which can be used on demand to schedule work items in a thread pool, we have to declare a struct with all intermediate variables optional, because they do not necessarily exist at all points during the iteration (inside other loops below the second one, or when the loop is over), and trigger all defer statements in the unrolled loops correctly when the condition is no longer true. It’s not an easy feat.

Surprisingly (or unsurprisingly), Claude Code also struggles to read/modify the unrolled/iterative version of the function. It regularly reports bugs in the perfectly correct code, and unable to write code with more than ~4 inner loops. So, it’s not a trivial task to unroll such code into Zig iterators manually or via AI.

The worst part, we have to modify this code constantly as our research advances.

You have to perform a similar feat in order to unroll a recursive function into an iterative function, to push out the stack onto the heap explicitly, to keep all intermediate state in an ArrayList instead of on-stack variable instances. This gives you const-space stack. There should be some sort of transpilation mechanism to unroll the loop with all states into a state machine. Doing it manually is very error-prone and unfriendly to humans.

Compute is compute is compute. There’s no difference between the languages, just advancements in data and code representations. You can code it all in AND and OR gates, if you have to. Why stop there?

[mperillo]

I tried running the group.zig example from the release note, but I got

1/1 group.test.sleep sort...FAIL (TestUnexpectedResult)
/home/manlio/.local/share/sdk/zig/0.16.0/lib/std/testing.zig:615:14: 0x12388f9 in expect (std.zig)
    if (!ok) return error.TestUnexpectedResult;
             ^
/home/manlio/code/lang/Zig/async/group.zig:31:9: 0x1238d3f in test.sleep sort (group.zig)
        try std.testing.expect(a <= b);
        ^
0 passed; 0 skipped; 1 failed.

Thanks.

[vulpesx]

@WillAvudim

we have to declare a struct with all intermediate variables optional, because they do not necessarily exist at all points during the iteration

have you tried a tagged union? Should better represent the states of your iteration and have less memory and safety check overhead.

[mlugg]

but I guess I trusted that more subconsciously.

Actually, that trust is misplaced, and that is a big part of why we disallowed this!

Runtime vector indexing can be an expensive operation: vector registers and instruction sets are designed for SIMD operation rather than scalar value extraction, and the memory layout of a SIMD register stored back into memory might be harder to work with than a simple array. Combine that with the values also being potentially bit-packed, and you can get extremely inefficient codegen. In fact, LLVM would usually lower runtime vector indexing more-or-less by unwrapping the vector into an array and then indexing into that array! If you’re going to be plucking elements out of a vector, then it probably shouldn’t be a @Vector type by that point: those are for SIMD operation and only hurt you when working with individual elements. In that case it’s a much better idea to do the vector-to-array conversion once so that future element accesses are fast.

Here’s a few examples of the old vector-indexing codegen next to the codegen for converting to an array and then indexing: godbolt. The conversion to an array is sometimes more instructions, but if you’re going to be accessing more than one element—which you almost certainly are—that will pretty much always pay for itself by the second array access!

[wbrian]

@WillAvudim

You should try it out first, before blindly rejecting the whole notion. E.g. rewrite parts of your code in Rust or Swift.

I’ve written a lot of Rust and a lot of Swift. Closure support in rust is tightly coupled to the ownership and borrowing model, while closures in swift rely on automatic reference counting where it’s easy to introduce strong reference cycles. Point being, it’s not something that you can simply bolt on to a language.