This is likely to go essay style, so if you are looking for a fast read, this is not it.
First this: I love std.Io and it’s concurrency/async concept. It’s beautiful. It’s unfinished. I worked a bit with zio, which is great. One of my long term side projects is a reimplementation of Nats (a message broker/communication framework), same vision, but focused on performance and lean resource usage and without the spaghetti code. One of the requirements is that such a system must handle a lot of connections and a lot of messages, which translates to a lot of Go routines, fibers, callbacks or whatever is responding to IO.
This is where you cannot just use async or concurrent whenever you have to wait for IO to do something, because every waiter get’s a stack, something presized from a few kB to MB. Small sizes lead to stack overflows or memcpy or various other issues that each have their price to avoid. Big sizes lead to massive amounts of wasted memory or if you are fancy at least address space.
It’s easy enough to write a custom event loop for use cases where you cannot use std.Io naively, but the problem keeps surfacing in other areas. For example, you need to parse a message header, but the header is not guaranteed to come in as contiguous memory area. You want to read the header before the message is complete, because the message routing is complex and takes time and why let other cores sleep while they have work to do. You don’t know whether the header is completely read until you start parsing it and hit the end of buffer. So you have to handle the case when you need to wait for more data.
Parsing a header requires access to the data. If the data is in cache, it’s there, if not, loading it takes much longer than the parsing itself. So you want to do the parsing in the thread that does the IO, because the data is then still in cache (depending on stuff). But if you do compute in a thread that does a lot of IO, everybody else has to wait for the compute, so that better be fast, really fast.
My header scanner uses simd and compared with a byte-by-byte scanner, it’s about 1.5-3x faster. Mind you, these a nano-seconds, not a big deal. But there is a reason why simdjson has such a great reputation. The scanner implemented using std tokenizers is 2-4x slower than the b-b-b scanner, despite also using simd, it just uses a fresh register load for every token you scan.
This scanner I wrote is surprisingly cool. I plan to use it for other things, given how convenient and fast it is. Like CSV, passwd, procfs stuff scanning. All of these might be used in performance critical contexts and all these contexts have to somehow handle fragmented input. You really don’t want to load an entire file into memory if you don’t know how big it is. Fragmented input means you need to pause scanning until you have data to scan.
When using std.Io, this is no issue, you just use a reader-kind of abstraction for your input and code away. If your input is not yet there, your scanner gets suspended and work continues once data is available and the core doing the work is doing other stuff. And this is not expensive, not much. You’re still doing high performance scanning, it’s the fault of IO that you wait. But, if you can’t use std.Io because the overhead for fibers or threads is too much for your use case, this is not an option. Then the scanner has to return something like EGAIN and somehow has to know how to proceed once data is available.
Here is the real problem: If you’re writing some library, you don’t want to write two versions, one colorblind and one with suspension. That’s the whole point of std.Io’s async/conc, getting rid of colors. But if colorless can, as a matter of principle, never get top performance because of the framework overhead that is immanent to any std.Io implementation, then you need to provide an escape hatch.
Zig without async can of course provide this escape hatch. You basically do what you would do in C, you write a state machine that allows resuming the code waiting for IO. Getting this right however can be rather hard. And it’s hard for each and every library that has to do it. That’s why people love async keywords so much, even if it means you have to write two versions of the same logic. It’s much easier to write correct state machines.
Most people will just use std.Io abstractions and it will be perfectly fine. Most library authors will do the same and it helps a lot to write performant and maintainable code. So what am I complaining about?
Zig is the language I would always choose - as a matter of first principles and concepts - if I want the best performance and the most control. This is what Zig does. Together with comptime, I can build zero cost abstractions. The set of design decisions in Zig so far is unique in this regard. There is no GC, no runtime, no frameworks. Zig has enough sugar to make it surprisingly easy and safe.
That is, until std.Io. This is not Zig the language, it’s just a library and you don’t even have to use it. You can implement std.Io yourself and tweak it. But no normal developer would ever do that under any normal circumstances. It’s hard.
Zig, compared to other languages, has a couple of disadvantages. A lot of functionality that is included in other ecosystems is not in the std library and also not yet provided by the community. The fact that the language is still evolving puts a maintenance burden on devs. That’s fine with me, I can wait, I can contribute, and it’s fun.
But what that means taken together is, that the ecosystem will evolve along the mainstream. Zig will eventually commit to std.Io and there will not be a sustainable path to write software with truly minimal resource usage and maximal performance. You can write optimal software, but then you have to write it all yourself. There will be no libraries that support stackless async, because to few users need it and everybody uses the one colorless interface. Even libraries that emphasize performance over everything else, will likely only support std.Io. And Zig will have a runtime. You can replace it, theoretically, but you likely won’t.
The thing I would like to discuss, after making this point above is this:
To implement my scanner in a reasonable way, I would provide a comptime parameter that has is something like either getMoreData(): u8 or getMoreData(): Future(u8). This parameter would then bleed into the function signatures of the methods providing scan results and also leak into some seams in the library. std.Io has a kind of future, zio does. But, to my knowlege they are either not public or somehow not really idiomatic Zig abstractions. There is no accepted Zig pattern for using futures, that’s the whole point of std.Io and removing async. So I would likely provide my own Future type and do my own state machine implementation, and so will everybody else.
This is not sustainable and it will in my estimation end up with people just paying the stack tax, Zig will have a runtime and it will be somewhere between excellent and good enough for most requirements and not the right tool if what you really want is both control over everything AND convenience.
With this particular experience, I get the impression that providing std.Io was THE RIGHT thing to do, but removing async was a MISTAKE. I think a language like Zig needs both, a colored async and a colorblind std.Io, unless colorblind and stackless are not mutually exclusive attributes, which they seem to be.
Your thoughts? Am I missing something?
