I think that you know about this already, but in case you don’t (or somebody else is interested): one footgun of epoll is that you don’t subscribe to the file descriptor but to the file description.
This means that after this:
int fd = accept(...);
epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, EPOLLIN);
close(fd);
Your epollfd is still subscribe to whatever fd was pointing to inside of the kernel, but now without the ability for you to unsubscribe.
Yes, this change depends on the fact that all sockets go to through my event loop, and I get to handle all the closes.
New boring release. Includes a number of important bug fixes, file operations which need to be delegated to a thread pool on macOS/BSDs are now cancelable, swaps the internal run queue in preparation for work stealing across executors, respects CPU count in Kubernetes/Docker when using .auto executors, plus integration with -fsanitize-thread for detecting races across multi-threaded coroutines. The next release will include work stealing, which will slightly regress performance of some operations, but should improve fairness overall, important for e.g. web servers. You will be able to use the runtime in three modes: 1) single-threaded 2) multi-threaded tasks pinned to their threads 3) multi-threaded tasks freely migrating across threads.
I’ve been hesitant to do this for a long time, because nobody would trust good benchmarks for a new project. However, since we recently added work stealing (which made the scheduler measurably heavier), I needed to see how are we doing objectively. I’ve tried to construct a set of benchmarks that specifically showcase all aspects zio, not just trying to show how fast is it. I’ve compared zio against Go, Rust/Tokio, C++/Asio and C++/PhotonLibOS. The results mainly show that zio is really not embarrasingly bad. It’s hard to win all the benchmarks, because some of them measure raw througput, some of them measure how well can the runtime serialize work that should be serialized, and some of them measure how well it can parallelize work that should be parallel. Go is truly a marvel of engineering, because it’s really good at pretty much everything. Not the best, but among the best everywhere, and it’s the most balanced system, it just doesn’t have bad edges. With the recent scheduler changes, zio is also doing pretty good all around, but there is still tuning to do. With work stealing, it’s even good as a general purpose thread pool for CPU-bound work.
I’ll do another release with these changes soon, and that will be most likely the end of zio’s active development. After that, I’m going to do some small changes, then release 1.0 and then keep using it in production myself.
I’m not sure if you’re aware, but the ClickHouse team recently open sourced their new stackful fibers implementation (C++ obv) and it might be a useful place to steal some ideas from (and/or add to the benchmark shootout)
Yes, I’m aware of it. There is nothing to take out of there, Go is still the state of the art of this architecture, but I can try to benchmark it
Maybe adding NVidia’s stdexec would be good for the benchmarks too, since that’s essentially just an implementation (with additions) of C++26’s std::execution API.
I was quick to dismiss this, as I only knew it from some media posts before. I had a deeper look and to me surprise, this is almost exactly the same design as zio. As is, zio is slightly faster than them at typical network benchmarks, but they actually do something interesting to avoid latency issues when running unresponsive tasks, which is worth exploring. They essentially implement work stealing also on the I/O level. That’s what costs them the performance difference, because they can’t use optimized io_uring setup, but on the other hand, they better deal with stalled tasks. Thank you for posting this.
Forgive me if I am wrong, but I don’t believe stdexec has networking support? It still is marked experimental despite being accepted into C++26. Networking is supposed to be on the horizon for C++29… at best. Little bit of a rant, but I would be surprised if any meaningful performance work went into that framework, given the committee’s record in the last decade.
It doesn’t have networking, yes, but most of the benchmarks also don’t use networking.
For what it’s worth - I publish some simple non-serious benchmarks with my Datastar api, across a range of different backends. This one isn’t measuring raw rps numbers so much as doing quite a bit of real work per request translating 100kb slabs of HTML into event stream format data, and pushing it out over the wire.
All of the Zig implementations win this one easily, since they are doing zero allocation in-stream transforms.
Of all the Zig implementations, it’s pretty close, but stdlib + zio coroutines wins on rps, memory usage, and latency. The margins are small, except for latency which is significantly better with zio.
Http.zig performs admirably, considering its “age”, but I think we are at a point now where the new stdlib approach nudges ahead. It’s taken years to get there, which is a huge compliment to Karl’s work.
There may well be room for improvement there with zio, but as it stands, it’s comfortably in 1st place on this use case, so that’s good enough for me until Zig 1.0 rolls around.
Rust works fine, but the numbers are unimpressive.
On the same benchmarks, Go is at least 2 times slower, but that’s due to doing a whole lot of GC backed allocations and string munching per request. On the wire Go is fine.
JIT compiled Erlang is a bit slower than Go, but surprisingly posts very very good tail latency numbers anyway .. it’s 2nd only to zig+zio. It might be 200 year old code now, but does show that good design is always hard to beat.
Bun (pre rust fork) didn’t fare well on this benchmark. Not bad, but not good either.
I’ve built a funny side-project, Python asyncio event loop, based on the low level event loop in zio. It’s a bit faster than uvloop, and also supports native async file I/O, unlike most other solutions for Python.
Another release, a fairly big one. I was delaying implementing work stealing since the early days, because I really wanted to do it right. There are simple approaches (like what the unfinished std.Io.Evented uses, for example), but they can actually make the whole thing slower in many situations. Despite having many alternative plans, I ended up using a model very similar to Go/Tokio. No matter what scenario I try, zio is now in the same league as these, even leading in many scenarios. Very specialized engines beat zio at pure networking, but it’s a tiny margin. One disadvantage is that the project is now getting quite complex, but the complexity makes it faster. Part of the complexity is also that I wanted to make it possible to run zio completely single threaded, so there are many comptime conditional that make sure the work stealing machinery doesn’t end up in single threaded builds.
The release also includes many fixes for bugs that work stealing started exposing, primarily on weakly ordered architectures like arm or riscv.
I’m not going paste the whole changelog here, it’s long. 
This is the first release where I had a bunch of external contributors, that makes me really happy, because peope find the project useful.
We are getting really close to zio 1.0. I’m now using this in production on my AcoustID project.
Thanks for creating this project, great work. I like the fallback change in linux from io_uring to epoll if the former is not supported.
I’m doing a silly benchmark for 1000 webrtc connections and I’m comparing between 0.16.0 and 0.17.0 versions. memory usage in 0.17.0 is 25 MB more than 0.16.0.
How are you counting memory? Stacks for coroutines are now allocated in slabs, 64 at once. This is specially to make it cheaper to launch 1000 tasks at once, but if you are including virtual memory reservations in your measurement, it will be higher because of the unused stacks. This memory costs the OS nothing, however.
If you have the benchmark public, I can have a look.
I’m just looking at top. In v0.17.0
v0.16.0
The config I use for the runtime
var r = try zio.Runtime.init(init.gpa, .{
.executors = .auto,
.stack_pool = .{
.maximum_size = 1 * 1024 * 1024,
.committed_size = 64 * 1024,
},
.enable_task_migration = false,
});