Like Prometheus, I’m just stealing fire from the gods. Actually, I didn’t have to steal anything given the graciousness and good will from the awesome people at TigerBeetle. This is the IO library component of tigerbeetle, with the TigerBeetle specifics removed.
There’s a sample multithreaded web server (aside from the tests) that demonstrates how to use the networking part of the library. This is a really impressive body of work by the folks at TigerBeetle; high performance, async, and cross-platform (Linux, macOS, and Windows). Enjoy!
In Andrew’s talk on 2024, where he talks about feedback from Tigerbeetle, it is mentioned their async is implemented with callbacks. So this does not require an older zig version (from before the languages async was disabled).
Way back in the beginnings of operating systems, input and output (IO) operations like reading and writing to files and sending and receiving from network sockets required the program to wait (block) until the operation in question finished. While blocked, your program can’t do anything else, just sit there waiting while other tasks that could have been attended in the meantime stay piled up unattended. This is known as blocking, synchronous IO, and it’s a sad story.
Then someone came up with the brilliant idea of a model where you can start an IO operation and not have to wait until it finishes (non-blocking). The operating system can monitor the operation for you and notify you when it’s done. This way, your program can perform other tasks in the meantime, maximizing efficient use of those precious CPU cycles. This model is known as non-blocking, asynchronous IO, and it has brought happens back to the hearts of programmers.
As with most other aspects of programming, quite a few strategies have been developed over time, trying to make async IO more ergonomic and programmer friendly. Models using callbacks, futures, promises, coroutines, and async / await mechanisms are some of the most popular. And now the hard part is learning them all because the mix of operating systems, programming languages, and IO libraries can produce a combinatorial explosion of such models.
That’s why this async IO library from TigerBeetle is so impressive; it hides the details of the Linux, macOS, and Windows async IO interfaces behind a single ergonomic callback model. Callbacks are a natural fit for async IO, since it’s like telling the OS: “Hey OS, do this bit of IO and when it’s done call this function please.” It’s a model made extremely popular by Node.js.
A little note: yes, non-blocking and asynchronous terms very often used as synonyms, but they are not:
non-blocking: a syscall (read()/write for instance) in case when the operation can not be performed right now (there is no data at the moment or no free buffers in case of write) just returns with error, and errno is set to EWOULDBLOCK/EAGAIN. You can use it for polling - and.btw, it is not always bad - if you know for sure, that in 99% cases it would not block, that’s ok.
asynchronous: you start an operation, and then after some time you get a notification when that operation is complete.
Windows socket API was asynchronous starting from Windows 3.11 for Workgroups.
Most Unices do not have completion notifications, instead they have readiness notifications (epoll in Linux, kqueue in FreeBSD). I’ve already mentioned an article about this somewhere, here is the link once again.
How does this relate to the “original” concurrency model in Zig ? I remember reading this blog post (among others - what color is your function is pretty entertaining) before starting to dabble in Zig. I thought “no colored functions? that’s cool, I wanna try this” … just to find out it was removed from the latest release of Zig. I have to add, since I got accustomed to go’s concurrency model, I find it really hard to use “colored” async/await. There’s still multi-threading (or even multi-processing), but running every little function in its own thread? I don’t know, seems like big guns for small problems
This is exactly why I harp on event driven state machines everywhere
With them you can achieve fine-grained concurrency within a single thread in any “normal” language without special coroutine/greenlet/fibers etc support on a compiler side.
Nice work! Yeah, TigerBeetle is a treasure chest of cool systems programming. I believe Mitchell’s also been inspired by the TB IO core when working on his event loop lib. It’s pretty awesome, too:
It is really very nice article, otherwise it wouldn’t be in my browser bookmarks
Basically, it’s about how to combine proactor pattern (Windows) and reactor pattern (Unix) in one cross-platform library. With io_uring we have more similarity with Windows IOCP, but in 2005 there was no io_uring yet.
As I see it, Zig’s first attempt at async IO was via the async / await model, albeit implemented in a a novel way using the function’s frame directly with suspend and resume points. It seems although it was a cool and novel idea, it clashes in critical ways with other aspects of the compiler which have higher priority in terms of achieving Zig’s goals. So it was sacrificed and removed. This has spurred discussion on whether this is something that the language should provide or if it should be left for the ecosystem to provide. The jury is still out on this.
This library is an example of how the ecosystem can fill in this gap. The only issue with letting the ecosystem handle issues like these is that there will be many different ways to do the same thing, increasing complexity. But at the same time, I think this allows for evolution of paradigms and models, not being locked-in by what was implemented in a language at some time in the past.
Same here. It boggles my mind how the majority of the industry has gone the async / await route, as if it was an easy model to understand and implement. The Go (and I belive Erlang / Elixir) model is much more intuitive to me. You just define normal functions and send them off to do their thing concurrently. If you need to communicate with them, use channels. The simplicity lowers the barrier for many more developers to actually use concurrent tools in the language and thus finally makes efficient use of multi-core hardware.
Good coherent terminology never hurts. Ambiguities in terms are often the source of confusion and misunderstandings. Using read() syscall as an example we have 4 variants of program/OS kernel behavior in the case when there is no data to be read at the moment:
block (go to ‘wait/sleep’ state, OS will wake up us when data will be there)
return with errno = EAGAIN (this is what I am calling “non-blocking”)
use readiness notifications (poll, select, epoll/kqueue)
use completion notifications (IOCP/io_uring) - this is true async mode.
Actually, I didn’t have to steal anything given the graciousness and good will from the awesome people at TigerBeetle. This is the IO library component of tigerbeetle, with the TigerBeetle specifics removed.
