Iofthetiger

Like Prometheus, I’m just stealing fire from the gods. :fire: Actually, I didn’t have to steal anything given the graciousness and good will from the awesome people at TigerBeetle. This is the IO library component of tigerbeetle, with the TigerBeetle specifics removed.

There’s a sample multithreaded web server (aside from the tests) that demonstrates how to use the networking part of the library. This is a really impressive body of work by the folks at TigerBeetle; high performance, async, and cross-platform (Linux, macOS, and Windows). Enjoy!

19 Likes

In Andrew’s talk on 2024, where he talks about feedback from Tigerbeetle, it is mentioned their async is implemented with callbacks. So this does not require an older zig version (from before the languages async was disabled).

1 Like

Yes, they implemented a uniform callback-based async IO abstraction layer over each operating system’s specific async IO interface.

3 Likes

Would you mind explaining what an async io library is used for? You know for the people that may not be as familiar "cough cough pointing to myself "

2 Likes

While all of that is true, the real triumph here is that pun… iofthetiger. How old were you when your parents realized that you’re a genius?

7 Likes

They still haven’t realized it. LOL But yeah, I just couldn’t let this golden opportunity for a pun to pass.

3 Likes

Way back in the beginnings of operating systems, input and output (IO) operations like reading and writing to files and sending and receiving from network sockets required the program to wait (block) until the operation in question finished. While blocked, your program can’t do anything else, just sit there waiting while other tasks that could have been attended in the meantime stay piled up unattended. This is known as blocking, synchronous IO, and it’s a sad story.

Then someone came up with the brilliant idea of a model where you can start an IO operation and not have to wait until it finishes (non-blocking). The operating system can monitor the operation for you and notify you when it’s done. This way, your program can perform other tasks in the meantime, maximizing efficient use of those precious CPU cycles. This model is known as non-blocking, asynchronous IO, and it has brought happens back to the hearts of programmers.

As with most other aspects of programming, quite a few strategies have been developed over time, trying to make async IO more ergonomic and programmer friendly. Models using callbacks, futures, promises, coroutines, and async / await mechanisms are some of the most popular. And now the hard part is learning them all because the mix of operating systems, programming languages, and IO libraries can produce a combinatorial explosion of such models.

That’s why this async IO library from TigerBeetle is so impressive; it hides the details of the Linux, macOS, and Windows async IO interfaces behind a single ergonomic callback model. Callbacks are a natural fit for async IO, since it’s like telling the OS: “Hey OS, do this bit of IO and when it’s done call this function please.” It’s a model made extremely popular by Node.js.

9 Likes

Thats is gonna be my kids bedtime story tomorrow for sure! Absolute gold lol

6 Likes

A little note: yes, non-blocking and asynchronous terms very often used as synonyms, but they are not:

  • non-blocking: a syscall (read()/write for instance) in case when the operation can not be performed right now (there is no data at the moment or no free buffers in case of write) just returns with error, and errno is set to EWOULDBLOCK/EAGAIN. You can use it for polling - and.btw, it is not always bad - if you know for sure, that in 99% cases it would not block, that’s ok.

  • asynchronous: you start an operation, and then after some time you get a notification when that operation is complete.

Windows socket API was asynchronous starting from Windows 3.11 for Workgroups.
Most Unices do not have completion notifications, instead they have readiness notifications (epoll in Linux, kqueue in FreeBSD). I’ve already mentioned an article about this somewhere, here is the link once again.

3 Likes

How does this relate to the “original” concurrency model in Zig ? I remember reading this blog post (among others - what color is your function is pretty entertaining) before starting to dabble in Zig. I thought “no colored functions? that’s cool, I wanna try this” … just to find out it was removed from the latest release of Zig. I have to add, since I got accustomed to go’s concurrency model, I find it really hard to use “colored” async/await. There’s still multi-threading (or even multi-processing), but running every little function in its own thread? I don’t know, seems like big guns for small problems :wink:

The first sentence from that article:

System I/O can be blocking, or non-blocking synchronous, or non-blocking asynchronous

So, “non-blocking” != “asynchronous”, these two are different things.

1 Like

This is exactly why I harp on event driven state machines everywhere :slight_smile:
With them you can achieve fine-grained concurrency within a single thread in any “normal” language without special coroutine/greenlet/fibers etc support on a compiler side.

2 Likes

Can you explain the pun please? Non-native English speaker asking… :slight_smile:

1 Like

Sure!

There was a famous song called “eye of the tiger” by the band Survivor: https://www.youtube.com/watch?v=btPJPFnesV4

The word “eye” is pronounced like “I”, so he replaces that in the name so now we get: I of the Tiger

But you have two things here, first IO (input/output) is the point of the library, so he adds that in - now we get IOf the tiger.

Second is that the library it comes from is “Tiger Beetle”, so this is the IO of Tiger Beetle.

So putting it all together, iofthetiger - “Eye Of the Tiger”… which is the IO of Tiger Beetle.

It’s exactly what the project is about but it’s also a reference to that famous song.

6 Likes

Nice work! Yeah, TigerBeetle is a treasure chest of cool systems programming. I believe Mitchell’s also been inspired by the TB IO core when working on his event loop lib. It’s pretty awesome, too:

3 Likes

That’s why I didn’t use them as synonyms, note the comma between non-blocking and asynchronous:

The comma denotes they are two separate things being used together. But maybe writing it as “non-blocking and asynchronous” would be clearer.

But you make a good clarification and I’m interested in reading the linked article, seems like a very interesting read. Thanks.

2 Likes

It is really very nice article, otherwise it wouldn’t be in my browser bookmarks :slight_smile:
Basically, it’s about how to combine proactor pattern (Windows) and reactor pattern (Unix) in one cross-platform library. With io_uring we have more similarity with Windows IOCP, but in 2005 there was no io_uring yet.

2 Likes

As I see it, Zig’s first attempt at async IO was via the async / await model, albeit implemented in a a novel way using the function’s frame directly with suspend and resume points. It seems although it was a cool and novel idea, it clashes in critical ways with other aspects of the compiler which have higher priority in terms of achieving Zig’s goals. So it was sacrificed and removed. This has spurred discussion on whether this is something that the language should provide or if it should be left for the ecosystem to provide. The jury is still out on this.

This library is an example of how the ecosystem can fill in this gap. The only issue with letting the ecosystem handle issues like these is that there will be many different ways to do the same thing, increasing complexity. But at the same time, I think this allows for evolution of paradigms and models, not being locked-in by what was implemented in a language at some time in the past.

4 Likes

Same here. It boggles my mind how the majority of the industry has gone the async / await route, as if it was an easy model to understand and implement. The Go (and I belive Erlang / Elixir) model is much more intuitive to me. You just define normal functions and send them off to do their thing concurrently. If you need to communicate with them, use channels. The simplicity lowers the barrier for many more developers to actually use concurrent tools in the language and thus finally makes efficient use of multi-core hardware.

4 Likes

Good coherent terminology never hurts. Ambiguities in terms are often the source of confusion and misunderstandings. Using read() syscall as an example we have 4 variants of program/OS kernel behavior in the case when there is no data to be read at the moment:

  • block (go to ‘wait/sleep’ state, OS will wake up us when data will be there)
  • return with errno = EAGAIN (this is what I am calling “non-blocking”)
  • use readiness notifications (poll, select, epoll/kqueue)
  • use completion notifications (IOCP/io_uring) - this is true async mode.

…couldn’t help myself from giving the link to Linus Torvalds reaction on AIO stuff back in 2016 :slight_smile: