The thread pool size is limited, can only use up to N threads. But unlike async, it can’t run the code in-place when there is no free thread. And unlike concurrent, it can’t fail if it reached the N limit. In my implementation, the queue is a linked list and the tasks contain the list nodes, so it can never be full. Even if you use constrained array-based queue, e.g. for work-stealing, there should be an overflow queue that’s unlimited.
1 Like
If it can’t run the code in-place, then you need somewhere to store the arguments. You can’t use an intrusive linked list for the queue, where would you store the node? You can’t use stack memory, you’re about to return from the function.
I just looked at the codebase and see that spawnBlocking is fallible because it heap allocates memory for the queue. Looks like it does grow unbounded…? I think this is where I dip out, I don’t like the LLM stench in here, sorry.
1 Like
Yes, closures need allocations, I meant thread count can’t grow unbounded.
1 Like
I’ve just released another version. From now on, I expect the API to be more or less stable. Some minor changes could be necessary, but I don’t expect anything major.
Three big changes:
- None of the the public APIs now accept the
rtparameter. It actually made it easier for multiple runtimes within the same process to coooperate. Additionally, it’s possible to communicate with the async context from a foreign thread. You can use e.g.zio.Channelorzio.Mutexfrom anywhere, and it will the right thing. Removing the parameter also makes it easier to support embedded, which is not a major goal for now, but I want to keep that in mind when doing decisions.
-
You can now use file and network I/O operations outside of the async context. You you use
zio.Filein a foreign thread, it will use regular blocking syscalls. I’ve done this as an easier upgrade path to Zig 0.16, because I need the equivalent ofstd.fsto use in a thread pool. I can’t depend on async file I/O to be efficient. -
There is now
zio.CompletionQueue, which is similar tostd.Io.Batch, but more general. It allows you to use any of the event-loop operations from a coroutine. It provides io_uring-like API no matter which backends you use. You usecq.submit()to add more requests, andcq.wait()orcq.timedWait()to iterate over completions. It uses no allocations internally, it’s all done via intrusive linked lists, and you can scale it to as many operations as you need. You can use it as small inner poller loop, you can also use it for more complex request handling, if you don’t want to use coroutine-per-connection, but also want to avoid callbacks.
3 Likes
@lalinsky Although you might say it’s not production ready, do you think dusty (derived from zio) is in a good enough place for basic web applications that just call other web services and communicate with a database? I wanted to consider writing a business application with it since it seems to be the most complete HTTP server that supports async in pure zig (a hard requirement since some vendors take minutes to respond).
@Illusion Yes, the HTTP client and server is ready and can be used in more serious apps, that’s the next plan for me, as well. However, you disqualified it right in the next sentence, Dusty is not pure Zig, it depends on a C project called llhttp (vendored in). Why is it a requirement?
It’s not really a requirement, it was just to qualify my words of “most complete HTTP server.” After all, I assume in pure C, or some other language there is another complete asynchronous HTTP server.
Edit: lol, I actually did get screwed by the C in the end. Errors on “undefined symbol: __libc_csu_fini and __libc_csu_init” which is a bit weird, since my OS has glibc 2.28, which should have them still. Idk, will try to debug.
Ok, then dusty/zio is definitely good for that use case. It’s essentially what I built it for. At first, I needed an async HTTP server that can communicate with other HTTP services without blocking, and later on also PostgreSQL database. All of that is already supported. I haven’t ran this in production yet myself, but I think it’s in a good shape.
1 Like
Heya - can you clarify exactly what you mean by “a HTTP server that supports async” or “another complete asynchronous HTTP server”
That can mean several completely different things, depending on what you mean
Most of the popular HTTP servers for earlier versions of Zig (ie - up to 0.15), pretty much all do non-blocking IO, concurrent handlers, use kqueue / epoll / io_uring under the hood for the network action, etc.
Dusty + zio, does all that, but also allows proper co-routines, which is great for lots of long lived handlers / actor model.
So there is a whole lot of different “async” going on here - and a whole lot of different types of “concurrent” as well.
Wondered which kind of async you need that was blocking you from considering the other servers out there that you have looked at.
To preface: I am not well versed with asynchrony and concurrency with respect to programming. All I was trying to do was migrate off of Python to a language that is statically typed and can reduce memory usage.
My problem was simple: you receive a web request, you process that in a handler. As part of that handler’s function, it needs to, in parallel, query a bunch of external web services to aggregate information. That act, of making concurrent web requests didn’t seem as straight forward to me with other implementations. With zio, it seems as simple as zio.spawn.
Cheers, fair enough, that makes sense.
If you are on 0.15.2 - you would use thread.spawn() I guess to do the same thing .. similar api, bit more overhead maybe because it’s using full threads.
With zig 0.16 (when it’s all finished), you will get the same thing using an Io.group with a bunch of concurrent jobs, either as threads or fibers. Gotta wait though.
Terminology wise - most languages and ecosystems use the term “async” to mean “both async and concurrent” … in the zig world, those terms are split and more narrowly defined, which may be confusing at times.
The main benefit of dusty + zio is that it’s not just a HTTP server implemented using async I/O, but you can use the same system also for outgoing client requests. In my opinion, the HTTP client in dusty is the easiest-to-use HTTP client for Zig. And then you have other client libraries as well, which you typically need for web dev, all integrated.
It could be that I just need to add some flags to the build config. Please let me know what OS are you using?
A little bit off topic, but because I recently dealed with a similar issue of understanding async/concurrent stuff in Zig (compared to Rust in my case) I think the following blog is really helpful for understanding (Got the link from Andrew Kelleys post an async I/O):
Though, most of you might already know this post and about its content in general 
2 Likes
This how the term got confusing. 
In all other contexts in computing, async / asynchronous means not blocking the current thread of execution, there are many ways to achieve that, but the concept is the same. On the other hand, io.async is specially allowed to block the current thread of execution.
That’s because in other languages the async keyword is overloaded to also mean concurrent. At the same time other languages don’t concern themselves with making sure that library code can be reused in both concurrent and non-concurrent applications, so they never bothered to split the two concepts apart.
We do in Zig, so we need to be more precise with our terminology.
And in fact io.concurrent is not allowed to block the current thread of execution.
Terminology pain is unavoidable, roughly for the same reasons as this movie scene:
7 Likes
OS would be RHEL8, more specifically 8.10.
1 Like