Thank you for taking the time to write this up.
I actually got the coroutine switching working on RP2040 (PIco W). The assembly is slightly different from ARM due to differences in how Thumb needs odd PC addresses, but it’s working now. I don’t know if I push the project further any time soon, but it’s nice that the foundation works well.
If anybody wants to play, here is the blinky PoC that alternates between two coroutines, each blinking a different pattern:
6 Likes
I’ve released v0.6.0 with a few changes, mostly the 32-bit CPU support, some memory optimizations, and async pipe/stdio support. Release v0.6.0 · lalinsky/zio · GitHub
I also resumed the work on docs, motivated by the other topic here. Picked a few simple projects to cover most of the functionality. I’d love to hear feedback on this:
7 Likes
If there is anyone seriously considering using zio, I’d like your opinion.
When I started working on the library, I made the choice to explicitly pass rt to all the functions. It felt like the right thing to do in Zig.
Internally the code already depends on thread-local state and there is no way to avoid it. It needs to be able to get the currently running task and that can’t be passed via the parameter. Using thread-local state for the current task is not that strange, std.Io.Threaded also does that, so I’m fine with that part.
However, what I realized is that in most places I either explicitly discard the rt parameter, or I use it inside assert checking if the current task belongs to the passed runtime. The assert is not practically helpful, it’s just a nicer way to discard it.
So I don’t really need it. At this point it’s just token, marking that the function will do something asynchronous. I could change the API to work without it and nothing would change internally, except for removing the discards.
Practically speaking, it bloats the generated code. I came to this whole thinking about the embedded use case where the runtime would be global, so passing it around is pointless.
What do you think? Keep it even though it’s just discarded. Or ditch it and just go with what is practically needed.
1 Like
I don’t yet use zio, but plan to. If rt isn’t being used, drop it. Your library is still early enough along that breaking changes are acceptable. I’m personally all for design improvements, even if I have to fix my code to use it.
1 Like
You know better than anyone whether you really won’t need it in the future. If not, removing it is best.
I’ll be honest, my main worry is that the API will be nicer to use than std.Io without it. 
The parameter is mostly just an annoyance to be comparable to std.Io in terms of ease of use, to motivate people to migrate away from my native API once 0.16 is ready.
Right now I’m updating my home-grown proof of concept WireGuard implementation into something more production ready. I’m using std.Io, but also testing with zio. I plan to use zio as the default IO implementation when running as an application, but remain agnostic when shipped as a library.
If you think it makes your zio easier to reason about, then remove the parameter as long as an std.Io compatible interface remains. But… I think it might be worth waiting until a release cycle or two after 0.16 to make a decision. After that it will be clearer how people actually use std.Io in practise. It might be that people want to instantiate multiple copies of zio for god knows whatever half-justified reason, and the parameter retains some use.
4 Likes
Thank you for the feedback, I really value it. No matter what happens to this parameter, the std.Io implementation would not change. All functions require io there and it would work as expected.
The thing is, this already works and will continue to work even without the parameter. I see it similar to Tokio, where there is both rt.spawn() and tokio::spawn(), and most other APIs require are dependant on the context, not the rt parameter.
There is one confusing thing, zio.Runtime.init() has the semantic meaning of: take this thread and turn it into a coroutine. At that point you can’t block the thread with blocking syscalls. You can have multiple runtimes, just not in the same thread.
The reason why I’m considering this is simple:
I need to be able to use filesystem/mmap/etc in tasks spawned from spawnBlocking. I don’t want to bring the cancelation machinery of std.Io.Threaded into these tasks, so I essentially need to build my own blocking I/O and I’m thinking about reusing the existing zio.fs namespace. Depending on the context, zio.fs.File would do async operation when running in a coroutine, and blocking operations elsewhere. I could add zio.Io to differentiate between this, but it would be literally just Io = struct { blocking: bool } and even that field would only be used in asserts (can’t do blocking ops in coroutine, can’t do non-blocking ops outside of coroutine), so it seems pointless.
3 Likes
Out of ignorant curiosity, why? Isn’t that the path forward?
Cancellation is modeled as errors, so a cancellation request is simply injecting an error. You have to handle errors, anyway, so what’s the issue?
As of right now, std.Io.Threaded doesn’t have the semantics I need for the thread pool. I need a traditional thread pool, you spawn up to N threads, you have a queue, you submit work to it, and submitting work to the thread pool can’t fail. Neither io.async nor io.concurent satisfy that.
I could implement a custom thread pool option on top of io.concurrent, but then I have the problem that I can only cancel my thread pool workers, not the individual tasks. So the cancellation logic is useless for me.
Alternative is, custom thread pool, and using std.Io.Threaded.global_single_threaded.vtable for running I/O operations on my own threads, but that’s fragile, because std.Io.Threaded depends on its own thread-local variables.
It just all seems messy to me, this is one area where I’ll wait until at least Zig 0.17.
3 Likes
I was able to understand point (1) enough to file an issue for it:
The other three I don’t really get. I would be interested in a deeper explanation, or an issue writeup like the one I just linked.
11 Likes
I’ll illustrate the second issue on a non-trivial example from the kind of applications I’m working on:
Imagine you are working on an audio streaming server. For simplicity let’s say you have a collection of files on attached storage, and you have some kind of dynamic playlists, that tell you how to arrange the audio files in order to play a radio station. The streaming server supports multiple codecs per station and each station/playlist is unique, because it has contextual ads inserted at some points.
You absolutely need to use evented I/O for the frontend, because you need to handle >100k active listeners at any point, you will need maybe 2-3 threads to serve that traffic.
Then you have the middle layer, that one is fetching the playlists for each listener, determining which files need to be processed, they might need to be chunked, they might need to be transcoded, and many many things. This is like a management layer, it can also be using evented I/O, because it might need to determine the playlists from a HTTP service, etc.
But then you have the low-level layer, you are processing the audio files using FFmpeg. That’s CPU intensive, and you absolutely need to use a dedicated thread pool for this. But the thread pool doesn’t live in isolation, it needs to be able to communicate with the higher layers, maybe using a set of queues, so the synchronization mechanisms between threaded and evented need to be compatible.
(This is a system I realistically built, so I have practical experience building it and then running in production, but it was not in Zig.)
With Io, this is currently possible on Linux, because you use futex in both versions. I guess you could somehow do it similarly on Windows, it will be slightly more awkward. But there is no OS-level mechanism that would allow you to do that on macOS/BSDs. So I don’t think you can provide guarantee that e.g. item added to Io.Queue from Io.Threaded can wake up task waiting in Io.Kqueue. That requires the threaded implementation to be aware of the evented implementation.
10 Likes
Thank you for the detailed explanation.
Isn’t this merely a missing function of std.Io which is an alternative to std.Io.concurrent that communicates that the called function will not cooperate with the I/O implementation? I.e. you could use it to wrap arbitrary C library code that you don’t control and that blocks without participating in std.Io machinery.
Evented impl would handle this by dispatching it in its internal thread pool that it already has for stuff like this (e.g. DNS). Threaded impl would handle it identically to Io.concurrent, no special handling needed.
I don’t think two I/O implementations are part of the equation here.
Yes, that’s exactly what is it. In zio, that’s called spawnBlocking, in Tokio it’s spawn_blocking, in Python’s asyncio it’s run_in_executor. If that’s available, evented could be potentially used as is. But it needs to work like a regular thread pool, not grow unbounded. So “concurrent” is maybe not the right term.
Tasks spawned this way don’t have support for cancelation nor timeouts, correct?
Not in my uses, no. It would be ideal if they could, but given that people would be using these for random C functions or code that depends on mmap, cancellation is not always possible.
1 Like
feel free to suggest a specific API
8 Likes
Can you elaborate on what you mean by not growing unbounded? Are you talking about the closure queue (i.e. tasks waiting to run in the pool)? What do you do when the queue is full then?