The New IO Interface - stackfull coroutines etc., just tell me everything/anything

Explain
,
1

Hello, first time here and first of all i want to say that i value Zig core Team and their work they did so far. I never loved programming language as much as i love zig. I am just learning so much from the language and its standard library. Like what dude invented std memory pool with self contained linked list is just as clever as could possibly be! Just perfect.

Tho I was recently messing with The New IO Interface introduced in 0.16.0 and so far i love it. Since IO interface is mainly bunch of very OS dependent functions and less shared logic, i found it confusing to be all stick and glued together in single Io/Threaded.zig file (speaking of Io.Threaded specifically). It’s pretty hard to follow but i am getting used to it, the reason i was digging in, is that i am heavy windows user (sadly more devs and end users are as well). So my plan was investigate options and experiment with my own IOCP based event loop (with NTDll only btw). Lucky I found a lot of code i could use directly inside Threaded.zig that already helps me with proper structs and windows dependent abstraction, sadly on other hand bunch of these windows dependent functions are not exported or available for std end users.

Right now, i am on my way to learn more about stackfull coroutines and fiber context switching directly on windows, i would like to ask if anyone knows any good resources where i could find more on this topic! So far its lot of assembly magic for me, thanks.

Anyway since i am posting this in “EXPLAIN” section, i would appreciate if anyone could explain or linked me some juicy resources related to following topics/questions.

  • Ideas behind implementing Io.Threaded as whole VS as very OS specific implementation for example Io/Threaded.Windows.zig and other OS alternatives, and if there are any benefits in implementing it as whole or vice versa.
  • Anything to stackfull curoutines / fiber context switches, any material related to that would be really appreciated!!!
  • How concrete the implementations of IO interface are? And if there is still chance it might change before 1.0 releases, if so what parts are might be considered and what are not.
  • Also i am curious what other members of this clever community found interesting that i might look at next, so share anything thank you

Again, shout out to the core team for their work they did and do!!!

2

Wait does zig now have stackful coroutines with stack switching? I didn’t know that (If yes how?).
In my project I ended up using 2 threads ping-ponging off each other and used Asyncify on WASM to simulate that - So if there’s direct language support now it would be perfect.

3

Yes, the fibers code has been in there since at least early 0.16 days, and it’s possible to build coroutines using it.

Have a read of the stdlib src trees, and you will see inline asm for x86 and arm for jumping between fibers.

You can see it in Io implementations for kqueue and Io uring for example. It’s just a bit of a pain though, since it’s embedded in Io implementations which haven’t been fully completed yet (ie - full networking support) .. but it’s still a great resource for doing low-consequence experiments or studying the asm code.

4

Green thread / fiber stack switching doesn’t require language cooperation so anybody could have (and some people indeed have) implemented it already before as a userland library, it just requires some assembly code for snapshotting and swapping cpu state.

It does require though that the machine support messing with CPU state (mainly program counter and stack pointer) and that’s not something WASM allows today, so you won’t be able to use Io.Evented until this proposal is accepted and implemented GitHub - WebAssembly/stack-switching: A repository for the stack switching proposal. · GitHub.

Languages with fibers like golang have to do weird stuff to be able to yield in WASM. Asincify does seem do do weird stuff indeed Pause and Resume WebAssembly with Binaryen's Asyncify.

For a non-hacky solution in Zig, it will either have to be stack switching support in WASM or Zig implementing stackless coroutines.

As an aside, I find the name ‘asyncify’ to be horrifyingly wrong given the definitions we developed. Interruptify would be a much better name, or even better “I can’t believe this is not an interrupt!”-ify.

1 Like
5

It’s also possible to yield and resume with “javascript promise integration” in browsers. That obviously limits it to certain wasm environments.

https://cloudef.pw/sorvi/#train-planner.sorvi implements threads this way.

6

I see, so that’s actual interrupt support for wasm, if I understand correctly. Actual VM suspension instead of stack unwinding.