This is a follow up post about async I/O in Zig that talks about these concepts in a more general manner. Hopefully this will clarify some of the ideas behind the new design that I feel were not fully grasped by a lot of people from the previous post.
25 Likes
the term asynchronous is crying in a corner
7 Likes
I’m glad these issues are brought to the forefront for discussion. I can see where you’re coming from in the case of async != concurrency.
From my understanding, concurrency just means the tasks are executed out of order, whether the tasks are executed in parallel, serially, or interleaved is inconsequential.
Async is related to concurrency but not the same thing. Async just means that your program can perform non-blocking operations, and out-of-order as an implication. A concurrent task while running in a concurrent system can block. While the task blocks, the system switches another task to run.
One thing I don’t agree is running async tasks in “single-threaded blocking mode.” There is no such thing. You simply cannot block async tasks. They will deadlock once they have dependency.
If you stick with aysnc tasks as non-blocking operations, both of your examples, file writing and client/server, would work. That means Server.accept() and Client.connect() cannot block. The non-blocking versions of accept() and connect() should be used.
did you not read the whole blog post?
there is an entire section about how the client/server example doesnt work without concurrency.
that depends on what the task is doing, the file writing example on the blog is an example of a task that doesn’t deadlock.
Running io.async in “single-threaded blocking mode” thing will be very difficult. Blocking-async is really not a thing. Marking something async means non-blocking. Server.accept() and Client.connect() should be made non-blocking if possible.
If Server.accept() and Client.connect() are legacy code that block, it would be better to have a concurrent thread pool to run them. The API to the concurrent thread pool can be async. Then you can have API like the following. (I don’t know the io API so I’m just pulling something out of the thin air.)
await io.async(threadPool.run(Server.accept, .{server, io}));
await io.async(threadPool.run(Client.connect, .{client, io}));
or threadPool.run() returns an io.async task object.
await threadPool.run(Server.accept, .{server, io});
await threadPool.run(Client.connect, .{client, io});
The file writing example has no dependency; that’s why it works.
You are taking async to always be non-synchronous.
The approach zig is working on is more expressing that code CAN be asynchronous, what actually happens is dependent on the io runtime being used.
Loris made a post about how that works, which I assume you haven’t read from your lack of understanding of the io interface
Can we get back to the issue? Running async calls on blocking functions with “single-threaded blocking mode.”
When calling a function in my code, I don’t know whether a function I’m calling and its callees would block or not, in order to wrap it with the correct flavor of io.async.
If functions can be marked as async or not in its signature, then the calling side is simple. The io.async wrapping can figure it out from the function signature - async marked is non-blocking and the one without is blocking. If function signature async marking is not supported in the language, it’s rather unpleasant for the developer making the function call to determine the correct io.async wrapping for every single async call to functions.
Unless the executive running the wrapped async tasks is threaded. In that case I don’t care when making async calls.
But I guess the “single-threaded blocking mode" requirement making it easy to deadlock, and thus requiring the use of asyncConcurrent as stated in the blog. But again, that just forces the blocking/non-blocking decision back to the developer making the function call.
I would rather we get rid of the “single-threaded blocking mode” requirement and always run in threaded (or event-poll) mode. Or we add async marking to function signature, so that the async wrapping can decide to use “single-threaded blocking mode” or threaded for the function call.
Nice, thanks for these definitions, it’s much clearer now.
Since asynchrony/concurrency are expressed in code as io.async()/io.asyncConcurrent(), which are used to schedule specific functions, I thought it also useful to reframe those definitions from the perspective of tasks:
-
Task is asynchronous if it may be executed out-of-order with other asynchronous tasks.
-
Task is concurrent if it is asynchronous and can be executed simultaneously with other asynchronous tasks.
2 Likes
what io async function you use, if at all, depends on your requirements of the function being called, less so on what the function does.
Regardless, non-obvious behaviour should be communicated through docs.
The developer is the best equipped to be answering that question. The function being called doesn’t know where it’s being called from or how. The io implementation can’t assume what the developer wants, that’s what the different async functions exist to communicate.
It was a poor choice of words to use “mode”, there arent async runtime modes in zig, much like allocators, there are concrete implementations that you then get an instance of an interface from, which is then passed to whatever needs it.
If you want to not use “single threaded blocking mode” that’s a choice you get to make, just don’t use that implementation.
That choice is exactly what zig’s async is made for, code can express the synchronicity/concurrency/lack there of, and the users of that code can decide what that expression actually boils down too.
yes, correct, “out of order” also includes making a bit of progress with one task and then make some progress with another asynchronous task. from the post:
A could be saved before B, or B could be saved before A, and all would be fine. You could also write some of A, then some of B, and then back to A, to finally complete writing B. That also would be correct, and in fact that’s what tends to happen when using evented I/O to save sufficiently complex files concurrently.
With my definitions all asynchronous tasks can be run concurrently, but for some it’s not just an option, it’s a requirement in order to be able to make progress.
In the post saveFileA is asynchronous with respect to saveFileB and Server.accept is asynchronous with respect to Client.connect, but Server.accept and Client.connect MUST also be executed concurrently.
2 Likes
I always find it a bit strange that Windows is never coming up in async/await discussions, since Win32 Events were already very similar to awaitable futures/promises, and without requiring the whole async/await machinery.
E.g. (pseudo code):
// block until a single operation finishes
const ev1 = write(file1);
wait(ev1);
// block until any operation finished
const ev1 = write(file1);
const ev2 = write(file2);
const ev3 = write(file3);
wait_any(ev1, ev2, ev3);
// block until all operations finish
const ev1 = write(file1);
const ev2 = write(file2);
const ev3 = write(file3);
wait_all(ev1, ev2, ev3);
wait, wait_any and wait_all are just called WaitForSingleObject() and WaitForMultipleObjects():
The main difference to Promises is that Events cannot carry a result or error payload.
1 Like
These Windows APIs are used internally in Zig, such as std.io.poll
But they seem like a bad model for the async/await api surface, for the reason you stated at the end.
1 Like
I’m sorry I still have problem with forcing the notion of “concurrency” into the async API.
Io.async at the API surface should be used for turning a function call into a future task, which can be run in any order and will be completed in a future time. That’s it. Whether a function call has a hard requirement to run under a thread (concurrent) is a separate issue.
I assume the Thread.spawn() will have an async version that returns an io.async task. It’s cleaner and more composible to wrap the Server.connect() call with Thread.spawn(io, Server.connect, …).
The thread calling the function can mark the io task completed when the function call returns.
In this way, the io.async API doesn’t need to make exception for the “concurrent” requirement, keeping it pure and simple.
No, it’s an integral part of the issue. If your task requires to be run concurrently (multithreading, green threads, stackless coroutines) then it must be invoked via io.asyncConcurrent.
If you use io.async, then Zig is allowed to run the function to completion immediately if it needs to. This is what happens when you choose a single-threaded blocking I/O implementation, or when any of the other Io implementations run out of resources (e.g. not enough memory to allocate a new green thread).
The example in the blog post is about a server and a client where the server task will not complete untill it can successfully communicate with the client.
If you do:
io.async(Sever.accept);
io.async(Client.connect);
Then Zig is allowed to run Server.accept immediately and block there, but that’s a bug because we need to guarantee that both tasks run concurently for the computation to complete.
io.asyncConcurrent guarantees that eventually io.async(Client.connect) will be executed.
6 Likes
I think having Thread.spawn(io, ...) would be better since we can continue to use dependency injection to abstract away the IO implementation from the thread logic, unless that’s not possible. That way we can continue to avoid having dual implementation of API function calls as much as possible.
In terms of having an io.asyncConcurrent call, would it be better to have concurrent as an optional argument? Since we are already doing io.async(SomeFunctionPointer, .{a, b} we could also have it like io.async(SomeFunctionPointer, .{a, b, .concurrentRequired}). That reduces the API methods and can take advantage of using an enum or just a simple Boolean to be explicit about the concurrency requirement.
I’m wrapping my head around the new asynchrony, concurrent, parallel definitions. I’m used to the more traditional parallel/concurrency stuff that’s done in Python, which is similar to a bunch of other languages most likely. There was this article I read last year that helped me better understand the terms and visualize the implementation a bit better. Going with the burger example, they didn’t really talk about the asynchrony part but I suppose there may have always been some sort of implicit assumption of expressing intent of async (at least at a higher level) in that everything in this event loop can be run concurrently, but if we obviously go at a line by line level, this isn’t true. For Zig we will have fine grain control to explicitly express this so this is why it seems we need to define the terms a bit clearer. So in my mind concurrency essentially assumed we would be running within a single thread, and you are doing multiple tasks within a single thread, where each task blocks and yields. So if you were using a single core / single threaded system, you could still maximize the usage of that core by using the “dead time” to do something else in the meantime. The parallel part was more about having multiple things happening at exactly the same time using different cores. Thus you could have multiple concurrent tasks running in separate cores all running in parallel. So all of these terms and assumptions I’ll need to redefine and think about them with the new blog posts Loris wrote (thanks for that, I’ve read them both and will need to re-look at them again 
).
Maybe I don’t understand the semantic contract of io.async() here since it runs the function immediately. I thought io.async()'s contract is to wrap the function call into an async task. The task can run immediately or later. It’s the await() that forces the task to run if it has not run yet.
if
io.async(Sever.accept);
io.async(Client.connect);
is run immediately and that forces an ordering of execution on Server.accept and Client.connect one after another, then the semantic is wrong.
Function calls wrapped with io.async() should be able to run in any order. It’s the
f_serverAccept.await();
f_clientConnect.await();
that forces an ordering.
In any case, moving the “concurrent” requirement to Thread.spawn() solves this problem.
f1 = io.async(io, Thread.spawn(io, Server.accept, ...));
f2 = io.aysnc(io, Client.connect, ...);
f1.await();
f2.await();
seems reasonable and composible.
From my understanding io.async is used to express intent that something /can/ be ran concurrently (asynchrony) not that it /will be/ necessarily. This is because the actual IO logic would be an implementation detail of the concrete object implementing the io interface. Thus you could have your code expressing intent at different places of the code using io.async, but could use a single threaded IO implementation, which would mean your code would work, would still have the intention clearly documented in the code (which parts can be concurrent or not), but the actual system would be single threaded with no event loop.
3 Likes
The solution that you are presenting it exactly what io.asyncConcurrent does, with the clarification that it doesn’t spawn a thread, but rather the appropriate unit of concurrency that belongs to the concrete implementation (a thread for the thread pool implementation, a green thread for the green therads implementation, etc).
4 Likes