Um, is there any alternative or zig way of using the C"s stejmp and longjmp mechanism?
Why would I need this? In simple terms, say like trying parse some custom binary data over some socket fd(s), the socket might return “WouldBlock” error while continuously reading and parsing, so I would like to do is if “WouldBlock” is returned while reading from the fd, I would like to terminate the current “parsing” context and save it’s state and continue when some new data is available from the saved state, and this “state” should be simple it should just tell the context what we’re parsing right now, say like we were trying to parse “id: [12]const u8” when we received “WouldBlock” error and would like to save this “state” “we were parsing id” and continue parsing “id” again when data is available.
So what would be a good way of doing it? In C/C++ using setjmp and longjmp feels okay. This feels more like a logical/design problem more than a Zig related problem, but still would like some suggestion as I am trying to implement something like this in Zig as it’s better alternative of C.
There is not something ready. It can be done using OS specific facilities to save and restore registers and I am planning to write such a library for panic recover purposes.
Zig seems to cover your need using the eventedI/O mode, but consider it experimental.
Yes, the most dangerous is that you are restoring the stack pointer to one previous position, so you are going to override anything on stack between setjmp and longjmp.
i.e. this is not a real continuation (you don’t have a copy of the stack).
hmm if i were to do this longjmp at the start of the function depending on the previous saved state using setjmp(which I will likely do if I encounter WouldBlock error) it should be fine? and I won’t be jumping from one function to another function, i was just thinking to replace a big switch case and use this setjmp to mark as the position where i would like to jump next time the parsing function is called.
if you are going to call them in the same function, and you don’t leave this function. There is no reason to call setjmp/longjmp, just use break or any other zig programming facility.
Setjmp/longjmp are included in musl, which is included in Zig, so you should be able to access them. If they are not already exposed in std, you can still declare them with the extern keyword and the linker should find them.
With that said, setjmp/longjmp was designed for error recovery. What you are describing is non-blocking io, so I don’t know if it would work well. Non-blocking io is the main problem that coroutines try to solve. Zig had coroutines builtin with the async and await keywords, but they have been removed. You can implement them by hand, in pure Zig. That would require turning your parsing function into a state machine object. Essentially, every time you need to wait for io, you save the state of this object, probably into an enum, and periodically check if the await has been satisfied, at which point you take the state that you had saved and call the appropriate function to continue the parsing.
You can find many examples online of doing it. I recently watched this talk, which implements this. However, the target of the talk was even deeper and more complicated than simply socket reading. The presenter’s goal was to hide the latency of memory reads. The idea is the same, it’s all non-blocking io, but reading a file or a socket is much easier than what he was doing, so maybe this isn’t a great talk if you’re new to the subject.
In my project, which involves reading medical images, state machine coroutines were not enough. I had some very complicated parsing to do. The files don’t have much regularity. The headers can vary from extremely long to extremely short. Many fields were optional. The parsing of the next fields depends heavily on the fields that came before. And the worst part is that some fields can take just a byte, while others can take megabytes. This means that, all of sudden, I could find myself starved of bytes to read, and had to block. I could not find a way to make the state machine work. The code became a huge mess of branches, and I couldn’t even finish it. At every tiny step, I would have to check if I had bytes to read, and implement a way to come back to the point where I had left.
This is where the other form of coroutines enter the scene. I had to implement what is called stackful coroutines, also known as fibers, as opposed to the state machine coroutines that I described above, which are called stackless coroutines. Fibers allow you to really stop a computation, at any point, without any changes to your code, and resume exactly where you left off. The idea is extremely simple. Freeze the stack and the registers, then come back later. Implementing it requires language support, which Zig doesn’t have, or assembly, which is what I had to do.
// may be called in a loop which checks for if some data
// is available on the socket fd.
pub fn parse(self: *This()) !void {
// checks whether some previous state was there or not...
if (self.has_saved_state()) |prev_state| {
longjmp(prev_state, 1001);
}
// parse1 returns the reading error raised by the socket fd
some_parsing1() catch |err| {
if (err == error.WouldBlock) {
var jmpbuf = undefined;
if (setjmp(jmpbuf) == 0)
self.save_state(jmpbuf);
return;
} else {
return err;
}
};
// other parses like this?
}
ah ye i think i can sort of understand the pain you had in that project and i am also kind of facing same sort of parsing issue where all the parsing steps depends on the previous step, i think this stackful coroutines would make my life much much easier, btw what kind of solution you came up writing assembly directly? or didn’t finish up yet?
I have sort of finished it. I’m already using it in my project. It decreased the time to load my test data from 155ms to 95ms. But I wrote it specifically for my use case. If you’re curious, the code is here.
I’ve made it for the windows 64 abi. Adapting it to Linux shouldn’t be too hard.