Alternative std.Io implementation in zio

lalinsky · December 18, 2025, 9:02pm

My point was that on modern Linux, you will not get OutOfMemory, you will get overcommitted pointer that isn’t backed by anything and using it might trigger SIGSEGV at any point.

invlpg · December 18, 2025, 9:09pm

The recursion case is an open problem, see: safe recursion · Issue #1006 · ziglang/zig · GitHub

mperillo · December 19, 2025, 8:48am

When you are in control of the stack usage, if there isn’t enough space for a coroutine, you just put it in a queue, or pause until another coroutine finishes and frees up some space.

If the queue is very large, doesn’t that mean an OOM may be caused by the queue allocator?

LucasSantos91 · December 19, 2025, 1:41pm

Yes, in which case you can go do other things and try again when memory becomes available. And If the current task cannot move forward until it posts the new task on the queue, you can suspend the current task, and come back later.

mperillo · December 19, 2025, 1:48pm

From my experience, recovery from a OOM on linux is not simple. In almost all cases I had to shutdown the PC.

invlpg · December 19, 2025, 2:20pm

On a desktop, if you’ve managed to free up enough RAM but still have a bunch stuck in swap that’s slowing your system down, I’ve had some success doing swapoff /dev/$SWAP_PARTITION; swapon /dev/$SWAP_PARTITION.

As long as nothing system-critical has been killed that should get you back to a responsive state (you can check what was killed afterwards in the logs once your system is responsive again, and decide then whether a reboot is immediately needed).

lalinsky · December 19, 2025, 10:10pm

To make my development easier, I’ve backported std.Io to Zig 0.15. I think it might also help with adoption, if people can start using it before Zig 0.16 is released. I’ve only backported the interface, I’m definitely not going to be backporting the threaded implementation, but if there is enough interest, I could make a simplified version for blocking use cases.

Updated example (same as before but using zio.Io instead of std.Io if running on Zig 0.16):

github.com/lalinsky/zio

examples/tcp_echo_server_stdio.zig

main

const std = @import("std");
const builtin = @import("builtin");
const zio = @import("zio");

const Io = if (builtin.zig_version.order(.{ .major = 0, .minor = 16, .patch = 0 }) == .lt) zio.Io else std.Io;

fn handleClient(io: Io, stream: Io.net.Stream) void {
    defer stream.close(io);

    std.log.info("Client connected from {f}", .{stream.socket.address});

    var read_buffer: [1024]u8 = undefined;
    var reader = stream.reader(io, &read_buffer);

    var write_buffer: [1024]u8 = undefined;
    var writer = stream.writer(io, &write_buffer);

    while (true) {
        // Use Reader delimiter method to read lines
        const line = reader.interface.takeDelimiterInclusive('\n') catch |err| switch (err) {

This file has been truncated. show original

lalinsky · December 19, 2025, 11:08pm

Also starting to write docs:

D-Berg · December 20, 2025, 10:24am

Works flawlessly for me for my blog server both with using a fba of 16MiB and gpa, now I do get more requests per sec with Io.Threaded but that is probably because of how I structured the code. I launch const n_workers = 4 * (std.Thread.getCpuCount() catch 1); workers with Io.Group concurrent which wait on a queue. One thing is that the cpu usage is a lot lower with zio and therefore I could probably bump the worker count a lot and get more req/s.

Excellent work done!

github.com/D-Berg/blog

src/server.zig

main

const std = @import("std");
const Io = std.Io;
const Allocator = std.mem.Allocator;
const net = Io.net;
const log = std.log;
const http = @import("http.zig");
const Router = http.Router;

const assert = std.debug.assert;
const Method = http.Method;
const Url = http.Url;
const Request = http.Request;
const Response = http.Response;

const Event = union(enum) {
    connection: Io.net.Stream,
    should_close,
};

pub fn Server(comptime Context: type) type {

This file has been truncated. show original

zio gpa heap

wrk -t8 -c400 -d30s http://127.0.0.1:6969
Running 30s test @ http://127.0.0.1:6969
  8 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   342.11us   97.59us   7.12ms   92.70%
    Req/Sec    17.57k    15.40k   46.53k    80.07%
  2631706 requests in 30.10s, 2.16GB read
  Socket errors: connect 155, read 0, write 0, timeout 0
Requests/sec:  87424.06
Transfer/sec:     73.54MB

zio fba no-heap

wrk -t8 -c400 -d30s http://127.0.0.1:6969
Running 30s test @ http://127.0.0.1:6969
  8 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   300.72us   85.38us   6.57ms   89.31%
    Req/Sec    19.98k    13.41k   53.30k    77.87%
  2991675 requests in 30.11s, 2.46GB read
  Socket errors: connect 155, read 0, write 0, timeout 0
Requests/sec:  99374.58
Transfer/sec:     83.59MB

Io.Threaded gpa heap

wrk -t8 -c400 -d30s http://127.0.0.1:6969
Running 30s test @ http://127.0.0.1:6969
  8 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   323.79us    1.02ms  88.27ms   98.99%
    Req/Sec    18.26k    10.76k   42.51k    56.20%
  2727203 requests in 30.07s, 2.24GB read
  Socket errors: connect 155, read 0, write 0, timeout 0
Requests/sec:  90689.68
Transfer/sec:     76.28MB

Io.Threaded fba (no-heap)

wrk -t8 -c400 -d30s http://127.0.0.1:6969
Running 30s test @ http://127.0.0.1:6969
  8 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   301.77us    1.02ms 109.47ms   99.77%
    Req/Sec    20.88k    11.88k   43.14k    65.98%
  3116399 requests in 30.08s, 2.56GB read
  Socket errors: connect 155, read 0, write 0, timeout 0
Requests/sec: 103618.46
Transfer/sec:     87.16MB

lalinsky · December 20, 2025, 10:40am

One comment on the code, you shouldn’t be doing this:

while (!io.cancelRequested())

Just let the accept() check it for you and return error.Canceled.

The function has been removed from the latest Zig master, but also it’s semantically wrong. Cancellation signaling is more complex than that.

lalinsky · December 20, 2025, 10:49am

Also, regarding the worker count, you can safely just call io.concurrent per request. It’s efficient enough to handle it. There are still some things in zio that I want to optimize, but it’s already fine. By default, you are running zio only on a single thread, but the single thread can handle a lot of connections.

D-Berg · December 20, 2025, 10:51am

Reason I have that is that when coupled with fba and Io.Threaded it runs out of memory resulting in ConcurrencyUnavailable, atleast it did before.

D-Berg · December 20, 2025, 11:04am

Checking master branch, io.cancelRequested is still there, looks like it got reintroduced but maybe one still should’nt be using it like I am.

lalinsky · December 20, 2025, 11:18am

Hm, the function is still defined on std.Io, but it’s missing in the std.Io.VTable, so it would fail to compile.

I was kind of glad that it got removed from the vtable, because it’s a tricky situation. In zio, I handle cancellations in a way, that allows you you safely recover from it and clean up as needed. When you run io.cancel, you receive only one error.Canceled and you need to propagate it until the task exits. That allows defers and things like that to run without any problem. In the native API, I have a mode that shields functions from cancellation, but there is no equivalent in std, so this behavior of just receiving one error makes more sense.

lalinsky · January 4, 2026, 6:50am

I think I have to abandon the plans for the std.Io implementations. I was hoping it would be more stable, but as I finished this, new wave of changes came and there is no way I can keep it updating the code. I’ll wait until Zig 0.16 is closer to release date. If there is a lot of interest, I could revisit this, but based on the feedback, there probably isn’t.

lalinsky · January 5, 2026, 11:56pm

I’ve completely removed the std.Io implementation. The interface is growing too large and zig master is breaking all the low level APIs, so it’s extremely hard to maintain two versions. This was a failed experiment. I have a feeling that version 0.16 will be zig’s python 3 moment. I think the std.Io interface is a great initiative, but removing/changing all the lower-level APIs is a mistake, in my opinion.

LucasSantos91 · January 6, 2026, 12:44am

Yeah, I think some of these APIs should be kept to be used as building blocks for the Io implementations. Like fs.File, a file abstraction that is independent of Io, could be used internally by the Io implementation itself.

andrewrk · January 6, 2026, 2:55am

Low level APIs are all still there. You’re thinking about the medium level APIs. The ones that had a hate letter written to them.

If you’re making an I/O implementation you should have already been dipping into the lowest layer. It’s a code smell if the removals from std.posix affected your code.

Python 2 was foolishly supported for over a decade after Python 3 was released. Zig isn’t 1.0 yet and there won’t be a 0.15.3.

lalinsky · January 6, 2026, 6:32am

What mainly affected me were changes to the std.os.linux and std.os.windows. When I saw what is happening to std.posix, I started creating my own wrappers. But I didn’t want to re-create the lowest level ones, which kept changing every day, as they were being used by std.Io.Threaded.

If I fully committed to Zig 0.16, I’d have probably just keep updating the code, essentially matching the development of std.Io.Threaded, but given that 0.15 is the more important target to me, I’d end up with two branches that are more and more diverging, and yet have to be kept in sync. And that’s exactly the problem I was facing with Python 2/3.

Any networking library, that people currently use, basically can’t be easily migrated to 0.16. So this will create a dependency problem, because a new ecosystem has to arise, exactly like Python 2/3.

I understand the need for the changes and the cleanups are necessary, but given the scope of them, it will slow down adoption.

My new plan is to basically wait for the release of 0.16 and then use std.Io.Threaded as the base layer for implementing all the vtable functions that are not relevant to my runtime and can’t be realistically done in non-blocking fashion.

andrewrk · January 6, 2026, 6:54am

Huh?? There’s barely any changes to std.os.linux that aren’t bug fixes. And you should definitely not be using the std.os.windows wrappers in an Io implementation!