Ourio doesn't reach EOF

I have a minimal program here https://codeberg.org/knightpp/ourio-repr/src/branch/main/src/main.zig.

The program uses ourio lib by @rockorager https://github.com/rockorager/ourio/ and designed to read from stdin and print number of bytes read. So then to use it I call echo test | zig build run. But when it’s started by redirecting file like this zig build run < file.txt → read doesn’t return 0, so there’s no way to determine EOF?

const std = @import("std");
const io = @import("ourio");
const posix = std.posix;

pub fn main() !void {
    var gpa = std.heap.DebugAllocator(.{}).init;
    defer _ = gpa.deinit();

    const allocator = gpa.allocator();

    var rt = try io.Ring.init(allocator, 16);
    defer rt.deinit();

    const stdin_fd: posix.fd_t = posix.STDIN_FILENO;

    const fd = stdin_fd;

    const original_mode = try posix.fcntl(fd, posix.F.GETFL, 0);

    var opts: posix.O = @bitCast(@as(u32, @intCast(original_mode)));
    opts.NONBLOCK = true;

    _ = try posix.fcntl(fd, posix.F.SETFL, @as(usize, @as(u32, @bitCast(opts))));
    defer {
        _ = posix.fcntl(fd, posix.F.SETFL, original_mode) catch |err| {
            std.log.err("could not restore fd's mode: {}", .{err});
        };
    }

    var app = Runner{ .in = fd };
    try app.start(&rt);

    try rt.run(.until_done);
}

const Runner = struct {
    in: posix.fd_t,
    buf: [4096]u8 = undefined,

    fn start(self: *Runner, rt: *io.Ring) !void {
        try self.read(rt);
    }

    fn read(self: *Runner, rt: *io.Ring) !void {
        _ = try rt.read(self.in, &self.buf, .{
            .ptr = self,
            .msg = 0,
            .cb = Runner.cb,
        });
    }

    fn cb(rt: *io.Ring, task: io.Task) anyerror!void {
        var self = task.userdataCast(Runner);
        const result = task.result.?;

        const n = try result.read;
        if (n == 0) {
            return;
        }

        std.debug.print("{d} bytes\n", .{n});

        try self.read(rt);
    }
};
1 Like

What if you run it on the build artifact rather than via zig build run?

1 Like

The same behavior. It just loops reading again and again.

$ echo test | ./zig-out/bin/atee
5 bytes
$ ./zig-out/bin/atee < build.zig.zon
2279 bytes
2279 bytes
2279 bytes
... # goes on

Switched to bash (previously was fish) and tried in a container but with the same result.

If using echo test | ./zig-out/bin/atee, standard input is a pipe, read bytes are consumed.

If using ./zig-out/bin/atee < build.zig.zon, standard input is a regular file, which can be read multiple times.

In your case the file is small enough to fit into the buffer and is read in one go, then you start another read, which is expected to return 0 bytes read, but seems to read the whole file again. You might dive into the read part of ourio to check whether there is something always reading files from the beginning.

1 Like

Thanks. I think I found something, at least it fixed my use case.

According to io_uring_prep_read(3) man page:

On files that support seeking, if the offset is set to -1, the
read operation commences at the file offset, and the file offset
is incremented by the number of bytes read. See read(2) for more
details. Note that for an async API, reading and updating the
current file offset may result in unpredictable behavior, unless
access to the file is serialized. It is not encouraged to use this
feature, if it’s possible to provide the desired IO offset from
the application or library.

On files that are not capable of seeking, the offset must be 0 or
-1.

If I change ourio/src/ourio/Uring.zig at 54a2eae0fe28d5258b0982ea66b77e3d340e376c · rockorager/ourio · GitHub to

        .read => |req| {
            const sqe = self.getSqe();
            sqe.prep_read(req.fd, req.buffer, @bitCast(@as(isize, -1))); // change is here, offset was always 0
            sqe.user_data = @intFromPtr(task);
            self.prepDeadline(task, sqe);
        },

I guess it’s safe to change the default?

1 Like

libxev uses -1 by default libxev/src/backend/io_uring.zig at 58507577fc87b89471809a1a23415bee1d81814d · mitchellh/libxev · GitHub