How to do basic file IO

Help
,
1

I am playing around with Zig, so I thought to create a basic program that reads from a file (or stdio) and writes to a file (or stdout). But I was getting pretty confused with Io, File, Reader and Writer. And I am still unsure about a few things.

I now have this for writing to stdout:

    const io = init.io;
    var stdout_buffer: [1024]u8 = undefined;
    var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buffer);
    const stdout = &stdout_writer.interface;
    try stdout.print("Hello world\n", .{});
    try stdout.flush();

And this for stdin:

    var stdin_buffer: [1024]u8 = undefined;
    var stdin_reader = std.Io.File.stdin().reader(io, &stdin_buffer);
    const stdin = &stdin_reader.interface;
    var buf: [1024]u8 = undefined;
    stdin.readSliceAll(&buf) catch |err| {
        if (err == std.Io.Reader.Error.EndOfStream) {
            std.log.info("End of stream reached", .{});
            std.log.info("Read buffer, stdin.seek={d}, stdin.end={d}, logicalPos={d}: \n{s}", .{ stdin.seek, stdin.end, stdin_reader.logicalPos(), buf[0..stdin_reader.logicalPos()] });
        } else {
            std.log.err("Error reading from stdin", .{});
        }
    };

Does this cover the basics for dealing with stdin and stdout?
Is logicalPos() the best way to find out how far the buffer got filled by the last call? (stdin.seek and stdin.end are both 0)
Is the .interface needed? (stdin() already returns a Reader so wouldn’t that already satisfy the interface?)

2

stdin() returns a File.Reader, which, perhaps confusingly, is an implementation of the Io.Reader interface, so it is a different type.

i think logicalPos is better understood as an offset into a file on disk. for stdin specifically, i don’t know what the behavior is…

3

You might be confusing a few things here. Which buffer do you mean?

When you call readSliceAll and pass it a 1KB buffer, it will either fill it fully or fail. You very rarely want to do that, unless you have a binary protocol that needs exactly 1KB chunks.

However, the reader also has a buffer and you might want to use fillMore/buffered/toss functions. Or you might want to use the take* functions which operate on the internal reader’s buffer.

If you want to perhaps read line by line, then takeDelimited is what you want.

2 Likes
4

Maybe have a look at this tutorial, it has some examples how to read line-by-line or per char/byte etc.:

https://codeberg.org/jmcaine/zig-notes/src/branch/main/file-io.md

5

I meant buf, the buffer I am reading bytes into with readSliceAll.

My simple goal (to begin with) was to read the entire input file at once. readSliceAll seemed the way to do that. Problem is, when your file is smaller than the buffer, it doesn’t fill the entire buffer, so I had to find out how many bytes actually got read. So that’s where the logicalPos() got in. But it looks weird to have to use something from stdin_reader to determine how many bytes were read from stdin. Kind of breaks using the interface.

However, the reader also has a buffer and you might want to use fillMore/buffered/toss functions. Or you might want to use the take* functions which operate on the internal reader’s buffer.

I looked at buffered(), and fillMore(), but those don’t advance the seek position. So I’m guessing I will be reading the same bytes over and over. takeArray() does advance the seekposition, but returns an error instead of the last few bytes if not enough bytes are left. (At least, that’s what the documentation says, I did not try it.)

The tutorial referred to by lukeflo, pointed me to readSliceShort() which I overlooked initially. It seems to do what I want, reading from the manual.

Out of time for now, but that’s going to be my next try.

6

It took me quite some time to iron out the details, but I now have something that works.

I have this bit in main():

    var infilename: ?[]const u8 = null;
    // with a bit in between to take an optional filename as a command-line argument
    const io = init.io;

    var input_reader = try getInputReader(io, infilename);
    var input = &input_reader.interface;

    var buf: [1024]u8 = undefined;
    var bytesRead = try input.readSliceShort(&buf);
    while (bytesRead == buf.len) {
        std.log.info("Read {d} bytes, data:\n{s}", .{ bytesRead, buf[0..bytesRead] });
        bytesRead = try input.readSliceShort(&buf);
    }
    std.log.info("Read {d} bytes, data:\n{s}", .{ bytesRead, buf[0..bytesRead] });

getInputReader() was first returning the interface, but that kept causing runtime errors. Took a long while and some llm help[1] to figure out what was wrong.

getInputReader just handles the choice between reading from a regular file or from stdin:

fn getInputReader(io: std.Io, maybeFilename: ?[]const u8) !Io.File.Reader {
    var in_buffer: [8192]u8 = undefined;
    if (maybeFilename) |infilename| {
        std.log.info("Input file: {s}", .{infilename});
        var infile: Io.File = std.Io.Dir.cwd().openFile(io, infilename, .{ .mode = .read_only }) catch |err| {
            std.log.err("Error opening input file", .{});
            return err;
        };
        return infile.reader(io, in_buffer[0..]);
    } else {
        std.log.info("No input file specified, using stdin", .{});
        return std.Io.File.stdin().reader(io, in_buffer[0..]);
    }
}

For this code, I don’t think I need a buffer inside the reader, since I am reading chunks anyway, but I did not see a way to set it up without the in_buffer that is passed to the reader.

Happy to hear whether this approach makes sense to experienced ziggers, but I can now move on with my toy project. Thanks to everyone who responded :folded_hands:

By the way, this was done in Zig 0.16.0

  1. the llm wasn’t actually that helpful, probably because I’m just using the default in VSCode ↩︎

7

Just in case anyone is trying something similar and wants to use this code as an example, the code above needed a bit more tweaking to be robust (no doubt easy to spot for any experienced zigger):

  • I wasn’t closing files
  • the buffer inside getInputReader() either needs to be allocated (and the freed later on, unless allocated in an arena allocator) or created on the stack in main()

Final version (hopefully):

fn getInputReader(io: std.Io, maybeFilename: ?[]const u8, in_buffer: []u8) !struct { Io.File.Reader, ?Io.File } {
    if (maybeFilename) |infilename| {
        std.log.info("Input file: {s}", .{infilename});
        var infile: Io.File = std.Io.Dir.cwd().openFile(io, infilename, .{ .mode = .read_only }) catch |err| {
            std.log.err("Error opening input file", .{});
            return err;
        };
        return .{ infile.reader(io, in_buffer), infile };
    } else {
        std.log.info("No input file specified, using stdin", .{});
        return .{ std.Io.File.stdin().reader(io, in_buffer), null };
    }
}

fn closeFile(io: std.Io, maybeFile: ?Io.File) void {
    if (maybeFile) |file| {
        file.close(io);
    }
}

And called like this in main():

    var in_buffer: [4096]u8 = undefined;
    var input_reader, const maybeInputFile = try getInputReader(io, params.input, in_buffer[0..]);
    defer closeFile(io, maybeInputFile);
    const input = &input_reader.interface;