How do I read text files in chunks?

Help
, ,
1

I’m attempting to use zig to solve the day 1 of advent of code 2025.

I would like to read chunks of a text file to parse the puzzle input.

(Silly question: Can you deallocate the buffer used to read the entire file after it is not longer needed?)

My initial idea was to use a buffer to read part of the file, advance by a single “step”, load the next character at the end of the buffer while discarding the first character in the buffer.
Like a window sliding across the text file. The bad news is: I don’t know how to do it.

Here’s what I have so far:

fn get_input(buffer: []u8) !usize {
    const dir = std.fs.cwd();

    const file = try dir.openFile("input.txt", .{});
    defer file.close();

    const read_b = try file.read(buffer);
    return read_b;
}

pub fn main() !void {
    var buffer: [20000]u8 = undefined;
    const buffer_len = try get_input(&buffer);
    const input = buffer[0..buffer_len];

    // input -> Instruction

    var head_idx: usize = 0;
    var inst_list_idx: usize = 0;
    var inst_list: [4257]Inst = undefined;

    while (head_idx < input.len) {
        var inst = Inst{ .move = 0, .ticks = 0 };

        if (is_letter(input[head_idx])) {
            const l = parse_letter(&head_idx, input);
            inst.move = l;

            assert(is_number(input[head_idx]));

            const n = try parse_ticks(&head_idx, input);
            inst.ticks = n;

            if (head_idx >= input.len or input[head_idx] == '\n') {
                inst_list[inst_list_idx] = inst;
                inst_list_idx += 1;
                head_idx += 1;

                print("Inst( {d} {d} )\n", .{ inst.move, inst.ticks });
            }
        }
    }
// ... more code
}
2

Hi @tau, welcome to Ziggit!

In the code provided, no. The buffer used is stack allocated and so you would not deallocate. You will only need to deallocate if you pass an allocator to something as a general rule.

For you “advance by a single ‘step’” case, you are likely looking for the std.Io.Reader interface. This provides functions for things like takeByte, peekBuffer, etc. You can see more here: Zig Documentation

Now, you can get the reader directly from the file:

const dir = std.fs.cwd();
const file = try dir.openFile("input.txt", .{});
defer file.close();

var read_buffer: [256]u8 = undefined;
var file_reader = file.reader(&read_buffer); // Returns std.fs.File.Reader
var reader = &file_reader.interface; // Points to the std.Io.Reader interface inside the std.fs.File.Reader

// Access the functions directly
_ = try file_reader.interface.peekByte();
// Or through the interface
_ = try reader.peekByte();

In this case, the reader will put bytes into read_buffer and then read more as needed. As you take more and more bytes, it will automatically load more bytes into the buffer when it runs out of the current buffered bytes. This should give your sliding effect.

3

Both are through the interface. reader is the pointer to interface.

4

Correct, perhaps I wasn’t clear enough. I meant more along ght lines of through the stored reader pointer. The point was illustrating how to do it in two different syntactic ways.

5

The syntax is the same, one is just shorter, which has the unfortunate risk of accidentally copying the interface when making reader instead of taking a pointer, which will cause illegal behaviour.

So I have been recommending to prefer file_reader.interface.foo(), it’s more verbose, but it doesn’t risk illegal behaviour which could do all sorts of hard to debug things.

1 Like
6

First of all, let me tell you that I’m very grateful for your response.
I’d usually search frantically for examples in somebody else’s code or in the documentation and I must admit that asking was a good idea.
Sorry if I’m asking basic stuff. I’m still learning the language.
I haven’t implemented your suggestions yet but I’m planning to do so.

7

I’m curious about this reader.interface business.
What is the deal with it?

8

The Reader/Writer interfaces do not store a pointer to the implementation’s state, unlike Allocator which does. This is a problem as most implementations require having their own state.

This is solved by 1) the interfaces passing a pointer to themselves to the implemented functions (needed for other reasons anyway) and 2) the implementation state storing the interface as one of its fields (a specific field).
The implementation functions use the assumption that the interface is in a particular field to do some pointer magic to get a pointer to its own state.

The foot gun is the implementation currently has no way to know if the interface is actually stored in its own state. so if you copy the interface out and use that copy, then the implementation will pointer magic itself an invalid pointer to some other memory. Using that pointer is now undefined behaviour.
If you’re lucky, you will get a crash that shows obvious signs of undefined behaviour/memory corruption, such as a segfault, or an switching on an invalid enum value (only in safe modes).
If you are not lucky, it will somehow work, only to later do weird things in production/mission-critical context.

The reason I recommend doing reader.interface is the alternative that @Southporter suggestion is one character away for undefined behaviour:

const valid = &reader.interface;
var invalid = reader.interface; // missing `&`

good news is runtime safety checks for this and other pointer magic are in the works

updates will be found at codeberg, you wont find it there currently as there are no updates.

edit: I should add that there are advantages to this ‘intrusive’ style interface:

  • it has a smaller memory footprint, Reader/Writer are already larger than Allocator
  • the implementation is allowed and expected to modify the interfaces own state. This allows the implementation to do so, outside of calls through the interface.
  • The above allows for implementation specific extensions, such as File.Reader/Writer can seek, and update the interface state accordingly automatically.
3 Likes
9

That’s part of why we have this forum. To ask questions and facilitate learning. Ziggit is full of people that want to help, so ask away.

3 Likes