Io.Reader version of `File.readToEndAllocOptions`

Help
1

I’m trying to upgrade one of my hobby projects to 0.15.1. I was using File.readToEndAllocOptions to get a [:0]const u8 from the file.
The reason I need a null terminated string specifically is that std.zon.parse.fromSlice requires it and I’m parsing a config file in zon.

The Io.Reader interface has a readAlloc that will get the full contents of the file, but no way to null terminate it. It gives the contents without the null. Any ideas on how to work around this?

Previous code:

        const config_data = try home.openFile(".config/mailbox/config.zon", .{ .mode = .read_only });
        const config_bytes = try config_data.readToEndAllocOptions(alloc, 2048, 64, @alignOf([:0]u8), 0);
        defer alloc.free(config_bytes);
        var status: std.zon.parse.Status = .{};  // This is changed to Diagnostics in 0.15.1
        const local_config = try std.zon.parse.fromSlice(Config, alloc, config_bytes, &status, .{
            .ignore_unknown_fields = true,
        });
2

I came up with a solution, but it’s a little janky. Basically allocating the buffer + 1 based on the length and then manually null terminating it. If anyone has a better solution, I would appreciate the suggestions. There’s a lot of len - 1 math here that would be easy to misplace.

        const config_data = try home.openFile(".config/mailbox/config.zon", .{ .mode = .read_only });
        var read_buf: [256]u8 = undefined;
        var reader = config_data.reader(&read_buf);
        var config_bytes = try alloc.alloc(u8, try config_data.getEndPos() + 1);
        defer alloc.free(config_bytes);
        try reader.interface.readSliceAll(config_bytes[0 .. config_bytes.len - 1]);
        config_bytes[config_bytes.len - 1] = 0; // Null-terminate for zon parser
        var status: std.zon.parse.Diagnostics = .{};
        const local_config = try std.zon.parse.fromSlice(Config, alloc, config_bytes[0 .. config_bytes.len - 1 :0], &status, .{
            .ignore_unknown_fields = true,
        });
3

I think you can use what std.fs.Dir.readFileAllocOptions uses?:

file_reader.interface.allocRemainingAlignedSentinel(gpa, limit, alignment, sentinel)

std.Io.Reader.allocRemainingAlignedSentinel

1 Like
4

That definitely looks like what I want, but that was added (the day) after 0.15.1 release. So that will be the answer going forward. Thanks for the link!

1 Like
5

Here’s a version that read the whole file directly into the Reader’s buffer using fill(), rather than doing it with 256 chunks. Should have the least amount of buffer re-allocation and copying.

    var file = try std.fs.cwd().openFile(filename, .{ .mode = .read_only });
    defer file.close();

    const file_size = try file.getEndPos();
    const read_buf = try allocator.alloc(u8, file_size + 1);
    var file_reader = file.reader(read_buf);
    try file_reader.interface.fill(file_size);
    read_buf[file_size] = 0;
    const data0 = read_buf[0..file_size:0];
2 Likes
6

allocSentinel is less tedious. not that it is too tedious.

1 Like
7

If you don’t actually need to keep the open File around after reading the data, you can just use std.fs.Dir.readFileAllocOptions (0.15.1 version) instead of handling the openFile → reader stuff yourself.

// 0.15.1
const config_bytes = try home.readFileAllocOptions(allocator, ".config/mailbox/config.zon", std.math.maxInt(usize), null, .of(u8), 0);
defer allocator.free(config_bytes);

// master
const config_bytes = try home.readFileAllocOptions(".config/mailbox/config.zon", allocator, .unlimited, .of(u8), 0);
defer allocator.free(config_bytes);
4 Likes
8

I forgot about that variant. That would definitely make it less tedious.