Unexpected outcome with std.io.bufferedWriter

Help
1

I am trying to take one file as input and a second file as a reformatted version of the old file. I am seeing a weird output where if I use a buffered writer the last 475 characters get left out of the file and with bigger files more characters are cut off. If I use the default file writer I do not see this issue. (using zig 0.13.0)

If I do this on small files (~2KB) nothing gets printed. For bigger file (4.5MB+) parts are cut off. The cutoffs appear to be within a 4KB boundary. If I start with a 14KB file, the output value is 12KB, 15KB to 12KB, 17KB to 16KB etc.
Am I using the buffered writer incorrectly? Or is there some nuance of it that I am missing? It appears it writes only when the 4096 byte buffer is full.

const std = @import("std");
const cwd = std.fs.cwd();
pub fn main() !void {
    const read_file = try cwd.openFile("E_coli.txt", .{.mode = .read_only});
    defer read_file.close();
    var buf_reader = std.io.bufferedReader(read_file.reader());
    const reader = buf_reader.reader();

    const new_file = try cwd.createFile("E_coli.fna", .{});
    defer new_file.close();

    var buf_writer = std.io.bufferedWriter(new_file.writer());
    const writer = buf_writer.writer(); 
//problem appears if the 2 above lines are present
//no issues if I use new_file.writer().write() directly

    _ = try writer.write(">E. coli chromosome\n");

    var line_buffer:[80]u8 = undefined; //enforces line size
    while (try reader.read(&line_buffer) != 0) {
        _ = try writer.write(&line_buffer);
        _ = try writer.write("\n");
    }
}
2

You likely need to call flush() on the writer. It is buffered, and isn’t actually writing to the underlying stream each time you write to it. When you call flush, you tell it to “write pending data in buffer now”, and should always be called before closing it.

3 Likes
3

It’s also a good idea to use readAll and writeAll to make sure that the entire line buffer gets processed.

4

@ForeverZer0 That did fix it. Thank you.

1 Like