Hello everyone,
I am currently working on migrating a small vector database storage engine to the new Zig 0.16.0 IO system. I am hitting a runtime issue (test failure) when trying to read a binary struct from a file using std.Io.File.Reader.
Here is the context: I have a Manifest structure that I write to disk and then try to read back inside a test block.
My Code:
const std = @import("std");
pub const Manifest = struct {
magic: [7]u8,
version: u32,
dimension: u32,
vector_count: u64,
metric: u8,
quantization: u8,
reserved: [6]u8,
};
pub fn writeManifest(
io: anytype,
path: []const u8,
manifest: Manifest,
) !void {
var file = try std.Io.Dir.cwd().createFile(io, path, .{});
defer file.close(io);
var write_buf: [1024]u8 = undefined;
var w = file.writer(io, &write_buf);
try w.interface.writeAll(std.mem.asBytes(&manifest));
}
pub fn readManifest(
io: anytype,
path: []const u8,
) !Manifest {
var file = try std.Io.Dir.cwd().openFile(io, path, .{});
defer file.close(io);
var manifest: Manifest = undefined;
const bytes = std.mem.asBytes(&manifest);
var read_buf: [1024]u8 = undefined;
var r = file.reader(io, &read_buf);
// This is where it fails at runtime
try r.interface.readSliceAll(bytes);
return manifest;
}
test "manifest persistence" {
const io = std.testing.io;
const file_path = "test.manifest";
defer std.Io.Dir.cwd().deleteFile(io, file_path) catch {};
const manifest = Manifest{
.magic = .{ 'Z', 'V', 'E', 'C', 'T', 'O', 'R' },
.version = 1,
.dimension = 768,
.vector_count = 10000,
.metric = 0,
.quantization = 1,
.reserved = .{0} ** 6,
};
try writeManifest(io, file_path, manifest);
const loaded = try readManifest(io, file_path);
try std.testing.expectEqual(manifest.dimension, loaded.dimension);
}
The Error:
When running zig test src/storage.zig, the compilation succeeds, but the test fails at runtime exactly on the line:
try r.interface.readSliceAll(bytes);
It seems like the reader is returning an unexpected EndOfStream or a failing error code, even though the file is successfully created and written to beforehand.
My Questions:
- What is the idiomatic way in Zig 0.16.0 to read/write a raw struct from a file using the new std.Io event loop / subsystem?
- Am I misusing the file.reader(io, &read_buf) stack buffers? Should they be sized or aligned differently for readSliceAll to succeed?
Any guidance on how the new intrusive Reader/Writer interfaces orchestrate unbuffered or buffered file reading in 0.16 would be greatly appreciated!
Thanks in advance!