Help using std.io.FixedBufferStream to get std.io.Reader from an ArrayList

Help
,
1

What I am really trying to do: test a function that accepts a std.io.Reader (as an anytype), by populating something in a test, and then passing that something as a reader to the function I’m testing.

How I am trying to do it:

  • Make an ArrayList
  • Call list.appendSlice with some values that may be determined at runtime
  • Wrap the ArrayList.items in a std.io.FixedBufferStream, so that I can use fbs.reader() to pass to the function I’m testing.

I’m still new to Zig, so I would not be surprised if there is a more idiomatic way to produce something like std.io.Reader – that answer is very welcome, but I would still like help understanding what specifically is wrong with this code:

const std = @import("std");

/// Wraps an ArrayList and provides a reader() method,
/// so that contents written to the ArrayList can be read back.
pub const RwArrayList = struct {
    list: std.ArrayList(u8),

    pub fn init(allocator: std.mem.Allocator) RwArrayList {
        return .{
            .list = std.ArrayList(u8).init(allocator),
        };
    }

    pub fn deinit(self: RwArrayList) void {
        self.list.deinit();
    }

    pub fn reader(self: *RwArrayList) std.io.FixedBufferStream([]u8).Reader {
        var fbs = std.io.FixedBufferStream([]u8){
            .buffer = self.list.items[0..self.list.items.len],
            .pos = 0,
        };
        return fbs.reader();
    }
};

test "appendSlice into RwArrayList's backing ArrayList, and then read back" {
    var x = RwArrayList.init(std.testing.allocator);
    defer x.deinit();

    try x.list.appendSlice("abcdef");

    // Sanity check.
    try std.testing.expect(std.mem.eql(u8, "abcdef", x.list.items));

    var buf: [256]u8 = undefined;

    var r = x.reader();
    const n = try r.read(&buf);
    try std.testing.expectEqual(n, 6);
    try std.testing.expect(std.mem.eql(u8, "abcdef", buf[0..n]));
}

test "without using RwArrayList" {
    var list = std.ArrayList(u8).init(std.testing.allocator);
    defer list.deinit();

    try list.appendSlice("abcdef");

    // Sanity check.
    try std.testing.expect(std.mem.eql(u8, "abcdef", list.items));

    var buf: [256]u8 = undefined;

    var fbs = std.io.FixedBufferStream([]const u8){
        .buffer = list.items[0..list.items.len],
        .pos = 0,
    };
    var r = fbs.reader();
    const n = try r.read(&buf);
    try std.testing.expectEqual(n, 6);
    try std.testing.expect(std.mem.eql(u8, "abcdef", buf[0..n]));
}

I have spent a couple hours trying to debug this, and I am really confused as to what is wrong.

These two tests should function identically as far as I can see. The bottom test not using the RwArrayList type passes as-is, but the top test crashes:

thread 51586468 panic: @memcpy arguments have non-equal lengths
/Users/hh/src/zig/lib/std/io/fixed_buffer_stream.zig:51:47: 0x100e62a7b in read (test)
            @memcpy(dest[0..size], self.buffer[self.pos..end]);
                                              ^
/Users/hh/src/zig/lib/std/io.zig:94:26: 0x100e60adf in test.appendSlice into RwArrayList's backing ArrayList, and then read back (test)
            return readFn(self.context, buffer);
                         ^
/Users/hh/src/zig/lib/compiler/test_runner.zig:208:25: 0x100f0100b in mainTerminal (test)
        if (test_fn.func()) |_| {
                        ^
/Users/hh/src/zig/lib/compiler/test_runner.zig:57:28: 0x100efb797 in main (test)
        return mainTerminal();
                           ^
/Users/hh/src/zig/lib/std/start.zig:608:22: 0x100efb233 in main (test)
            root.main();
                     ^
???:?:?: 0x188d43153 in ??? (???)
???:?:?: 0xad1affffffffffff in ??? (???)
error: the following test command crashed:
/Users/hh/src/zig/.zig-cache/o/8dd9b57c064c4d8ef5ad716811b57c5c/test --seed=0x7ddccf97

What have I done wrong in the RwArrayList type that the returned FixedBufferStream.Reader causes a crash when we try to read from it?

2

This is a case of returning a pointer to a stack value:

fbs is stored on the stack and invalidated after returning from the function.
fbs.reader() stores a mutable reference of fbs in the reader. This is needed to track state (how many bytes have been read already?).
So in essence you are returning a reference to fbs and using it after the stack value is invalid.

The solution would be either to allocate the fbs (or store it somewhere else, like inside the struct), or to return it and let the caller construct the reader.

1 Like
3

Thanks. I was aware of the possibility of returning a dangling pointer in Zig, but for some reason I expected the failure mode to obviously indicate that problem. Now that I’ve lost some time debugging this issue, I have a better idea of what to look for next time.

4

I stumbled across std.fifo today, which I now assume would have been the more appropriate standard library utility for writing to a value that can then be used as a reader elsewhere.