std.Io.File.readPositional's buffer requirement bugged?

Help
1

Heya’s Everyone :relaxed: I’m new here and just starting my Zig journey, I am working on some small post production video tools and enjoying zig!

I’m currently on zig master, enjoying all the new shiny things, I am playing around with remuxing a video, using new std.Io.Reader and Io.File interfaces and what not… which is going pretty well… I am also trying out random / offset reads via std.Io.File.readPositional but cannot get it to work, it errors with:

error: expected type '[][]u8', found '[]u8'

Here’s my fn:

/// Extract a single frame's data from the file
fn extractFrame(
    io: Io,
    file: Io.File,
    frame_info: FrameInfo,
    allocator: Allocator,
) ![]u8 {
    const frame_data = try allocator.alloc(u8, frame_info.size);
    errdefer allocator.free(frame_data);

    // Read from file at the exact offset
    
    // No worky:
    // errors with 'error: expected type '[][]u8', found '[]u8'`' even though it requires a []u8 buffer...
    _ = try file.readPositional(io, frame_data, frame_info.offset);


    // Worky: feels wrong ?
    // direct use of vtable with single element array of slices to match 
    // the vtable's [][]u8 parameter for vectored I/O 

    var buffers = [_][]u8{frame_data};
    _ = try io.vtable.fileReadPositional(io.userdata, file, &buffers, frame_info.offset);

    return frame_data;
}

Am I missing something? Or is std.Io.File.readPositional’s buffer requirement bugged a in progress dev/master issue?

std ref: Zig Documentation
pub fn readPositional(file: File, io: Io, buffer: []u8, offset: u64) ReadPositionalError!usize

2

Are you passing the buffer and offset arguments in the wrong order? It’s a very weird error message if that’s the case.

3

If you look at the source code for the readPositional() implementation it’s pretty easy to see that it’s incorrect and doesn’t match the vtable signature. As you note the Io stuff is very new and most related APIs are in flux and currently lack tests, but it probably wouldn’t hurt to open an issue if you can’t find an existing one.

// std.Io.File.readPositional
pub fn readPositional(file: File, io: Io, buffer: []u8, offset: u64) ReadPositionalError!usize {
    return io.vtable.fileReadPositional(io.userdata, file, buffer, offset);
}

// std.Io.VTable.fileReadPositional
fileReadPositional: *const fn (?*anyopaque, File, data: [][]u8, offset: u64) File.ReadPositionalError!usize
3 Likes
4

Apologies, my initial write up of the post had the function argument order wrong, but the issue/error still remains:

/Users/fq/.zvm/master/lib/std/Io/File.zig:219:60: error: expected type '[][]u8', found '[]u8'
    return io.vtable.fileReadPositional(io.userdata, file, buffer, offset);
                                                           ^~~~~~
/Users/fq/.zvm/master/lib/std/Io/File.zig:219:60: note: pointer type child 'u8' cannot cast into pointer type child '[]u8'

I think it could be related to this ongoing issue / plan : Move Filesystem APIs to std.Io · Issue #25738 · ziglang/zig · GitHub

5

I spotted this earlier and I was thinking whether it was me just setting up the frame_data buffer wrong… but upon further reflection I think you are right that its an source error, and still being worked on by the Zig team!

Bring on release 0.16 eh! :slight_smile:

6

As an aside, I got the same outcome to work via the specific std.Io.File.Reader file_reader.seekTo : Zig Documentation

A bit of std.Io.File.Reader vs std.Io.Reader confusion maybe mixed in there for good luck ha!

fn extractFrame(
    io: Io,
    file: Io.File,
    frame_info: FrameInfo,
    allocator: Allocator,
) ![]u8 {
    // std.Io.File.Reader.seekTo also can facilitate a positional read via: 

    var buffer: [4096]u8 = undefined;
    var file_reader = file.reader(io, &buffer);

    try file_reader.seekTo(frame_info.offset);

    // Now get the reader interface for actual reading part...
    var reader = &file_reader.interface;

    const frame_data = try reader.readAlloc(allocator, frame_info.size);

    return frame_data;
}

Weee!

7

I also saw some inconsistencies when using the new Reader interface with a File, where seekBy and seekTo calls didn’t behave properly. I ended up rewriting my code to just use read calls against the File itself in order to get the work completed. The first call to seekBy or seekTo would get it to the first location in the file I wanted, but subsequent calls didn’t update the position.

I believe that std.Io: Writer and Reader bug fixes related to sendFile, delimiters, Limited, and seeking by andrewrk · Pull Request #25512 · ziglang/zig · GitHub fixed some of this, but overall it seems to still be a work in progress.

2 Likes
8

I just came across this, and it did surprise me too, but it is done on purpose to be able to use the iovect structure, as for example visible in Uring.zig fileReadStreaming, the kernel accepts multiple arrays as input, so that function accepts [ ]const[]u8.
Later (post 0.16) that was then (for the better, and as pointed out by sc68cal) moved to readVec and readVecPosition function removing the confusion.