I'm too dumb for Zig's new IO interface

Media
1
11 Likes
2

The replacement for read is readVec, it has almost the same behaviour, besides supporting vectored reads and end of stream is now an error. I get why it’s a fundamental operation of the interface, it allows the implementation to optimise vectored reads, I don’t get why there isn’t a read wrapper that takes a single buffer.

reader.interface() doesn’t make sense because it’s a wrapper for a field access does make sense because the fields change depending on the OS, though they could have just used the same field name, it also does prevent the foot gun of accessing the interface from a temporary that then tries to access the implementation state which doesn’t exist any more because it was temporary.

I also agree that non-optional parameters should not be in an options type.

It’s still a massive improvement, not just in performance, I think the api is better, but it does lack a few simpler functions even if they would just be wrappers over the more complex functions, especially since most of the api is wrapper functions already.

You’re not too dumb for the new api, you just have no knowledge or experience with it, and there isn’t a clear guide to updating.

15 Likes
3

Me practising…

  • Is down here the way to go?
  • And… when using directly File.write() it is all syscalls without buffering am I right?
  • Is there a way to keep track of the number of written bytes?
// Create
var out_file: std.fs.File = try std.fs.createFileAbsolute("C:/Tmp/testing.txt", .{});
defer out_file.close();

// Use a buffer.
var write_buffer: [1024]u8 = undefined;
var writer = out_file.writer(&write_buffer);

// Get the interface.
const writer_interface: *std.io.Writer = &writer.interface;

// Go.
for (0..42) |i|
{
    try writer_interface.print("writing a line {}", .{ i });
    try writer_interface.writeByte(10); // yeah i know i can use `\n`
}

// Do not forget to flush
try writer_interface.flush();

I like the writer interface with all its functions.

One thing I find philosophically unclear (I think I read it in another thread as well):
This line is not really intuitive.
const writer_interface: *std.Io.Writer = &writer.interface;
because inside File.Writer it is declared as
interface: std.Io.Writer
You really have to KNOW how to do it… It is difficult to deduce from the zig-source.

3 Likes
4

FWIW Re tracking number of bytes written, the 15.1 release notes say:

CountingWriter Deleted

  • If you were discarding the bytes, use std.Io.Writer.Discarding, which has a count.
  • If you were allocating the bytes, use std.Io.Writer.Allocating, since you can check how much was allocated.
  • If you were writing to a fixed buffer, use std.Io.Writer.fixed, and then check the end position.
  • Otherwise, try not to create an entire node in the stream graph solely for counting bytes. It’s very disruptive to optimal buffering.
3 Likes
5

think about it like this, cause this is literally what it is:
you have a struct you are passing to multiple things, and you want any changes they make to the fields of the struct to be propagated back to you for the other uses of that instance of the struct.

reader/writer have state now, in addition to the implementation state, remember that and you will be fine.

i dont save a pointer to the interface in a variable, I like to see which implementation instance it is referencing when I use it, but yes that code is fine.

yes

2 Likes
6

Okidoki thanks.

Are you writing all the time something like this?
writer.interface.write(...)

7

I meant I don’t do const writer_interface = &writer.interface;

Rather, I do writer.interface.function and &writer.interface when I need to pass it to something.

I find it more explicit, as I can see what writer/reader I am using, rather than an interface pointer that could be pointing to the wrong thing by accident. Names do help, but it’s just one less mistake to make.

7 Likes
8

I’m bit torn on the interface field name as technically struct could implement multiple interfaces.

1 Like
9

when it only implements one and the type is named after the interface i dont see any problem using interface as a name, also reader.reader is generally bad naming.

5 Likes
10

As far as I can tell the fieldname is only used to get the pointer to the Io.Reader or Io.Writer instance and within the virtual methods to get to the implementation via @fieldParentPtr, both can be changed (within the impl you would have to replace the virtual table functions with another implementation), if one struct should have multiple.

However in that case you also could consider having them in multiple different structs which are composed instead.

11

that being said, the context to understand this naming is the same as to understand interface so I don’t think It’s that bad.

12

Better than reader.writer :slight_smile:

4 Likes
13

reader.writer.source.sink.recieve()

7 Likes
14

This does not seem right to me…

const line = reader.interface.takeDelimiterInclusive('\n') catch break;

How can we achieve

while (readline()) |line|
{
    // process line
}
15
16

Yes… I tried this try, but that is invalid syntax

while (try reader.takeDelimiterExclusive('\n')) |line| {
}

The example is a bit cumbersome.

EDIT:
Ok this is reasonable at the end.
else |err |return err;

I even didn’t know this was syntactically possible

17

Personally I feelDelimiterError.EndOfStream should not be an error at all.
I will look some more around in the interface… It’s all new.

1 Like
18

only streamDelimiter errors on end of stream even if there was more than 0 read.

the others dont.

19

Very common for text files to not have the \n on the last line of the file, so you’d need extra logic for that case.

1 Like
20

I wrote myself (need it many times for my tests) this little thing for when max linesize is known.

pub const TextFileReader = struct
{
    allocator: std.mem.Allocator,
    buffer: []u8,
    file: std.fs.File,
    reader: std.fs.File.Reader,

    pub fn init(filename: []const u8, allocator: std.mem.Allocator, linebuffer_size: usize) !TextFileReader
    {
        const file = try std.fs.openFileAbsolute(filename, .{});
        const buf = try allocator.alloc(u8, linebuffer_size);
        return
        .{
            .allocator = allocator,
            .buffer = buf,
            .file = file,
            .reader = file.reader(buf),
        };
    }

    pub fn deinit(self: *TextFileReader) void
    {
        self.allocator.free(self.buffer);
        self.file.close();
    }

    pub fn readline(self: *TextFileReader) !?[]const u8
    {
        const line = self.reader.interface.takeDelimiterExclusive('\n') catch |err|
        {
            return if (err == std.io.Reader.DelimiterError.EndOfStream) null else err;
        };
        return line;
    }
};

using it like this

var tr = try TextFileReader.init(my_filename, my_allocator, 1024);
defer tr.deinit();
while (try tr.readline()) |line|
{
    std.debug.print("{s}\n", .{ line });
}

Please shoot at it if you want.

1 Like