When to use buffer in std.Io.net.Stream.writer

Help
,
1

I just started to learning networking. First, I tried to create client and server. client send a string and server receives it. But there is a problem. I can’t understand when I use buffer in std.Io.net.Stream.writer(stream: Stream, io: Io, buffer: []u8). I thought it was used as a message to be sent. But I couldn’t find any ways to use this buffer; How it is used?

    // I want to send "abc\n" to server.
    var arr = [_]u8{'a', 'b', 'c', '\n'};
    const buffer: []u8 = arr[0..]; 
    // Writer型
    var writer = stream.writer(io, buffer);
1 Like
2

The buffer is used internally by the writer. For example, if you use 256 byte buffer and write a few small things, they won’t be sent until you call flush. It’s just a temporary memory for the writer to store the data in the meantime.

1 Like
3

I recognize that buffer is a storage until sent. so, the strings in buffer can be sent by the below code? (Actually, it’s not working well..)

     var writer = stream.writer(io, buffer);
     defer writer.interface.flush() catch |err| {
         std.debug.print("error: {any}\n", .{err});
     };

server side:

	var conn = try stream.accept(io);
	defer conn.close(io);
	var buffer: [24]u8 = undefined;

	// Reader型
	var reader = conn.reader(io, &buffer);
	std.debug.print("conn.buffer: {s}\n", .{reader.interface.buffer});
	const msg = reader.interface.takeDelimiterExclusive('\n') catch |err| {
	    std.debug.print("err: {any}\n", .{err});
	    continue;
	};
4

No, the initial buffer data will not be sent. This is the typical pattern.

var buffer: [256]u8 = undefined;
var writer = stream.writer(io, &buffer);
try writer.interface.writeAll("msg\n");
try writer.interface.flush();
1 Like
5

To add to the example code that you were provided, I think the problem here is with your conceptual understanding.

Both readers and writers use a “buffer” as temporary storage for the purpose of optimization. The act of telling the OS to read or write some bytes is relatively expensive. To avoid asking the OS to write too frequently, this is what happens:

  • You create some kind of “writer” via stream.writer
    • Note that this is just some data structure that happens to be called “writer”, presumably because it’s for writing bytes somewhere. It’s not yet a std.Io.Writer.
    • When you create the “writer” you hand it a pointer to a buffer, and it stores that pointer internally so it can refer to the buffer later.
    • writer.interface refers to a std.Io.Writer
  • You call writer.interface.write("some_bytes")
    • The std.Io.Writer stores "some_bytes" in the buffer.
    • Nothing else happens yet.
  • You keep calling writer.interface.write(...)
    • The std.Io.Writer keeps storing what you give it in the buffer.
    • If the buffer is full, the std.Io.Writerwill take the contents of the buffer and tell the OS to actually write them. This check and then call to the OS happen as part of write(...). All of the data that was written is cleared from the buffer (note: this will not always be all of the data!).
    • By only writing to the OS when the buffer is full, all of the smaller writes beforehand are very fast.
  • You keep doing this until you’re done.
  • You call writer.interface.flush()
    • There may still be data in the buffer (either because it hasn’t been filled up yet or because there was data not written by the last call to the OS).
    • If you don’t call flush(), this data is lost forever.
    • If you do call flush(), the std.Io.Writer is forced to tell the OS to write the remaining data.

So, for your small learning project where you’re writing small amounts of data, you must call flush() in order to see any output at all.

2 Likes
6

Thanks everyone. now, I recognize
・buffer that is in stream.writer is for storing, not for being sent.
・When I call stream.writer, I can pass buffer that hold some strings, but this string never be sent because it is for storing.

is it right?

7

The content of the buffer that you send to the writer will never be read. At some point, a syscall will receive the buffer to read it, but it will only read data that was written by writer’s code, not content that was in the buffer before it was sent to the writer.

1 Like
8

I can pass buffer that hold some strings, but this string never be sent because it is for storing

I don’t think I fully understand what you’re saying here.

The buffer is an array, not a string. The writer assumes that the buffer is empty when you create the writer, so it doesn’t matter whether you’ve put strings, numbers, or anything else in the buffer before giving it to the writer. The writer will overwrite the contents of the buffer when you call write.

1 Like
9

the writer assumes the buffer does not contain any meaningful data, it only knows about the data it puts in the buffer. So by the time the buffer is sent it has already been overwritten.

You could mess with its internal state so that it knows about the pre-existing data, but that is sketchy, better to just use the api to avoid breaking things.

2 Likes
10

I’m sorry for late reply.
I understand that
・when I get a buffer, I’ll do this.

    var str: [1024]u8 = undefined;

    var reader = file.reader(io, str[0..]);

・I also can get a buffer like this

var str = [_]u8{‘a‘, ‘b‘, ‘c‘, ‘\n‘};

const buffer: [_]u8 = arr[0..];

var reader = stream.reader(io, buffer);

but 'a', 'b', 'c', '\n'are ignored the buffer is used as writable storage for reading data, not as input data itself.