PR 24858 changed sendFileAll, and now it always requires a buffer, can somebody please help me understand why is this ok?

Hi, everyone! I’ve recently posted a question, Should standalone test simple/cat/main.zig actually work or just be buildable?, which led me to the creation of PR 25188. But digging a bit deeper, I noticed that the requirement for buffer in sendFileAll is a recent addition.

I may be missing something, but I am not sure why it is ok to hard limit usage of sendFileAll only for buffered writers? If my understanding is correct, in the case of the sendfile syscall, it is not necessary to create a buffer in user space, which means this is an artificial limitation put on sendFileAll users only because of the fallback case.

The code in question

+/// The caller is responsible for flushing. Although the buffer may be bypassed
+/// as an optimization, this is not a guarantee.
+///
+/// Asserts nonzero buffer capacity.
pub fn sendFileAll(w: *Writer, file_reader: *File.Reader, limit: Limit) FileAllError!usize {
+    // The fallback sendFileReadingAll() path asserts non-zero buffer capacity.
+    // Explicitly assert it here as well to ensure the assert is hit even if
+    // the fallback path is not taken.
+    assert(w.buffer.len > 0);
1 Like

Not saying that this is the case but this is in the interface and not any specific target implementation, so I’d imagine there’s a software fallback for targets that don’t have sendfile. This would likely require a buffer.

Following the comments referenced seems to confirm this: https://ziglang.org/documentation/0.15.1/std/#std.Io.Writer.sendFileReadingAll

Equivalent to sendFileAll but uses direct pread and read calls on file rather than sendFile. This is generally used as a fallback when the underlying implementation returns error.Unimplemented, which is why that error code does not appear in this function’s error set.

PS: The assertion is done through writeableSliceGreedy in sendFileReading.

PPS: I noticed you mention the fallback case, so you probably know all of the above :wink: but this is likely being done for consistency in portability. The interface can’t guarantee that sendfile exists on every platform, so if you’re using the interface, you should be consistent, otherwise, use local implementations of sendfile directly out of std.os.linux and call it a day. One thing you will likely see right away (even if yes, functionality may exist in an equivalent syscall) is that there’s no sendfile in Windows. :slight_smile:

1 Like

Yes, I believe this is the case as well. I understand the need for the buffer in fallback implementation, but my question is: why is it ok to force assert sendFileAll for the buffer, even on the systems that do have sendfile support? This means that even on supported systems we can’t use sendFileAll without buffered writer

Yeah, sorry, I just updated with an edit. :slight_smile:

I definitely think it has to do with implementation guarantees. Again, haven’t dug in too much, but if you search the stdlib for sendfile most implementations exist in the std.c namespace - only Linux has one in std.os. I’m not too sure if that’s telling WRT implemenations, but it could mean that most other implementations would require libc?

1 Like

Yeah, this is probably the reason. Thanks! I think this could be added as clarification to documentation.

1 Like