Most examples that I have seen on the forums here have used buffer sizes of 1024, or 4096. I believe on most systems C’s BUFSIZ is 8192.
What sizes should be recommended for various purposes?
It all comes down to how much memory you’re willing to spend versus the overhead of the Reader/Writer implementation.
If the implementation has very high overhead, it makes sense to spend more memory on a bigger buffer to reduce the total calls to the implementation.
If the implementation has low overhead, and/or your program is restricted in terms of memory then it makes sense to use a smaller buffer.
I’m aware that this isn’t a very satisfying answer, but as with most software advice: it depends. When the performance of the IO is very important to the application, I’d advise measuring and benchmarking your program with different buffer sizes to make a more informed decision.
EDIT:
I’d add that the performance gain going from a buffer of 1024 bytes to 4096 far less significant than the gain of going from no buffer to any buffering at all. Many times there’s not much to be gained from spending much time fine-tuning the buffer size.
3 Likes
The answer is really the hated “it depends”, though I would say that the sizes listed are good for a general-purpose buffer. If you need one that is smaller or larger, typically you will know this already by the nature of the data you are using.
For example, if you are using a reader/writer for compression, it makes sense to use a buffer that matches the maximum window size of the algorithm, but you would already know this simply by the context. On the other hand, if your data isn’t even 8192 bytes in size, it makes no sense to use a buffer that large.
Each and every situation is going to have its own “sweet spot”, but figuring out where that might be is not often worth the trouble unless it is in a hot-path that is critical to squeeze every CPU cycle out of it. For that reason, I tend to just stick to the old 1024’s, 4096’s, and 8192’s unless I have (known) good reason not to.
There seems to be a semantic significance to the buffer size. For example, you cannot takeByte() with a zero-sized buffer.
- How should libraries communicate their preferred minimum buffer size?
- How can libraries determine their minimum allowable buffer size?
- Should users use the minimum buffer size or arbitrarily pick some multiple of 1024?
Docs and asserts
That’s domain specific.
It depends 
ideally the docs should communicate why the minimum buffer size is what it is, that should make that decision a lot easier, it should also specify the maximum/minimum it can read at once if possible.
You also want to consider what you know about data in your specific use case, and how you would like to use the buffer.
If for example the data you’re working on at a time is larger than the max read size, you can use a buffer large enough for your data, fill it beforehand, then you can operate out of just the buffer for your actual data processing.
I find this talk by the great Titus Winters extremely relevant here: https://youtu.be/J6SNO5o9ADg
You should 100% go watch every talk Titus ever gave but if I had to do a TLDW:
Good buffer size depends not only on things everybody else mentioned in this thread but also on the current year! Good buffer size 10 years ago is bad buffer size in 10 years. That is why whatever buffer size you choose, WRITE A COMMENT! Somebody might be looking at the code in 5 years and wondering if they can increase the buffer size but worried that the magical 4096 number might have some deeper reason other than that the developer just liked it. I usually write at least “buffer size picked arbitrarily” next to my buffers so that nobody is scared to change that in the future.
Another great idea from the talk is ‘configuration based on intent’. Which means you don’t say you want a buffer of size 2048 but you say you want to have buffer size optimized for latency.
Specific implementation might be a set of constants in the std: buffer_size_for_cpu_bound = 4096, buffer_size_for_memory_limited = 64, buffer_size_for_latency = 512. The huge benefit here is that every 5 years somebody increases these buffers and all applications get updated to the up-to-date good values.
(BTW i completely made up these specific values, please don’t copy them from me)
8 Likes
I can’t say anything specific for Zig, but when dealing with file I/O for several MB sized files on Windows using C or Python (when hard disks where used) it was a big difference between eg 4 KV buffer size or 1 MB buffer size. One should at least benchmark some different values with the application, on a system that roughly matches the intended target system (VM or actual hardware, OS, hard disk or SSD), for example testing 4 KB, 64 KB, 1 MB.
1 Like
Having buffers sizes in std won’t work, because buffer size is mostly dependent on your data, its origin, then what you’re doing with it.
Relying on a size provided by std is just as arbitrary as picking a random number.
The TLDW snippet and ‘configuration based on intent’ is correct.
2 Likes
Though it would be a semantic improvement to have one designated “I don’t care” value in the stdlib that’ll work well enough for many use cases. If you read code that doesn’t use that value, then you can assume there is some reason for it.
1 Like
Better yet, create a package that provides these sort of don’t care values, that way people can filter out all dependencies that depend on that package, when they are on some severely constrained system, that really does have to care about everything.
And also I can use it as a litmus test, to reconsider whether I actually want to depend on that dependency if that don’t care package isn’t just an optional dependency.
I think for many packages it might be best if they allow the user to configure buffer sizes, so that these arbitrarily picked numbers mostly reside within applications instead of being hardcoded into libraries.
1 Like
That people don’t care is only a part of a the problem. Even when the developer cares and does a good job, in 5 years the buffer size is going to be wrong.
Letting the library user set the buffer size does not solve the issue since the user is going to just set it to some hardcoded constant and in 5 years it will be wrong again.
1 Like
Lots of good comments here already. Ultimately what I settled on for z2d was actually to remove anything that was generating any of the new I/O readers/writers and just use file I/O directly, to avoid this question altogether.
Why? Turns out most of the places I needed a buffer were already buffered in one way or another.
In z2d, there’s only really two places where I/O is done: when fonts are loaded from disk (and not directly passed in via external buffer) and during PNG export. When loading from disk, fonts are already read in whole into memory, and the PNG export IDAT encoder (which currently relies on the old stdlib zlib compression functionality, so hopefully that gets added back in some form soon 
) is backed by a 16KB buffer that stores the compressed data until it’s full, after which the chunk is written out. There are, of course, other chunks that are much smaller that get written out unbuffered under this model, but these are generally just metadata packets that are only written once per file.
So turns out I didn’t really need to add extra buffers on top of those and it was only complicating things, or causing me to over-think what buffer size to choose and totally overshooting it by starting with 16KB ones. 
One thing I did do was make a helper for writeInt because it doesn’t exist in std.fs.File.
So all of that to say that you might want to think about where you’re doing buffering already too and act accordingly, that way you’re not doubling up unnecessarily (which probably has its own issues!).
1 Like
I guess the library could ship benchmarks that are run during compilation to automatically pick a buffer size, that would be kind of fancy and interesting.
Maybe only if you don’t provide an explicit size via options?
But with cross compilation, it would have to make sure that the benchmarks actually run for the target system, not the host. (and that they aren’t emulated with qemu if that would cause different results from native execution, but also different hardware running the application for the same target could have different results…)
To put the choosing buffer size anxiety into perspective, here’s a personal anecdote:
I’ve used the C fread API since the 90’s and I don’t ever recall needing to change its buffer size, and well, I guess I’m not even sure of it can be changed. I’ve just assumed it’s something like 1 or 2 kB and it’s always worked quite alright for a lot of things.
OTOH, there’s probably never been a situation when no buffering at all wouldn’t have been some kind of a performance problem.
2 Likes
Would you also categorically ignore Zig packages that just use 2048 and 4096 without any explanatory comments?
Lots of software will and already go this path; my point is that
// (please don't bikeshed the name/namespace)
let buffer: [std.Io.DefaultBufferSize]u8 = undefined;
is better than
let buffer: [4096]u8 = undefined; // arbitrary value
is better than we will actually be seeing most often:
let buffer: [4096]u8 = undefined;
2 Likes
Another idea, especially for libraries, would be to allow this to be downstream-tunable. Pick a sensible default that you think makes sense, and then allow it to be changed using build options (or comptime parameters, depending on how you want to expose it). You can combine this with with documentation and assertions (like @vulpesx mentioned) to make sure it can’t be tuned in a way that would cause things to break.
This absolves you from having to think about every edge case; you can just direct people to the correct way to tune things, possibly adjusting the default if the standard cases dictate it.
EDIT: I also see that @Sze pretty much mentioned this already I think just seeing @smlavine’s examples just make me think “yeah, this seems like a job for Module.addOptions”
1 Like
I wouldn’t exclude those packages always, but I think it depends on the code you are writing and it would be good to have tooling that allows you to find and reject such packages.
With the case of your example, maybe somebody could write a Zig linter which you could run as a build step, that then rejects dependencies with fixed buffers that don’t have an explanation.
I don’t think there is necessarily one right way, but I think having it as a separate package would make it easier to spot such uses.
There also could be cases where some buffer will never get a better size (for example there could be algorithmic constraints), in such cases it would be better to add a comment that explains this, but if DefaultBufferSize is the low friction way to get a buffer, then maybe nobody will ever think about it long enough to discover that.
That is where my hesitancy comes from with adding things that may make things easy, but could lead to code that may be less simple than it could be. If some code can find some number of asserts that result in some ideal buffer size I find that strictly better than something picked arbitrarly, that is completely independent from what the program actually does with that buffer and may be picked by a number of people who write very different programs on very different hardware.
Here are some rules of thumb:
- In a stream pipeline, buffers belong to edges not nodes. The node that connects the reader end of the pipeline to the writer end of the pipeline should be empty; the rest populated.
- In the middle of a stream pipeline, minimal buffer size that satisfies the constraints of each stream implementation is optimal. This requires reading documentation for the two stream implementations you are connecting and choosing the higher requirement.
- In the ends of a stream pipeline (file, network socket) ideal buffer size is sweet spot between evicting CPU cache lines and overcoming syscall overhead. This is typically 1K, 2K, or 4K, but can vary across operating systems, architectures, or application-specific details. I expect the difference between these three values to be fairly inconsequential.
9 Likes
I have the gut feeling that with a good enough code base and long enough pipelines, you will end up with many buffer in the middle too.
1 Like
Please don’t recommend leftpad like dependencies…
3 Likes