It’s also now become a language proposal:
4 Likes
Great! Thank you 
Unfortunately, I need to stay with 0.15.1 for a while, so I need to use that readVec workaround…
I just independently came to the same conclusion, I’m too dumb for this. I really like Zig, but these new interfaces are making me feel like std::iostream in C++, i.e. something I’d rather avoid. I wonder how this turns out.
I’ve found this new API, paraphrasing the Zen Master Mumonkan’s description of a koan, to be like a red hot ball of iron, stuck in my throat, which I can neither spit out nor swallow. It’s corrupted my dreams and obsessed me to the point where I can barely sleep at night. I woke up with a migraine at 4 AM and all I could think about was how a buffer slice in my brain is pointing to undefined memory and causing segmentation faults. While trying to wrap my head around the purpose and logic of writeHeaderSplatLimitFinish I discovered a buffer slicing bug, introduced by Andrew in a frantic commit, and realized it was causing horrific protocol violations in the HTTP chunked encoding body writer when streaming tarballs on Windows 11, an issue that has been haunting the issue tracker with no rational explanation. Yet I love the thing. I started composing an amateur prose poem about it, it starts like:
This world is small on purpose. There is a table. There are symbols you can touch. There is a courier who comes and goes with practiced grace. Beyond the door is an inscrutable machinery that will, in its season, make your symbols appear on another table in another room.
You keep a table for your work. It is wide enough for some number of symbols and no more. What sits on the table is yours: close, immediate, owned. You place symbols there in the order you mean them. Order is meaning.
You write what fits, you ring when you must, you flush where the form demands. Your friend peeks, takes, tosses, discards, and sometimes waits. The system is obedient and courteous.
The right to a small table, where one can arrange symbols in concrete immediacy, is ancient and inalienable.
Beings withdraw into their depths, encrypt themselves with virtual functions and opaque pointers—but causation occurs in the drama of interaction. For anything to happen at all, there must be a concrete buffer, shared and vulnerable, in the very interface of reality.
I remain confused and enthusiastic. I find myself now able to implement the vtable protocol correctly and without complexity—and even understanding why it is the way it is, how it relates to scheduling and concurrency, how it improves on the state of the art. I am working on some pedagogical materials. In fact I think this API can be, as it has been for me, a provocation and invitation to really grow as a programmer.
6 Likes
Is that function really existing hahaha.
1 Like
Spot the bug! zig/lib/std/Io/Writer.zig at b7ab62540963d80f68d0e9ee7ce18520fb173487 · ziglang/zig · GitHub
1 Like
uses header.len instead of buf.len to get copy_len
Doesn’t truncate buf to copy_len when adding it to vecs
1 Like
Not the bug per se, but:
if (vecs.len - i == 0) break :v;
Is a weird way to spell
if (vecs.len == i) break :v;
I can’t actually tell if truncating buf is needed here? It looks like this gets measured from each call site but there are several. At minimum, if buf.len must always be less than copy_len at that point, that fact should be asserted.
1 Like
I suspect it’s spelled that way because you get an additional assertion “for free” in safe modes (because of usize underflow). Seen this idiom elsewhere in the Io codebase.
3 Likes
same here
1 Like
I think the right way to express how I feel about that, is to say that it’s less boring than I like.
Makes me sit up and pay attention to a line of code which doesn’t need it.
That could be typical mind fallacy on my part, however. There’s code like this line which is doing a similar sort of thing, and also doing it in the most obvious way possible. In a sense, vecs.len - i == 0 is just a special case of that. If you squint. Perhaps after writing enough buffer-copying code, it seems like the most obvious thing to do.
I’ll keep doing it the boring way though.
2 Likes
Yuck. This is the kind of terse, impenetrable code that everybody would point to as evidence of being a terrible programming language if this were C.
I was all prepared to excoriate people for not using a separate assert(vecs.len >= i), but then a little alarm went off in my brain.
Sadly, I suspect this is necessitated by the fact that assert() is not guaranteed to get removed under different release levels and has semantics that you might not expect by virtue of being a function…
See discussion here: Key semantics of std.debug.assert
That’s doesn’t mean I still don’t consider it yucky …
I think this is pointing back to the need for an @assert() built-in for hot-paths that is guaranteed to disappear in -DReleaseFast or -DReleaseSmall …
does not apply, the condition has no side effects.
Also, the actual assert is removed just the same as safety checks, because it is a safety check for reaching unreachable
it is not at all necessary and should be written in a more clear manner.
3 Likes
Interestingly, it’s also evidence that Zig is a less terrible language than C. If this were C it would be dangerous since - in C exhibits twos-complement wrapping, with the analog in Zig being -%
Zig’s - is overflow checked, and readers of Zig code must keep that context in mind.
Let’s take a look at a bit more of the code at hand:
remaining -= copy_len;
if (remaining == 0) break :v;
if (vecs.len - i == 0) break :v;
C programmers unfamiliar with Zig might look with suspicion at remaining -= copy_len;
If this wraps around in C, the following remaining == 0 check would fail to break out, leading to obscure bugs.
But Zig is better! It won’t wrap around, but rather fail with a panic in safe mode.
The way that block is written is kinda nice once you keep overflow in mind. There are two checks there, but no superfluous assert calls.
I agree it might initially be harder to read, but I also think it’s one of these patterns that can become part of the safe-zig-vocabulary.
We all just need to come across it a few times before it’s second nature. A rite of overflow passage, so to speak.
1 Like
It is superfluous, in situ, because if the underflow check is replaced with a normal bounds check, the slicing will fail bounds-checking due to the same condition.
Hard agree. One of the best things about the language, combined with wrapping and saturation also being operators.
Hmm, no I don’t think this should be encouraged. There’s a way that line reminds me of C, in fact: the culture (cult, if you will) of doing terse, ‘clever’ things and being proud of it. Zig as a culture is like, 70% committed to a different path, we should get those numbers up.
There’s a Brian Kernighan quote I like which applies here:
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
We should favor reading code over writing it. That looks more like this:
// [...]
var i: usize = 0;
defer assert(i <= vecs.len);
v: {
// etc
for (data[0 .. data.len - 1]) |buf| if (buf.len != 0) {
// etc
if (remaining == 0) break :v;
if (i == vecs.len) break :v;
};
// etc
}
return w.vtable.drain(w, (&vecs)[0..i], 1);
Boring. Not as clever as possible, rather less so in fact. One floor wax, one desert topping.
Conserves as many brain cells as possible for the rest of the code. Which are needed here, this is not easy code to reckon with.
I don’t want to overstate the case here, it does pattern-match to the task at hand, that’s mitigating in context.
I’m going to stick to boring bounds checks which look like bounds checks, though. That’s what I plan to encourage: code which does obvious things in the most obvious way.
5 Likes
Interesting thread and maybe @andrewrk can explain why he wrote the code this way.
2 Likes
I certainly agree on that sentiment, though also think certain idioms are so good they should be adopted. Not entirely sure if this is one of them, but I doubt the intention here was to be clever.
3 Likes
That way of writing it has objectively better semantics, but subjectively worse readability (I agree that it feels counterintuitive). I’m testing a conjecture that our subjective opinions are wrong (including my own) and that if we get used to writing it the objectively better way, then our intuition will align more closely with reality.
13 Likes
semantics, as in “here we’re evaluating the size of a segment, as opposed to a single value”, or something else?
Apparently the new Writer and Reader haven’t come out as that of the utter level of complexity yet. Not as much as Andrew aimed to.
Andrew. Stop to remedy everything. Just stop.