The example in the post is using the “direct mode” of std.compress.zstd.Decompress (i.e. giving it a zero-length buffer) which actually has much more unsafety than detailed in the post:
opened 05:45AM - 27 Aug 25 UTC
bug
standard library
### Zig Version
0.16.0-dev.38+37f4bee92
### Steps to Reproduce and Observed Be… havior
`flate.Decompress` uses `direct_vtable` (which sets `.discard = discardDirect`) only when `r.buffer` has a length of 0:
https://github.com/ziglang/zig/blob/9399fcddce0bcd8e987b053f3946aa1b0ff2ef0a/lib/std/compress/flate/Decompress.zig#L84
This means that nothing about the `discardDirect` implementation makes sense, since it all assumes that `r.buffer` has a length > 0:
https://github.com/ziglang/zig/blob/9399fcddce0bcd8e987b053f3946aa1b0ff2ef0a/lib/std/compress/flate/Decompress.zig#L117-L139
For example: no matter what, the first `if` will be true and call into `rebase` which is guaranteed to hit integer overflow when evaluating this assertion (`0 - flate.history_len`):
https://github.com/ziglang/zig/blob/9399fcddce0bcd8e987b053f3946aa1b0ff2ef0a/lib/std/compress/flate/Decompress.zig#L106
This also means that the `rebase` implementation doesn't make sense when `buffer.len == 0`, so `direct_vtable` using `rebaseFallible` is also guaranteed illegal behavior if it ever gets called. And the same problem exists for the `readVec` implementation.
---
Any `Reader` function that calls into `vtable.discard` will reproduce this, but here's one example:
```
head -c 4096 /dev/zero | gzip > test.gz
```
```zig
const std = @import("std");
test "flate discard direct" {
const compressed = @embedFile("test.gz");
var in: std.Io.Reader = .fixed(compressed);
var decompress: std.compress.flate.Decompress = .init(&in, .gzip, &.{});
_ = try decompress.reader.discardRemaining();
}
```
```
$ zig test flate-discard.zig
thread 374932 panic: integer overflow
/home/ryan/Programming/zig/zig/lib/std/compress/flate/Decompress.zig:106:37: 0x1199ca9 in rebase (std.zig)
assert(capacity <= r.buffer.len - flate.history_len);
^
/home/ryan/Programming/zig/zig/lib/std/compress/flate/Decompress.zig:118:57: 0x119926b in discardDirect (std.zig)
if (r.end + flate.history_len > r.buffer.len) rebase(r, flate.history_len);
^
/home/ryan/Programming/zig/zig/lib/std/Io/Reader.zig:273:35: 0x102b6d2 in discardRemaining (std.zig)
offset += r.vtable.discard(r, .unlimited) catch |err| switch (err) {
^
/home/ryan/Programming/zig/tmp/flate-discard.zig:10:47: 0x102b0ec in test.flate streamExact direct (flate-discard.zig)
_ = try decompress.reader.discardRemaining();
^
```
### Expected Behavior
`flate.Decompress.direct_vtable` to have functions that take `buffer.len == 0` into account
(the issue is about flate but it applies to zstd as well)
Any usage of Decompress in “direct mode” that calls into vtable.rebase/discard/readVec is guaranteed to trigger illegal behavior.
See this comment for context . As I commented here , I think there is room for improvement on the documentation surrounding this, but there’s also some inherent problems with this use case that will hopefully get some attention soon.
2 Likes