Hey, just wanted to let people know I got my zipcmdline project working on 0.15.1, which, required porting the deflate compressor to 0.15.1. The port was pretty straightforward, basically what you’d expect with the API changes, I don’t remember touching much actual logic. A few generic types around Readers/Writers became concrete types which was nice. I also added some testing to the project before the upgrade to 0.15.1 so I know it works in at least some cases.
Command line tools to extract/create zip files.
11 Likes
Does this mean a zig-0.15 release of anyzig is on the way?
2 Likes
adria
October 6, 2025, 12:25pm
3
I didn’t know you had one. I also have an implementation in zignal , to handle PNG files (loading and saving). I’ve tested it with many images, but I occasionally encounter some errors.
Docs & Source:
In case you missed it, Kendall Condon contributed a new flate compressor that landed last week.
master ← gooncreeper:deflate
opened 12:33AM - 20 Sep 25 UTC
Implements deflate compression from scratch. A history window is kept in the wri… ter's buffer for matching and a chained hash table is used to find matches. Tokens are accumulated until a threshold is reached and then outputted as a block. Flush is used to indicate end of stream.
Additionally, two other deflate writers are provided:
* `Raw` writes only in store blocks (the uncompressed bytes). It utilizes data vectors to efficiently send block headers and data.
* `Huffman` only performs Huffman compression on data and no matching.
The above are also able to take advantage of writer semantics since they do not need to keep a history.
Literal and distance code parameters in `token` have also been reworked. Their parameters are now derived mathematically, however the more expensive ones are still obtained through a lookup table (expect on ReleaseSmall).
Decompression bit reading has been greatly simplified, taking advantage of the ability to peek on the underlying reader. Additionally, a few bugs with limit handling have been fixed.
### Comparison to Zlib
zlib achieves a 1.00% better compression ratio at the default compression level and 0.77% better at the best compression level. It
seems that zlib selects slightly different matches, however the total matched bytes is less. In the future, it would be nice to figure this out and be on par with zlib.
Here is a benchmark of the performance versus zlib using the equivalent parameters (i.e. levels).
With default compression level:
```
Benchmark 1 (20 runs): sh -c ./zpipe<sample
measurement mean ± σ min … max outliers delta
wall_time 252ms ± 1.07ms 250ms … 255ms 1 ( 5%) 0%
peak_rss 5.46MB ± 97.4KB 5.32MB … 5.64MB 0 ( 0%) 0%
cpu_cycles 1.19G ± 4.44M 1.19G … 1.21G 2 (10%) 0%
instructions 1.83G ± 665 1.83G … 1.83G 3 (15%) 0%
cache_references 117M ± 904K 116M … 120M 1 ( 5%) 0%
cache_misses 1.66M ± 931K 942K … 5.00M 1 ( 5%) 0%
branch_misses 13.6M ± 9.84K 13.6M … 13.7M 1 ( 5%) 0%
Benchmark 2 (22 runs): sh -c ./std-deflate<sample
measurement mean ± σ min … max outliers delta
wall_time 228ms ± 841us 226ms … 229ms 0 ( 0%) ⚡- 9.7% ± 0.2%
peak_rss 5.45MB ± 116KB 5.24MB … 5.61MB 0 ( 0%) - 0.2% ± 1.2%
cpu_cycles 1.07G ± 1.33M 1.07G … 1.08G 1 ( 5%) ⚡- 9.8% ± 0.2%
instructions 2.18G ± 825 2.18G … 2.18G 0 ( 0%) 💩+ 18.9% ± 0.0%
cache_references 95.0M ± 435K 94.1M … 96.1M 1 ( 5%) ⚡- 18.7% ± 0.4%
cache_misses 874K ± 326K 499K … 1.94M 1 ( 5%) ⚡- 47.3% ± 25.7%
branch_misses 6.30M ± 18.3K 6.24M … 6.32M 2 ( 9%) ⚡- 53.7% ± 0.1%
```
With best compression level:
```
Benchmark 1 (7 runs): sh -c ./zpipe<sample
measurement mean ± σ min … max outliers delta
wall_time 803ms ± 5.75ms 798ms … 815ms 0 ( 0%) 0%
peak_rss 5.48MB ± 120KB 5.24MB … 5.61MB 0 ( 0%) 0%
cpu_cycles 3.85G ± 30.5M 3.83G … 3.92G 0 ( 0%) 0%
instructions 5.32G ± 1.11K 5.32G … 5.32G 0 ( 0%) 0%
cache_references 414M ± 1.47M 412M … 416M 0 ( 0%) 0%
cache_misses 7.91M ± 1.12M 6.15M … 9.30M 0 ( 0%) 0%
branch_misses 28.6M ± 15.2K 28.6M … 28.7M 0 ( 0%) 0%
Benchmark 2 (7 runs): sh -c ./std-deflate<sample
measurement mean ± σ min … max outliers delta
wall_time 797ms ± 1.19ms 795ms … 798ms 0 ( 0%) - 0.8% ± 0.6%
peak_rss 5.50MB ± 82.3KB 5.35MB … 5.60MB 0 ( 0%) + 0.3% ± 2.2%
cpu_cycles 3.82G ± 2.11M 3.82G … 3.82G 0 ( 0%) - 0.7% ± 0.7%
instructions 8.19G ± 508 8.19G … 8.19G 0 ( 0%) 💩+ 54.1% ± 0.0%
cache_references 345M ± 1.02M 344M … 346M 0 ( 0%) ⚡- 16.8% ± 0.4%
cache_misses 4.63M ± 393K 4.20M … 5.44M 0 ( 0%) ⚡- 41.5% ± 12.4%
branch_misses 6.98M ± 41.8K 6.93M … 7.02M 0 ( 0%) ⚡- 75.6% ± 0.1%
```
Benchmark for decompression vs before:
```
Benchmark 1 (113 runs): sh -c ./std-inflate-old<sample.gz
measurement mean ± σ min … max outliers delta
wall_time 44.1ms ± 474us 43.3ms … 46.0ms 12 (11%) 0%
peak_rss 5.48MB ± 112KB 5.23MB … 5.70MB 0 ( 0%) 0%
cpu_cycles 194M ± 487K 193M … 197M 5 ( 4%) 0%
instructions 459M ± 524 459M … 459M 7 ( 6%) 0%
cache_references 1.90M ± 46.2K 1.80M … 2.18M 7 ( 6%) 0%
cache_misses 38.1K ± 3.95K 33.8K … 65.1K 7 ( 6%) 0%
branch_misses 3.16M ± 3.87K 3.15M … 3.18M 4 ( 4%) 0%
Benchmark 2 (126 runs): sh -c ./std-inflate-new<sample.gz
measurement mean ± σ min … max outliers delta
wall_time 39.9ms ± 662us 38.2ms … 42.3ms 4 ( 3%) ⚡- 9.5% ± 0.3%
peak_rss 5.47MB ± 104KB 5.18MB … 5.65MB 0 ( 0%) - 0.1% ± 0.5%
cpu_cycles 173M ± 241K 173M … 175M 4 ( 3%) ⚡- 10.6% ± 0.0%
instructions 410M ± 321 410M … 410M 2 ( 2%) ⚡- 10.7% ± 0.0%
cache_references 1.84M ± 38.7K 1.71M … 2.09M 3 ( 2%) ⚡- 2.9% ± 0.6%
cache_misses 36.2K ± 1.61K 33.1K … 40.8K 1 ( 1%) ⚡- 4.9% ± 2.0%
branch_misses 2.58M ± 3.36K 2.58M … 2.59M 0 ( 0%) ⚡- 18.3% ± 0.0%
```
<details>
<summary>Test details</summary>
Creating the sample data (unix): `find lib/std -type f -name '*.zig' -print0 | xargs -0 cat > sample`
Creating the compressed sample data (unix): `gzip -9k sample`
Standard library compressor
`zig build-exe -OReleaseFast std-deflate.zig --zig-lib-dir lib`
```zig
const std = @import("std");
pub fn main() !void {
var wbuf: [65536]u8 = undefined;
var dbuf: [std.compress.flate.max_window_len]u8 = undefined;
var r = std.fs.File.stdin().reader(&.{});
var w = std.fs.File.stdout().writer(&wbuf);
var d = std.compress.flate.Compress.init(&w.interface, &dbuf, .gzip, .default) catch
return w.err.?;
_ = r.interface.streamRemaining(&d.writer) catch |e| return switch (e) {
error.ReadFailed => r.err.?,
error.WriteFailed => w.err.?,
};
d.writer.flush() catch return w.err.?;
w.interface.flush() catch return w.err.?;
}
```
Zlib compressor
`zig cc -O3 zpipe.c -lz`
Source from https://www.zlib.net/zpipe.c
Standard library decompressor
`zig build-exe -OReleaseFast std-inflate.zig --zig-lib-dir lib`
```zig
const std = @import("std");
pub fn main() !void {
var rbuf: [4096]u8 = undefined;
var wbuf: [std.compress.flate.max_window_len]u8 = undefined;
var r: std.fs.File.Reader = .init(.stdin(), &rbuf);
var w: std.fs.File.Writer = .init(.stdout(), &wbuf);
var inflate: std.compress.flate.Decompress = .init(&r.interface, .gzip, &.{});
_ = inflate.reader.streamRemaining(&w.interface) catch |e1| return switch (e1) {
error.ReadFailed => switch (inflate.err.?) {
error.ReadFailed => r.err.?,
else => |e2| e2,
},
error.WriteFailed => w.err.?,
};
w.interface.flush() catch return w.err.?;
}
```
</details>
14 Likes
I did miss this, but that is great news. I have had three separate projects that relied upon it that have been in limbo waiting for it in (i.e. read-only instead of read/write). I was trying to avoid rolling my own or bringing in 3rd party dependencies for what I hoped would be a short-term regression in features, so I am very glad that my gambit paid off, and my trust in the Zig team and community was well-placed.
1 Like
