In need of Parquet Arrow Decompression

Howdy - it’s been a minute :slight_smile:

I am currently looking for a clean way to decompress non-encrypted parquet files in the arrow format within a Zig context. My use case is real time processing (files are not not on disk - they are coming in through an http context). I’d rather not spin up a child process, so I’m curious about what options there are right now?

I’ve looked at the Apache Arrow library and it’s not ideal. The communication protocol is going to be rough between the C-code and Zig and I still need to first compile their C++ library and link through their lib c headers.

Any thoughts? Thanks!

2 Likes

If you are comfortable with rust, there is a rust version here: arrow-rs/parquet at master · apache/arrow-rs · GitHub
Start with arrow-rs/parquet/src/bin/parquet-rewrite.rs at master · apache/arrow-rs · GitHub exporting it as static library callable from C.

1 Like