Packaging Zig as Python packages

Rust has a fantastic tool in maturin GitHub - PyO3/maturin: Build and publish crates with pyo3, cffi and uniffi bindings as well as rust binaries as python packages in order to write libraries in Rust but provide them as Python packages.

This is being used with great success by the Ruff / uv projects GitHub - astral-sh/ruff: An extremely fast Python linter and code formatter, written in Rust.

It’s fantastic since this allows your library to reach out users that otherwise wouldn’t use them. Is there anything similar for zig?

Zig already provides cross-compilation to multiple targets, so for executables this seems like a simpler problem to solve as you just need to package different executables for each target you want to support (?) but for libraries the problem seeems a bit more difficult.

Does anyone know of already existing tools? Or how easy it would be to implement something similar?

Note: interesting talk https://www.youtube.com/watch?v=O0MmmZxdct4

1 Like

I think you’re looking for Ziggy Pydust: Ziggy Pydust

Edit: seems like it won’t work with zig 0.13 just yet, due to changes in zig

1 Like

Zig is available on PyPi.
See also: The Python Package Index Should Get Rid Of Its Training Wheels

2 Likes

More information:

I think you are confused here. The author mentions creating Python packages written in Zig. But the title is very confusing in terms of his intent.

1 Like

His comment was incredibly useful as it led me down the path of discovering cargo-zigbuild + setuptools-rust projects, which do similar things to what I want.

Also the project he linked ends up in the zig-pypi GitHub - ziglang/zig-pypi: The Zig programming language, packaged for PyPI repository which is another repository that packages a zig project into python, in this case, the whole programming language.

3 Likes

I have been using my setuptools-zigfor various simple extensions over the last 5 years. It is an extension for setuptools, which I recently extended to work with the ziglangpackage. So you specify the build dependency for your package to be on setuptools-zig (and ziglang) and the .c and .zig files in your sources will be compiled with zig.Since you can specify a specific version (ziglang<=15.0 ), this prevents language changes to break things unintentionally.

It now also handles more complex situations, like the ruamel.yaml.clib ( a handful of .c files, one of which generated from Cython), on Linux ( Ubuntu and Alpine tested) and macOS. Windows still has some problems.

2 Likes

Since I keep using this thread as a way to share useful links in relation to the question of “Zig and python, can they work together?”.

Here’s another great talk by Loris Cro, which is exactly what I wanted to find when I originally started this thread https://youtu.be/HPmefnqirHk?si=3PuqtJJ5zlAJ60x2

Loris actually announced that talk at SYCL while interviewing me (https://youtu.be/Y9F5zg3tErI?list=PL5AY2Vv6EsfSgmueLbzGVJpWIw8cIFl8J&t=2816), and I totally missed that (probably the nerves by being on stage), so I did not watch that video until much later.

That is a good talk, but unfortunately Loris didn’t know that if you nowadays do `python -m pip install some-non-wheeled-source-package` a wheel is made, cached and installed.

I have been working on compiling the C extension for ruamel.yaml with setuptools-zigthe last couple of days. Uploading one source is however only part of the problem. Getting things to work initially and testing is still an extensive matrix (six Python versions (py39-py314) 5 OS-es (Linux, Linux-musl, macOS, Win32, Win64) * 1-2 architectures ( Intel/Amd, arm, maybe more). And then it might work different on older versions of an OS, processor, etc.

I cannot test all of these combinations in the first place, so I try to get through a cross-section. And of course I ran into (link) problems in the last platform tested (Windows), because of a bug in the toolchain, and now need to recheck the workaround on the others as well :frowning:

1 Like

You can also check how I am doing this in zignal

Basically, use the raw cpython API from Zig, see which patterns repeat, and factor them out in: zignal/bindings/python/src/py_utils.zig at master · bfactory-ai/zignal · GitHub

1 Like

This thread is all coming together now :smiley:

I’ll have to watch your talk too, seems very interesting and on topic. I guess keeping your code as much Zig as possible can help with some of the grandfathered issues coming from C but you will of course still have to test all different permutations.

Also thanks @adria I saw your thread of zignal yesterday and read through some of the bindings code. This is exactly what I was looking for.

1 Like