Following on from mentioning in the what’s everybody working on thread, I had been meaning to share a basic working example of how to build and package a Python extension module in Zig - as I couldn’t find a good example kicking around when I got started!
I figured I’d showcase some basic examples of calling a function from Zig in Python, building the module, and getting the naming / exports right. I included a few helpers / patterns that I find helpful, but the idea here is that you can build whatever error handling / module creation stuff you need on top of this.
As noted in the repo - I’d love to have a full wheel-building subsystem working from the Zig build system, but I’m not great at it and I don’t think you can create ZIP archives (which are the python wheel format of choice) from std zig, so I show how to package a wheel via a basic Python script and a build script.
I’m sure others have done this better than I (and I am aware of larger projects like ziggy-pydust which try and make this easier) - but I was pleasantly surprised how straightforward this was with standard CPython!
Hopefully this helps someone and I’d welcome any feedback on this approach as well!
I am sorry to hear you manage to miss setuptools-zig which has been available from PyPI since 2020. It includes an example to call a zig functions from Python, builds .whl and more.
I didn’t miss this actually - but good to know it’s still being worked on! This is a great project for a more full-featured approach and with a Python-first ecosystem.
The example here is more about trying to have zig do the build work and using translate-c (rather than cImport, as it’s now deprecated) without additional dependencies. The pure zig approach ends up being quite a bit shorter which is nice!
But your project is probably better for folk who want a bit of zig intermingled with Python in a single library
maybe my take https://codeberg.org/dasimmet/python-zig-zon is interesting for you then.
The idea is to build the entire python source distribution (a .tar.gz file) using zig build, include everything needed to turn it into a wheel (by building a native extension using a different top level build step and zig build as well) when installed from pip.
Interesting. I just took a quick look so need to play around more, but am I right that the approach is:
write your .zig file linking an expected Python header which has CPython bindings
Your build step will then handle translate-c - ing the Python header when built by user. This is done at build time after distribution (I think?)
The sdist file then contains: your actual zig code, a copy of the sdist dependency for the build step, and a PKGINFO describing how to build it
At pip install time, you then depend additionally on ziglang (or setuptools-zig) to download and compile on the client?
This is cool though and like the above project definitely more full featured than my example. Nice to see all this stuff in the area!
I think for both solutions it feels odd to me to have to get users to download and compile their own version of zig in the Python ecosystem, rather than ship binary wheels that are the right format. However, I guess you gotta cross the line from zig to python at some point regardless!!
Pretty much yes. the sdist archive ships a fixed setup.py that contains the hook for setuptools to run zig build --fork=<path to the zig-python-sdist dependency> when the wheel is built. For all “dynamic” values i generate a .json file that setup.py can load, and the PKG-INFO file is generated from build.zig as well.
and I’d welcome any feedback on this approach as well!