I’ve been working on a project / library in zig that is mostly centered around computing history for text.
Everything works great zig, I can export the library as a module and use it in another zig project.
I also wrote bindings that takes the module and wraps the functions and exports functions with a c callconv, as well as some other niceties like sensible errors (I have an optional onError function and a getLastError function). I also did the same for wasm, though it does things slightly differently and I also export a alloc and free functions (so I can pass in strings).
I then also manually (with the help of an llm) wrote the some bindings, (including a header file because getEmittedH is broken) to make it nicer to use in other languages.
So now I can use my library from C (or anything that can use the ABI and only needs a header file), Go using CGO and purego, or Go using wazero, JS using Wasm in Bun/Node/Browser.
Let me give some examples, I have this function
pub fn make(allocator: std.mem.Allocator, sequenceNum: u16, new_text: []const u8, old_text: []const u8) Error!CompressedBytes {
...
and on the bindings I have it as
// ffi/lib.zig
// there is also a version of this function that doesn't take the string lens and uses null terminators, but that is for c
export fn historicMakeLen(sequence_num: c_ushort, new_text: [*c]const u8, new_text_len: c_uint, old_text: [*c]const u8, old_text_len: c_uint, out_bytes: *[*]const u8) callconv(.c) c_int {
...
// ffi/wasm.zig
export fn make(sequence_num: u16, new_text: [*]const u8, new_text_len: u32, old_text: [*]const u8, old_text_len: u32) callconv(.c) u64 {
...
And in the end I want a nice function I can call, like this for example in go
func Make(sequenceNum uint16, newText, oldText string) (CompressedBytes, error) {
...
When linking using the static / dynamic libraries I can take new and old strings, use runtime.Pinner to get a static pointer to the first elements, get the lengths, get a *byte for the return arg
then call it with
ret := historicMakeLen(sequenceNum, cNewText, uint32(len(bNew)), cOldText, uint32(len(bOld)), &outPtr)
(there is a bit of a difference, using CGO I also have to do unsafe.Pointer and cast it, but purego loads the dll at runtime, so this works)
Then I have to check the return for the error value (-1 in this case), and if it was an error to call my nicified getLastError and return that. Otherwise I take the length which was the returned value of the function and use it to make a slice with the pointer that was populated from the return args, then I copy it so the bytes are managed by go and free the provided value.
Wasm is much the same, but I have to use the provided alloc function to allocate some space for each string, copy it in using the exported memory,
and because that was so much of a pain, I did not want to also make you use alloc to allocate a space for a pointer that the function fills in as an arg, and you then have to read, and make sure to free it.
So I took advantage that wasm32 is 32 bit, and concatenated the ptr and len into one packed item and returned that as a u64. Now the error case is 0, and if you get that you also do getLastError and pass the error up, but if its not an error you now only have to do a bit mask and shift to get the length and pointer, that you can then use to read from the exported memory, copy it so that Go or JS will manage it, free the original and return that.
The point is that this is a pain
So writing all these nice bindings for different languages and runtimes gets very tedious and the combinations explode very quickly.
And this is definitely something that can be generated by introspecting the exports (and imports) of the FFI layer. Though manually editing some things might be required.
I was thinking of different ways to do it, and settled on having an IR (zon or json).
That is then used together with either scripts that take it and generate the file output file,
or to have more zon or json files that have templates for things like, this is a how you do a function, this is how you load a value, etc. that then gets inserted as needed.
The IR can be generated, but this is giving me some issues, since I could not find a way to get the exports and imports at build time.
So you could generate the IR from the FFI files (or maybe even from the base lib) by using the std.zig.Ast,
but this might have issues, since if there is an export or an extern that is not in the root file you will have to also go over the imports.
For example I have this extern, it’s only on wasm and is not in the root file
const Timer = if (!isWasm)
std.time.Timer
else
struct {
start_time: u64,
extern fn readTimeNs() callconv(.c) u64;
pub fn start() !Timer {
return .{
.start_time = readTimeNs(),
};
}
pub fn read(self: *Timer) u64 {
const time: u64 = readTimeNs();
std.debug.assert(time > self.start_time);
return time - self.start_time;
}
};
This will not be exposed via the ast approach, so another option is building the library and then inspecting what it exports,
but this seems flaky to me because you need external tools (or write parsers for different formats in zig) to be able to extract what is needed.
But then you lose out on things like argument names and doc comments.
Any thoughts or suggestions would be appreciated, also did I miss anything obvious?
