IO across the C ABI boundary

Help
,
1

I’m in the inital stages of rewriting a Database system I wrote in C in Zig with the intention of providing a C API to replace the old system but I’m not sure of the best way to handle Io in this.

Pretty much all IO operations now go through the Io layer and for the Zig module its trivial to just allow passing an Io object explicitly, for the C API this isn’t possible because it doesn’t have that Io layer, what would be the best way to do Io in this case?

If I were to simply target mainstream Zig targets like Linux then a global Io.Threaded instance would be good enough but I also target devices that require a custom Io implementation so thats sadly not an option.

I could just add a build option which then makes it pick between different Io implementations but if someone else needs their custom then they would be required to edit the code, is there any good way to deal with this?

One idea I have had is a Callback function implemented Zig that takes a opaque pointer and modifies it as if it were an Io pointer, that way someone could add their own zig file that implements only 1 function and get the functionality needed.

2

Could you not have a C function that lets you set the global io? (You can pass it as pointer and deref)

Other option is to let the user of the zig module / artifact to addImport to it that contains the Io instance.

I dont think std_options kind of approach works for you here since I assume you are compiling a static library.

1 Like
3

My first thought as well but the fact that the Io struct isn’t extern makes it difficult to pass around without ugly hacky workarounds.

Maybe the best option is to make it importable and then simply check if root has a global io implementation by name i.e. @hasField(root, "default_io")
That way I can default to Io.Threaded and replace it in a minimal Zig project, still kind of ugly though.

4

Considering the Io struct is 2 pointers you could also make extern version of it and convert between.

5

The vtable has pointers to functions without the c calling convention.

You would have to make a zig wrapper to convert arguments to and call the c ABI. That is another vtable, which may or may not be acceptable.

6

You can have pointers to auto layout structs in extern structs. You only have to make extern version of the std.Io. In this case C callconv does not matter since its being used by zig code anyways. C ABI is used as the ABI boundary.

2 Likes
7

Yes, I didn’t realise you were only talking about 1 of problems.

8

I would probably do something like this:

In the C API, have an init and a deinit function.
The function implementations internally can .init()/.deinit() on a global io implementation.

So something like this:

pub export fn db_init() Result {
    // ...
    global_io.init() catch |err| switch (err) {
        // error handling
    };
    // ...
    return .success;
}

pub export fn db_deinit() void {
    // ...
    global_io.deinit();
    // ...
}

The type of the Io implementation would be chosen via a buildsystem config.

So a user would either do this:

const default_io_db = b.dependency("db", .{
    .target = target,
    .optimize = optimize,
});
const db = default_io_db.module("db");

or this:

const custom_io_db = b.dependency("db", .{
    .target = target,
    .optimize = optimize,
    .use_custom_io = true,
});
const db = custom_io_db.module("db");
const my_io = b.createModule(...);
db.addImport("custom_io", my_io);

And inside of the buildsystem description, I would just pass the “use_custom_io” option down to your code with false as a default.
My code would then look something like this:

const config = @import("config");

var global_io: if (config.use_custom_io) @import("custom_io") else std.Io.Threaded = undefined;

And of course I would document that the user can chose the io implementation by giving the "db" module an import called "custom_io" which follows the necessary interface.

2 Likes
9

This was more or less my current plan, minus some details.

I don’t think just doing global_io.init() would work because this isn’t a reliable pattern, Io.Threaded for example takes extra arguments for it.

Having a generic function that just returns io might be better and allows for more complex setups:

var global_io: std.Io = undefined;
const init_io: fn () std.Io = if (config.use_custom_io) @import("custom_io").init_io else init_default_io;
const deinit_io: fn () std.Io = if (config.use_custom_io) @import("custom_io").deinit_io else deinit_default_io;

var default_io_impl:  std.Io.Threaded = undefined;
fn init_default_io() std.Io {
    default_io_impl = .init(std.heap.c_allocator, {});
    return default_io_impl.io();
}

pub export fn db_init() Result {
    global_io = init_io();
    return .success;
}

// pretend I also wrote deinit functions here

(essentialy pseudocode, but I hope you get what I mean)