Figuring out how to use translateC

I’m trying to wrap my head around integrating C dependencies in a project and compile them with zig. Thanks to this topic I have a working prototype that uses @cImport. It looks like this:

// build.zig
const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    const lib_mod = b.createModule(.{
        .root_source_file = b.path("src/root.zig"),
        .target = target,
        .optimize = optimize,
    });

    const exe_mod = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
    });

    exe_mod.addImport("apollonius_lib", lib_mod);

    // load miniaudio dependency
    const miniaudio_dep = b.dependency("miniaudio", .{});

    // compile as a static library
    const miniaudio = b.addLibrary(.{
        .linkage = .static,
        .name = "miniaudio",
        .root_module = b.createModule(.{
            .root_source_file = null,
            .optimize = .ReleaseFast,
            .target = target,
        }),
    });
    miniaudio.root_module.addCSourceFile(.{ .file = miniaudio_dep.path("miniaudio.c") });
    miniaudio.linkSystemLibrary("pthread");

    // link the static library into the executable
    exe_mod.linkLibrary(miniaudio);
    exe_mod.addIncludePath(miniaudio_dep.path("."));

    const exe = b.addExecutable(.{
        .name = "apollonius",
        .root_module = exe_mod,
    });

    b.installArtifact(exe);

    const run_cmd = b.addRunArtifact(exe);

    run_cmd.step.dependOn(b.getInstallStep());

    if (b.args) |args| {
        run_cmd.addArgs(args);
    }

    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);

    const lib_unit_tests = b.addTest(.{
        .root_module = lib_mod,
    });

    const run_lib_unit_tests = b.addRunArtifact(lib_unit_tests);

    const exe_unit_tests = b.addTest(.{
        .root_module = exe_mod,
    });

    const run_exe_unit_tests = b.addRunArtifact(exe_unit_tests);

    const test_step = b.step("test", "Run unit tests");
    test_step.dependOn(&run_lib_unit_tests.step);
    test_step.dependOn(&run_exe_unit_tests.step);
}

Apparently, it is necessary to compile miniaudio in ReleaseFast mode to avoid tripping ubsan.

// main.zig
const std = @import("std");
const lib = @import("apollonius_lib");

const c = @cImport(@cInclude("miniaudio.h"));

pub fn main() !void {
    var result: c.ma_result = undefined;
    var engine: c.ma_engine = undefined;

    result = c.ma_engine_init(null, &engine);
    if (result != c.MA_SUCCESS) {
        return error.FailedToInit;
    }
    defer c.ma_engine_uninit(&engine);

    result = c.ma_engine_play_sound(&engine, "./amenbreak.wav", null);
    if (result != c.MA_SUCCESS) {
        return error.FailedToPlaySound;
    }

    std.time.sleep(10 * std.time.ns_per_s);
}

Now, I would like to expose miniaudio as a module. Taking inspiration from this post, I modified my build.zig to compile miniaudio like this:

    const miniaudio_mod = blk: {
        const dep = b.dependency("miniaudio", .{});
        const tc = b.addTranslateC(.{
            .root_source_file = dep.path("miniaudio.h"),
            .target = b.graph.host,
            .optimize = .ReleaseFast,
        });
        const mod = tc.createModule();
        mod.addCSourceFile(.{
            .file = dep.path("miniaudio.c"),
        });
        mod.linkSystemLibrary("pthread", .{});
        break :blk mod;
    };

    exe_mod.addImport("miniaudio", miniaudio_mod);

It builds, but crashes at runtime. I’m not completely sure of what I’m doing here, so here are a few questions:

  • is my code doing something equivalent to what the @cImport is doing in the first version?
  • compilation is slower: 35 seconds for the initial version, 79 seconds for the second version (both with .zig-cache deleted before of course). If @cImport calls translate-c behind the scene, why the difference?
  • why .target = b.graph.host and not .target = target?
  • the runtime crash is the same as the one when you compile in debug mode. If I compile everything with ``–release=fastit works. It looks like the.optimize = .ReleaseFastpassed toaddTranslateC` is ignored?

Anyway, thanks everyone, it is tough to find up to date information about the build system, and this community is gold. Also, big shoutout to @kristoff - your recent video tutorial on the build system helped a lot.

You should use .target = target here, .target = b.graph.host in my post was only necessary because the library was going to be used by the build process.

This shouldn’t be the case:

Try building with zig build --verbose to see whether zig translate-c and the relevant part of zig build-exe is invoked with -OReleaseFast or not.

You can look at dcimgui (GitHub - floooh/dcimgui: All in one Dear ImGui source distro for C++, C and Zig. - basically a ‘Dear ImGui souce distribution’ which can also be used in Zig projects) as example.

Here Dear ImGui and its C bindings are built into a library:

Here the translateC step is defined which translates the cimgui.h C header into a Zig file:

…and here this Zig file is declared as a module (which is the actual module that’s used by upstream projects):

…and finally here, the C library is added as static linker dependency to the Zig module so that any upstream user of the Zig module automatically also links with the C library:

…a project which uses this package first declares it in the build.zig.zon:

…in the build.zig gets the dependency:

…and uses it like this:

…ignore anything in the build.zig files that mention WASM or Emscripten, since this requires a lot of special-case-code tha’s not needed for regular native builds.

So in your case, the original library (dimgui)'s source code is embedded in your repo. What if the library is fetched from an external source? Then the header file will be somewhere in zig-out. How would you indicate include path to the translateC function?

In my project, I used:

  xxx.addIncludePath(.{ .cwd_relative = b.getInstallPath(.prefix, "") });

But it looks fishy. And when I use that project as a dependency of another project, then zig build complains that the header is not found…

project foo
  |
  \_ project bar
        |
        \_ c library

Building bar is ok, the header compiled by translateC is found in zig-out. But building foo fails.

In project bar’s build.zig the header is installed:

lib.installHeadersDirectory(upstream.path("."), "readline", .{
  .include_extensions = &.{
    ..
  } });

Given a Build.Dependency (e.g. what you define via build.zig.zon and lookup via Build.dependency()), you can get a LazyPath relative to the dependency in the Zig cache via Build.Dependency.path('...'), this LazyPath can then be injected into the TranslateC step as include directory.

1 Like

So finally took the time to have a look and it seems Build.Dependency.path('...') leads to a path in the .cache folder that contains the unzipped dependency. But that’s not the “install” folder of the dependency. My translateC depends on the folder structure generated by the installation of the dependency.

The dependency is readline, it expects the headers to be in:

<installation_prefix>/readline/*.h

In readline, like, I expect, in some C libraries, the folder structure of the source can be different from the folder structure of the installed library. So what you need is really the installation folder of the dependency.

So I am not sure that the Dependency.path('...') solves this problem.

I used nukkeldev/miniz in a project, which amalgates all the .h and .c files of richgel999/miniz into a single header/source file.
In his build.zig, nukkeldev sets the header output path like this:

    const header_output = outer: {
        const gen = b.allocator.create(std.Build.GeneratedFile) catch oom();
        gen.* = .{ .step = &run.step, .path = ".zig-cache/generated/miniz/miniz.h" };
        break :outer LP{ .generated = .{ .file = gen } };
    };

Which I can then use in my own build.zig:

const miniz_dep = b.dependency("miniz", .{ .BUILD_HEADER_ONLY = true });
...
my_module.addIncludePath(miniz_dep.path(".zig-cache/generated/miniz"));

You can probably do something similar, but for translateC.

Yes, I can always figure out a workaround, but I am looking for the idiomatic solution to this problem that should be pretty common when depending on C libraries.

As an aside, I didn’t know about miniz, thanks for that!

I would try to use 0.14.0 Release Notes: Allow Packages to Expose Arbitrary LazyPaths by Name

With that it shouldn’t really matter where the path is (whether it is in the cache or installed) your package just has to create a lazy path to it in some way and then you can provide it as a named path to dependent projects.

I have not needed b.addNamedLazyPath("generated", generated_file); so far since with most C libraries, the header files are already available, maybe in an include folder, and you can use those upstream header files with addTranslateC, instead of the ones in zig-out.
For example, you can use addTranslateC and addModule in the build.zig of project bar:

    const c_api = b.addTranslateC(.{
        .root_source_file = upstream.path("include/lib.h"),
        .target = target,
        .optimize = optimize,
    });

    c_api.addIncludePath(upstream.path("include"));

    _ = b.addModule("c_api", .{
        .root_source_file = c_api.getOutput(),
        .target = target,
        .optimize = optimize,
    });

Then addImport this module in the build.zig of your foo project :

    const bar_dep = b.dependency("bar", .{
        .target = target,
        .optimize = optimize,
    });
...
    exe.linkLibrary(bar_dep.artifact("lib"));
    exe.addImport("c", bar_dep.module("c_api"));

Which you can finally @import() in the main.zig of your foo project:

const std = @import("std");
const c = @import("c");

So readline.h has something like this:

#if defined (READLINE_LIBRARY)
#  include "rlstdc.h"
#  include "rltypedefs.h"
#  include "keymaps.h"
#  include "tilde.h"
#else
#  include <readline/rlstdc.h>
#  include <readline/rltypedefs.h>
#  include <readline/keymaps.h>
#  include <readline/tilde.h>
#endif

the READLINE_LIBRARY macro is defined during the readline compilation process (defined in config.h). Which means that, when you include readline.h in a regular project, it expects the include in a particular folder (${prefix}/include/readline/*.h).

For the translateC step, I already do this:

lib.installHeadersDirectory(upstream.path("."), "readline", .{ .include_extensions = &.{
        "chardefs.h",
        "history.h",
        "keymaps.h",
        "readline.h",
        "rlconf.h",
        "rlstdc.h",
        "rltypedefs.h",
        "tilde.h",
    } });
// and later
translateC.addIncludePath(.{ .cwd_relative = b.getInstallPath(.prefix, "include") });

Which is the best I could come up with. I would prefer to have something like lib.getInstallIncludeFolder but it does not seem to exist. Anyway, this works for bar. I can compile an executable in bar that links properly and readline works.

The problem is when adding bar to foo and the translateC step is run during foo’s build, it does not find readline.h. Either the readline.h is not installed (I don’t see anything in foo’s zig-out folder but maybe those headers are installed somewhere in .cache? I don’t see anything) or either the getInstallPath executed in bar’s build.zig is not the same as the one from foo.

I also probably lack the knowledge on how to debug build.zig. --verbose does not produce much and I don’t see a way to generate a visualisation of the build graph to see where a step might be missing.

Ok so I was finally able to produce something that, although idiotic, works for me and makes sense. In foo’s build.zig:

    // readline is the zig project that "wrap" libreadline a la
    // allyourcodebase (the bar project I was talking about)
    const readline_dep = b.dependency("readline", .{});
    // I extract from readline the libreadline dependency (the c
    // library)
    const libreadline_dep =
      readline_dep.builder.dependency("libreadline", .{});
[...]
    // Then I make a dependency between the executable
    // (from the `foo` project) to the install step of libreadline
    exe.step.dependOn(libreadline_dep.builder.getInstallStep());