Trying to test a library where the original share library denpending on the user terminal location

Help
, ,
1

I know it sounds a bit crazy about the title, so let me tell you what exactly happened. I will also find the author of the library if it is not solvable at the zig level.

Awhile back, I have asked a question about linking c header that loads a dynamic library (.dll / .so). The solution works and I did manage to use the library to writing something; however, for the sake of convenience, I am thinking of porting the library into zig for my future projects.

The library consists a bunch of dynamic libraries for different os and architectures, along with a header file, so I have layered the library like so and this is the reduced form that represented all the necessary files:

libs
  |-- sunvox
     |-- windows
     |  |-- lib_x86_64
     |  |  |-- sunvox.dll
     |-- sunvox.c
     |-- sunvox.h
src
  |-- zunvox.zig
build.zig
build.zig.zon

Inspired by the zaudio build.zig, I have written the following code to build the project:

    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    _ = b.addModule("zunvox", .{
        .root_source_file = b.path("src/zunvox.zig"),
        .target = target,
    });

    const sunvox = b.addModule("sunvox", .{
        .target = target,
        .optimize = optimize,
    });

    const sunvox_lib = b.addLibrary(.{
        .name = "sunvox",
        .root_module = sunvox,
        .linkage = .static,
    });

    b.installArtifact(sunvox_lib);

    sunvox.addIncludePath(b.path("libs/sunvox"));
    sunvox_lib.linkLibCpp();

    sunvox.addCSourceFile(.{
        .file = b.path("libs/sunvox/sunvox.c"),
        .flags = &.{
            "-DSUNVOX_MAIN",
        },
    });

Followed by the testing module in the same function:

    const test_step = b.step("test", "zunvox tests");

    const test_module = b.addModule("test", .{
        .root_source_file = b.path("src/zunvox.zig"),
        .target = target,
        .optimize = optimize,
    });

    const tests = b.addTest(.{
        .name = "zunvox-tests",
        .root_module = test_module,
    });

    tests.linkLibrary(sunvox_lib);

    b.installArtifact(tests);

    test_step.dependOn(&b.addRunArtifact(tests).step);

In order to build the header, inspired by zaudio, I have added a mock c file like shown (sunvox.c):

#define SUNVOX_IMPLEMENTATION
#include "sunvox.h"

To ensure everything works before I go through the remaining functions, I have only called the function for loading the dll:

const std = @import("std");

const SvError = error{
    FailedToLoadDll,
};

pub fn loadDll() SvError!void {
    if (sv_load_dll() < 0) {
        return SvError.FailedToLoadDll;
    }
}

extern fn sv_load_dll() c_int;

test "init sunvox library" {
    try loadDll();
}

And here is the problem: When I build the project right now, the test fails because the load_dll() function in the c header can’t load the share library, but if I put the sunvox.dll in the zig-out/bin, I can observe the following behavior:

  • running zig build test where the terminal located at the build.zig file can’t identify the .dll file
  • running zig build test where the terminal located at the compiled binaries (after you do cd zig-out/bin) passes the test where the .dll is found.

Based on these findings, the header file and the .dll file is highly dependent on the location of the terminal. For a quick hack, I can copy the .dll file at the location where build.zig is located, but here comes in a couples of problems:

  • If I ship this library, this means at the user end, they must manually put the .dll at the project root location to compile their applications, but this is counter intuitive because this makes cross compilation impossible since the library has compiled for different os and architectures where their name are identical.
  • if the users change their directory location for their terminal, their test will fail once again because the program can’t find the .dll at their current terminal location.

Meanwhile, I know how to ship the dynamic library into the final binary, by using addIntallFile() to copy the .dll into zig-out/bin:

b.getInstallStep().dependOn(&b.addInstallFile(b.path("libs/sunvox/windows/lib_x86_64/sunvox.dll"), "lib/sunvox.dll").step);

But the problem is, since this function can only add files under the zig-out folder, and it is not designed for putting the dynamic library at the build.zig level, we can’t copy the file back at the build.zig level which we shouldn’t do as well. Besides, if I load that as a zig dependencies using zig fetch, since the library will be located in the cache folder, at the user perspective, there is no way to find the .dll and copy that to the build.zig level in the first place.

Thus, the questions are:

  • Is it possible to let zig build test recognize the .dll or other share libraries during testing phase?
  • Is it possible to copy the .dll to the zig-out/bin when the library is linked as a dependency?
  • Am I linking the dynamic library wrong for building a library? If so, what is the preferred way to handle .dll or .so at build.zig?
2

Is it possible to let zig build test recognize the .dll or other share libraries during testing phase?

Try redefining the following line:

test_step.dependOn(&b.addRunArtifact(tests).step);

as:

var test_step_run = b.addRunArtifact(tests).step;
test_step.dependOn(&test_step_run);

This is so that we can call the following:

test_step_run.setCwd(b.path("zig-out/bin"));

This should set the current working directory when running the tests to the correct directory to find your DLL.

Is it possible to copy the .dll to the zig-out/bin when the library is linked as a dependency?

Should be no problem, dependencies have an artifact() method you can use to get a *std.Build.Step.Compile, which you can then install with b.addInstallArtifact().

1 Like
3

Thanks for your solution, with a bit of modification from your code:

var test_step_run = b.addRunArtifact(tests); // without taking that as a step
test_step.dependOn(&test_step_run.step);

test_step_run.setCwd(b.path("zig-out/bin"));

Although the test doesn’t work if zig-out/bin is not present, this is a step of improvement since performing a zig build before a zig build test could properly test the library. For that part, I could attempt to find a way to conditionally pick the .dll for testing.

At the library side, since I have uploaded the library as onto GitHub, I tried to load that as a dependencies for my project using zig fetch. Since my build.zig structure is based on zaudio, I tried to similar way to load the dependency as shown:

const zunvox = b.dependency("zunvox", .{});
exe.root_module.addImport("zunvox", zunvox.module("zunvox"));
exe.root_module.linkLibrary(zunvox.artifact("sunvox"));

b.installArtifact(exe);

When I run zig build, seems like the dependency is download, but the header and the c file is missing from the library:

run
└─ run exe zunvox_lib_test
   └─ compile exe zunvox_lib_test Debug native
      └─ compile lib sunvox Debug native 1 errors
error: failed to check cache: ...\zig\p\zunvox-0.0.0-kHSYN_oaAACsiA4mWfWQpH-qh6gYVTDnsQdDSMOKkQdu\libs\sunvox\sunvox.c' file_hash FileNotFound

Is it something I have done wrong when I create the module or linking the dependency? Considering I am not quite sure why zaudio break the library down into two modules, but I copied the similar structure where I though it should also works for my library due to the similar library structure despite the additional .dll:

_ = b.addModule("zunvox", .{
    .root_source_file = b.path("src/zunvox.zig"),
    .target = target,
});

const sunvox = b.addModule("sunvox", .{
    .target = target,
    .optimize = optimize,
});

const sunvox_lib = b.addLibrary(.{
    .name = "sunvox",
    .root_module = sunvox,
    .linkage = .static,
});
4

You need to add libs to your .paths in your build.zig.zon, the package manager only keeps what is mentioned by paths.

1 Like
5

Thanks! I didn’t know that the path in build.zig.zon also matters; after I have added the directory, and re-applying the zig fetch after committed the change, the c and the header are now properly loaded into the dependency, along with the shared libraries.

The final steps remains: with the updated library, I tried to build a project, I still can’t pull the share library out from the dependency to the zig-out directory. However, if it is stored in the app data folder, how can I obtain the share library for my project?

1 Like
6

Alright, I have found my way out:

My previous two attempts were the following code:

b.getInstallStep().dependOn(&b.addInstallBinFile(b.path("libs/sunvox/windows/lib_x86_64/sunvox.dll"), "sunvox.dll").step);
b.getInstallStep().dependOn(&b.addInstallFile(b.path("libs/sunvox/windows/lib_x86_64/sunvox.dll"), "lib/sunvox.dll").step);

It works at the library side and it does copy the share library to the bin, but it couldn’t copy into my project as a dependency. I am not sure about the reason for now, but I would assuming that because the library is installed as a dependency, which it has its own “build.zig” where the install function only work “locally” to the dependency, so it doesn’t apply to my current project. Thus, we need to explicitly to install that files with a function that has a file location pointing to my current project location.

To do achieve that behavior, since I am currently working on some libraries from the zig-gamedev, I found that “zwindows” does require some .dll for their library, and they need to build a function to specifically copy the share library into the project. Hence, based on the code they have, I did the similar trick as shown:

At the library side, I have a function for the project to install the share library as shown:


pub fn installSunVoxDll(
    step: *std.Build.Step,
    sunvox_dll: *std.Build.Dependency,
    install_dir: std.Build.InstallDir,
) void {
    const b = step.owner;
    step.dependOn(
        &b.addInstallFileWithDir(
            .{ .dependency = .{
                .dependency = sunvox_dll,
                .sub_path = "libs/sunvox/windows/lib_x86_64/sunvox.dll",
            } },
            install_dir,
            "sunvox.dll",
        ).step,
    );
}

At the project level, I need to explicitly import the share library:

// install the original dynamic library
@import("zunvox").installSunVoxDll(&exe.step, zunvox, .bin);

With such setup, if I build the project, the share library will be appeared at the project bin folder. This means, with some branching to fetch the share library based on the architecture, I will able to compile a binary with the correct share library at project.

7

I think we have found the solutions, so I am going to summarize here.

To handle the share library that depends on the working directory, we could use:

If you need the .dll or .so (or any files) exported for your project while this library is loaded as a dependency, you need to declare a function at the build.zig of the library:

pub fn installSunVoxDll(
    step: *std.Build.Step,
    sunvox_dll: *std.Build.Dependency,
    install_dir: std.Build.InstallDir,
) void {
    const b = step.owner;
    step.dependOn(
        &b.addInstallFileWithDir(
            .{ .dependency = .{
                .dependency = sunvox_dll,
                .sub_path = "libs/sunvox/windows/lib_x86_64/sunvox.dll",
            } },
            install_dir,
            "sunvox.dll",
        ).step,
    );
}

And call that function into the build.zig of your project:

// install the original dynamic library
@import("zunvox").installSunVoxDll(&exe.step, zunvox, .bin);

Finally, we need to specific the path of the your c library in the build.zig.zon such that the dependency will include the c code to build the library for your project:

With these setup, you should able to build a library with a shared object.

Alright, with the working setup, I will study the code and functions to see how and why this works, but most importantly, thank you for all who helps for setting up the library!