Zig searches for libvulkan.so but on my system it is libvulkan.so.1

Help
1

Hello, I am planning on using the zig vulkan bindings generator, but as an exercise, I wanted to try linking to my system’s vulkan loader through the build system and using the c headers first so I can develop my understanding.

On fedora, I installed the vulkan-loader package. If I run rpm -ql vulkan-loader, it lists the install paths for all of its files, which includes libvulkan.so.1 in several locations.

/etc/vulkan
/etc/vulkan/explicit_layer.d
/etc/vulkan/icd.d
/etc/vulkan/implicit_layer.d
/usr/lib/.build-id
/usr/lib/.build-id/98
/usr/lib/.build-id/98/7c6206801e376057ad9c0644a425621e6e7a38
/usr/lib64/libvulkan.so.1
/usr/lib64/libvulkan.so.1.4.313
/usr/share/doc/vulkan-loader
/usr/share/doc/vulkan-loader/CONTRIBUTING.md
/usr/share/doc/vulkan-loader/README.md
/usr/share/licenses/vulkan-loader
/usr/share/licenses/vulkan-loader/LICENSE.txt
/usr/share/vulkan
/usr/share/vulkan/explicit_layer.d
/usr/share/vulkan/icd.d
/usr/share/vulkan/implicit_layer.d
/etc/vulkan
/etc/vulkan/explicit_layer.d
/etc/vulkan/icd.d
/etc/vulkan/implicit_layer.d
/usr/lib/.build-id
/usr/lib/.build-id/a3
/usr/lib/.build-id/a3/2eaf56840c481778153de48194c409b35d9410
/usr/lib/libvulkan.so.1
/usr/lib/libvulkan.so.1.4.313
/usr/share/doc/vulkan-loader
/usr/share/doc/vulkan-loader/CONTRIBUTING.md
/usr/share/doc/vulkan-loader/README.md
/usr/share/licenses/vulkan-loader
/usr/share/licenses/vulkan-loader/LICENSE.txt
/usr/share/vulkan
/usr/share/vulkan/explicit_layer.d
/usr/share/vulkan/icd.d
/usr/share/vulkan/implicit_layer.d

However, zig build reports that it cannot find the library when I attempt to link to vulkan:

error: error: unable to find dynamic system library 'vulkan' using strategy 'paths_first'. searched paths:
  /usr/local/lib64/libvulkan.so
  /usr/local/lib64/libvulkan.a
  /usr/local/lib/libvulkan.so
  /usr/local/lib/libvulkan.a
  /lib64/libvulkan.so
  /lib64/libvulkan.a
  /lib/libvulkan.so
  /lib/libvulkan.a
  /usr/lib64/libvulkan.so
  /usr/lib64/libvulkan.a
  /usr/lib/libvulkan.so
  /usr/lib/libvulkan.a

As far as I can tell, this appears to be due to the .1 suffix at the end of the installed lib.

The main file is the default one created by zig init-exe. My build.zig:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    const steps = .{
        .run = b.step("run", "Run the app"),
        .@"test" = b.step("test", "Run tests"),
    };

    const root_module = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
        .imports = &.{},
    });

    root_module.linkSystemLibrary("vulkan", .{});

    const exe = b.addExecutable(.{
        .name = "learn_vulkan",
        .root_module = root_module,
    });

    b.installArtifact(exe);

    const run_cmd = b.addRunArtifact(exe);
    run_cmd.step.dependOn(b.getInstallStep());
    steps.run.dependOn(&run_cmd.step);

    if (b.args) |args| {
        run_cmd.addArgs(args);
    }

    const exe_tests = b.addTest(.{
        .root_module = exe.root_module,
    });
    const run_exe_tests = b.addRunArtifact(exe_tests);

    steps.@"test".dependOn(&run_exe_tests.step);
}

Question:

  1. Is this how I should link to the vulkan loader?
  2. How do I tell zig to look for the .1 suffix?
  3. Is vulkan-loader actually the right package?

I am not familiar what the best practices for using system libraries for software development, so I am a bit confused here. I also feel like I have vague memories of doing things like ln -s libfoo.so.1.4.3 libfoo.so before to work around issues with applications expecting libraries without the version suffix. If anyone has some wisdom to share on this topic I would be very interested in understanding what is going on!

2

I believe that by default zig build tries to use pkg-config to find the specific link flags for a given library. I don’t see any .pc files in the package contents you posted, you may need the vulkan-loader-devel Fedora package to get the headers and pkg-config files.

3 Likes
3

Interesting, it seems that vulkan-loade-devel includes not only the pkg-config file, but also the missing libvulkan.so.

/usr/lib64/cmake/VulkanLoader
/usr/lib64/cmake/VulkanLoader/VulkanLoaderConfig-release.cmake
/usr/lib64/cmake/VulkanLoader/VulkanLoaderConfig.cmake
/usr/lib64/cmake/VulkanLoader/VulkanLoaderConfigVersion.cmake
/usr/lib64/libvulkan.so
/usr/lib64/pkgconfig/vulkan.pc

Edit: I found this in the vulkan loader documentation:

For Linux and MacOS, shared libraries are versioned based on a suffix. Thus, the ABI number is not encoded in the base of the library filename as on Windows. On Linux an application wanting to link to the latest Vulkan ABI version would just link to the name vulkan (libvulkan.so). A specific Vulkan ABI version can also be linked to by applications (e.g. libvulkan.so.1).

It appears that having the abi version is common, but not universal. However, it seems one should only worry about that when using dlopen to manually link the vulkan loader.

If I compile with --verbose I see that zig build indeed calls pkg-config vulkan --cflags --libs. When I run that, it outputs -lvulkan which doesn’t seem to indicate anything about the abi version. If I check the libraries in /usr/lib/, most have the suffix:

~ > ls /usr/lib/ | rg ".so(.[0-9])+\$" | wc -l
473
~ > ls /usr/lib/ | rg ".so\$" | wc -l
46

If my package was installed on a system without vulkan-loader-devel, would my program find the non-devel version with the suffix?

4

At risk of going off on a bit of a tangent here, but is it common on Fedora to not just include the symlinks (i.e. libvulkan.so -> libvulkan.so.1) in /usr/lib?
I have always had the expectation/assumption that the suffix-free variation would exist if the dependency was installed, and would not require a development package for the distros that use this pattern in their packaging. Likely my own naivety with such distros, but I would assume that any dependency that links against libvulkan would expect this, no? Or is this just unique to libvulkan/libGL type libraries and their more involved dynamic linking process to find the correct runtime implementation?

5

I would have thought this as well, but looking at the libs installed in my system contradicts this. I remember some weird situations requiring me to make the symlink myself for llvm on ubuntu, but I am new to fedora and this is my first time seeing anything like this.

6

libvulkan.so.1 represents version 1 of Vulkan, which is the current version (1.4 to be more precise). Major versions will only be changed for ABI incompatible revisions, which hasn’t happened yet. A future Vulkan 2 would be ABI incompatible with your program that was compiled for version 1. It doesn’t make sense to have an ABI agnostic shared library (libvulkan.so without the 1 at the end). This file shouldn’t exist is any OS. In those where it does exist, it should be a symlink to the current version. As long as the current version is 1, this won’t cause problems. But, as you can see, you should specify that you want the version 1. That is the recommended approach by Vulkan and Linux. On Windows, where such versioning is not standard, the dll is called vulkan-1.dll, for the same purpose.
So, to answer your question, specifying the 1 at the end ensures the widest compatibility with OSs.

1 Like
7

Yes, it is clear to me now that I should do that when using dlopen.

However, I am not sure how to do this with zig build or if I should worry about it at all. If I recall correctly (I might not have tried vulkan1), vulkan-1/vulkan.1/vulkan1 as the name for the system library searches for libvulkan-1.so/libvulkan.1.so/libvulkan1.so.

The vulkan.pc file includes version information, specifying it is Version 1.4.3, but I don’t know if or how zig build makes use of that information. Calling pkg-config with the same args as zig build only outputs -lvulkan. I think the distro’s tooling should handle this somehow, but since installing vulkan-loader-devel also installed libvulkan.so I’m not sure how to verify that since as far as I am able to tell, zig will search for libvulkan.so.

8

Yeah, I don’t see how to tell Zig that you are looking for a specific version of a shared library. When targeting Windows, you can just do module.linkSystemLibrary("Vulkan-1", .{ .preferred_link_mode = .dynamic}). That won’t work for Linux, because Zig will look for libvulkan-1.so, which is not correct. I think this warrants opening an issue.

1 Like
9

It is standard to search for version agnostic library file (.so/.a) during link-time on the developer system doing the build. As explained these have no real purpose at runtime on actual user systems, this isn’t what is linked (at least when dynamically linking .so).

Your library gets linked via -l <library name> (eg: -l vulkan / -lvulkan) and that matches say libvulkan.so. This should be a symlink (typically) and resolves to actual versioned library.

This file might not be the actual one linked into your program/library being built, for example libvulkan.so.1.2.3 may be the version on the system but the DT_SONAME entry in the library’s metadata might be libvulkan.so.1 which then symlinks to the real versioned library on the system that is resolved at runtime.

You can inspect your built binary with a tool like patchelf --print-needed path/to/binary to get a list of direct dependencies linked dynamically. Whereas a command like ldd path/to/binary will resolve recursively from it’s own links and show any child links along the way (anything that uses dlopen() later at runtime of your program won’t be caught here, and can be a more common problem for end-users depending how a developer has tried to load for libraries at runtime).

With Nvidia CUDA, rather than the typical symlink .so placeholders to satisfy the linker, I’ve seen .so stubs that instead satisfy the linker without requiring the build machine to have the multi-GB libraries available that are only required at run-time. This has been a common area of confusion with some developers where I’ve seen they’ve mistakenly thought that .so is needed at runtime (which also happens when some developers load additional libraries at runtime explicitly via dlopen() calls).

You can run the command ldconfig with default search paths it’s configured for, or with your own additional directories and this will build a lookup cache (/etc/ld.so.cache) for runtime resolution, a side-effect of this is it will create symlinks for libraries found ensuring there is a symlink for any library that does not have the same filename as it’s DT_SONAME (which programs are linking to at build/link-time). This is what end-users systems will have, such that a libvulkan.so.1.2.3 may additionally have a symlink libvulkan.so.1 and/or libvulkan.so.1.2.

Beyong the -l vulkan link syntax, you can use -l:libvulkan.so syntax which is more explicit (you must provide the full library name to link to, instead of implicitly searching for lib<name>.so), this syntax allows you to link to a library by it’s filename directly if you are missing the libvulkan.so symlink that -dev/-devel packages provide. Link wise, it will still behave the same and link to the DT_SONAME however.

You would generally avoid this alternative syntax as it rarely has any benefit to prefer it, and if you do stray from say -l:libvulkan.so to some more specific like -l:libvulkan.so.1.2.3, then it would be less flexible for others to build from. It’s most common to see when static linking such as -l:libvulkan.a, which could also be expressed as -Wl,-Bstatic -l vulkan -Wl,-Bdynamic (-Wl,<linker arg> prefix is necessary if using clang/gcc/zig cc instead of a linker directly), which will switch to static linking and then back to dynamic (default) after. Zig’s linker is a bit more flexible with some additional options to control static vs dynamic precedence and how they’re used with the search paths (see zig build-lib --help for more info).

Finally, at runtime, if you’re making you’re own dlopen() calls, you very well may find that users don’t have the developer friendly .so symlinks installed (or like with the CUDA situation, a developer tries to run their program on the build system that has search resolution picking the .so stubs which are invalid at runtime), so requesting a library via it’s name alone may not work so well. Instead you could query by filename, you’d want this to be the equivalent of the DT_SONAME entry (libvulkan.so.1 in your case, not whatever your system may have such as libvulkan.so.1.2.3) and that should work well.

If it’s a custom built library that you bundle with your program, you can also load that instead, and ensure that it links to any other libraries bundled instead of the system libraries by giving priority with patchelf --set-rpath '$ORIGIN' or similar.

I know I’ve blabbed on quite a bit, but as someone who went through similar confusion long ago, hopefully there’s helpful tidbits covered above :sweat_smile:

I wrote this with a Zig agnostic stance. I’m not too familiar with that side of things. I know that Zig removes any system paths from search if you customize the target from native, even if it’s the same triple effectively.

Often I’d run into issues with /usr/lib64 not being searched when building on Fedora as a result, but with zig cc I can add that via -L. I’m not sure with zig build however, other than modifying a zig.build or similar it seems to lack the ability to add additional search paths that are actually checked? :man_shrugging:

3 Likes