Zig cc dynamic C lib runtime error on NixOS

Given this flake

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-23.11";
    flake-utils.url = "github:numtide/flake-utils";
    zig-overlay.url = "github:mitchellh/zig-overlay";
    zls.url = "github:zigtools/zls";
  };

  outputs = inputs:
    let
      inherit (inputs) nixpkgs zig-overlay flake-utils zls gsl;
      systems = [ "x86_64-linux" ];
    in
    flake-utils.lib.eachSystem systems (system:
      let
        pkgs = import nixpkgs { inherit system; };
        # zig = zig-overlay.packages.${system}."0.11.0";
        zig = zig-overlay.packages.${system}.master;
        zls_pkg = zls.packages.${system}.default;
        gsl = pkgs.gsl;
        blas = pkgs.blas;
        pkgconfig = pkgs.pkg-config;
        gcc = pkgs.gcc;
      in
      {
        devShells.default = pkgs.mkShell {
          packages = [ zig zls_pkg gcc pkgconfig ];
          nativeBuildInputs = [ gsl.dev blas pkgconfig ];
          buildInputs = [ gsl.dev blas pkgconfig ];
        };
      });
}

and the C program main-gsl.c

#include <gsl/gsl_cdf.h>
#include <stdio.h>

int main(){
    double bottom_tail = gsl_cdf_gaussian_P(-1.96, 1);
    printf("Area betweeen [-1.96, 1.96]: %g\n", 1 - 2*bottom_tail);
}

this works: gcc main-gsl.c -lgsl -lblas && ./a.out

Area betweeen [-1.96, 1.96]: 0.950004

but this does not: zig cc main-gsl.c -lgsl -lblas -lc && ./a.out

./a.out: symbol lookup error: /nix/store/0f2jj365fr1z5cfi73bvxzh7sbwg0ns7-gsl-2.7.1/lib/libgsl.so.27: undefined symbol: cblas_ctrmv

ldd a.out gives

linux-vdso.so.1 (0x00007fff78a78000)
libgsl.so.27 => /nix/store/0f2jj365fr1z5cfi73bvxzh7sbwg0ns7-gsl-2.7.1/lib/libgsl.so.27 (0x00007f3184400000)
libc.so.6 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libc.so.6 (0x00007f3184218000)
libm.so.6 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libm.so.6 (0x00007f31847af000)
/nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/ld-linux-x86-64.so.2 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib64/ld-linux-x86-64.so.2 (0x00007f3184891000)

My understanding is that the blas libs were not linked explicitly, ld tries to resolve the symbols at run time and fails.

ldd a.out on the gcc’d a.out:

linux-vdso.so.1 (0x00007fcd8c10f000)
libgsl.so.27 => /nix/store/0f2jj365fr1z5cfi73bvxzh7sbwg0ns7-gsl-2.7.1/lib/libgsl.so.27 (0x00007fcd8be00000)
libblas.so.3 => /nix/store/jx5r3fyz26nvrj5zcx2f7fc2yszfalwb-blas-3/lib/libblas.so.3 (0x00007fcd8a380000)
libc.so.6 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libc.so.6 (0x00007fcd8a198000)
libm.so.6 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libm.so.6 (0x00007fcd8a0b8000)
libpthread.so.0 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libpthread.so.0 (0x00007fcd8c102000)
libgfortran.so.5 => /nix/store/4avaj23y8h6hq8wck73250w77y0s6kvi-gfortran-12.3.0-lib/lib/libgfortran.so.5 (0x00007fcd89c00000)
libgomp.so.1 => /nix/store/myw67gkgayf3s2mniij7zwd79lxy8v0k-gcc-12.3.0-lib/lib/libgomp.so.1 (0x00007fcd8a073000)
/nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/ld-linux-x86-64.so.2 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib64/ld-linux-x86-64.so.2 (0x00007fcd8c111000)
libquadmath.so.0 => /nix/store/4avaj23y8h6hq8wck73250w77y0s6kvi-gfortran-12.3.0-lib/lib/libquadmath.so.0 (0x00007fcd8a02c000)
libgcc_s.so.1 => /nix/store/4avaj23y8h6hq8wck73250w77y0s6kvi-gfortran-12.3.0-lib/lib/libgcc_s.so.1 (0x00007fcd8c0e1000)
libdl.so.2 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libdl.so.2 (0x00007fcd8c0da000)

I am trying to understand why this happens to learn more about Zig and NixOS - I am more familiar with high-level languages in the scientific computing realm, i.e. Python and Julia.

Try with zig cc -target x86_64-linux by default zig applies host specific optimizations so the blas headers might look for different symbols that were compiled in. Nixpkgs are always packaged with -mcpu=baseline to have reproducible output. -target and -Dtarget (for zig build) always use the baseline for that target similar to -mcpu=baseline Exe files not interchangeable among identical Linux systems - #6 by andrewrk

Thanks for taking the time and the explanations!

I tried with zig cc -target x86_64-linux src/main-gsl.c -lgsl -lblas but I get:

error: unable to find Dynamic system library 'gsl' using strategy 'paths_first'. searched paths: none
error: unable to find Dynamic system library 'blas' using strategy 'paths_first'. searched paths: none

You may want to try with $(pkg-config --libs --cflags blas gsl) instead

zig cc -target x86_64-linux $(pkg-config --libs --cflags blas gsl) src/main-gsl.c && ./a.out

bash: ./a.out: cannot execute: required file not found

Most likely BLAS and GSL aren’t in your LD_LIBRARY_PATH, if you run ldd on ./a.out you’ll see entries for “not found”

Ah, I had a similar idea, but I thought LD_LIBRARY_PATH would have been taken care of by the flake.nix…

ldd ./a.out

./a.out: error while loading shared libraries: /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libc.so: invalid ELF header

Huh, what does readelf -a say on the a.out?

readelf -a ./a.out

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x1001470
  Start of program headers:          64 (bytes into file)
  Start of section headers:          5192 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         9
  Size of section headers:           64 (bytes)
  Number of section headers:         28
  Section header string table index: 26

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .interp           PROGBITS         0000000001000238  00000238
       0000000000000019  0000000000000000   A       0     0     1
  [ 2] .dynsym           DYNSYM           0000000001000258  00000258
       0000000000000090  0000000000000018   A       5     1     8
  [ 3] .gnu.hash         GNU_HASH         00000000010002e8  000002e8
       0000000000000024  0000000000000000   A       2     0     8
  [ 4] .hash             HASH             000000000100030c  0000030c
       0000000000000038  0000000000000004   A       2     0     4
  [ 5] .dynstr           STRTAB           0000000001000344  00000344
       000000000000004e  0000000000000000   A       0     0     1
  [ 6] .rela.plt         RELA             0000000001000398  00000398
       0000000000000048  0000000000000018  AI       2    15     8
  [ 7] .rodata           PROGBITS         00000000010003e0  000003e0
       0000000000000031  0000000000000000 AMS       0     0     8
  [ 8] .eh_frame_hdr     PROGBITS         0000000001000414  00000414
       0000000000000014  0000000000000000   A       0     0     4
  [ 9] .eh_frame         PROGBITS         0000000001000428  00000428
       000000000000003c  0000000000000000   A       0     0     8
  [10] .text             PROGBITS         0000000001001470  00000470
       0000000000000090  0000000000000000  AX       0     0     16
  [11] .init             PROGBITS         0000000001001500  00000500
       0000000000000003  0000000000000000  AX       0     0     1
  [12] .fini             PROGBITS         0000000001001503  00000503
       0000000000000003  0000000000000000  AX       0     0     1
  [13] .plt              PROGBITS         0000000001001510  00000510
       0000000000000040  0000000000000000  AX       0     0     16
  [14] .dynamic          DYNAMIC          0000000001002550  00000550
       0000000000000120  0000000000000010  WA       5     0     8
  [15] .got.plt          PROGBITS         0000000001002670  00000670
       0000000000000030  0000000000000000  WA       0     0     8
  [16] .debug_loc        PROGBITS         0000000000000000  000006a0
       000000000000007f  0000000000000000           0     0     1
  [17] .debug_abbrev     PROGBITS         0000000000000000  0000071f
       000000000000014f  0000000000000000           0     0     1
  [18] .debug_info       PROGBITS         0000000000000000  0000086e
       00000000000003c0  0000000000000000           0     0     1
  [19] .debug_str        PROGBITS         0000000000000000  00000c2e
       0000000000000164  0000000000000001  MS       0     0     1
  [20] .comment          PROGBITS         0000000000000000  00000d92
       000000000000007d  0000000000000001  MS       0     0     1
  [21] .debug_frame      PROGBITS         0000000000000000  00000e10
       0000000000000030  0000000000000000           0     0     8
  [22] .debug_line       PROGBITS         0000000000000000  00000e40
       0000000000000258  0000000000000000           0     0     1
  [23] .debug_aranges    PROGBITS         0000000000000000  00001098
       0000000000000080  0000000000000000           0     0     1
  [24] .debug_ranges     PROGBITS         0000000000000000  00001118
       00000000000000a0  0000000000000000           0     0     1
  [25] .symtab           SYMTAB           0000000000000000  000011b8
       0000000000000120  0000000000000018          27     4     8
  [26] .shstrtab         STRTAB           0000000000000000  000012d8
       0000000000000105  0000000000000000           0     0     1
  [27] .strtab           STRTAB           0000000000000000  000013dd
       0000000000000069  0000000000000000           0     0     1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), l (large), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  PHDR           0x0000000000000040 0x0000000001000040 0x0000000001000040
                 0x00000000000001f8 0x00000000000001f8  R      0x8
  INTERP         0x0000000000000238 0x0000000001000238 0x0000000001000238
                 0x0000000000000019 0x0000000000000019  R      0x1
      [Requesting program interpreter: /lib/ld-musl-x86_64.so.1]
  LOAD           0x0000000000000000 0x0000000001000000 0x0000000001000000
                 0x0000000000000464 0x0000000000000464  R      0x1000
  LOAD           0x0000000000000470 0x0000000001001470 0x0000000001001470
                 0x00000000000000e0 0x00000000000000e0  R E    0x1000
  LOAD           0x0000000000000550 0x0000000001002550 0x0000000001002550
                 0x0000000000000150 0x0000000000000150  RW     0x1000
  DYNAMIC        0x0000000000000550 0x0000000001002550 0x0000000001002550
                 0x0000000000000120 0x0000000000000120  RW     0x8
  GNU_RELRO      0x0000000000000550 0x0000000001002550 0x0000000001002550
                 0x0000000000000150 0x0000000000000ab0  R      0x1
  GNU_EH_FRAME   0x0000000000000414 0x0000000001000414 0x0000000001000414
                 0x0000000000000014 0x0000000000000014  R      0x4
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000001000000  RW     0x0

 Section to Segment mapping:
  Segment Sections...
   00     
   01     .interp 
   02     .interp .dynsym .gnu.hash .hash .dynstr .rela.plt .rodata .eh_frame_hdr .eh_frame 
   03     .text .init .fini .plt 
   04     .dynamic .got.plt 
   05     .dynamic 
   06     .dynamic .got.plt 
   07     .eh_frame_hdr 
   08     

Dynamic section at offset 0x550 contains 18 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libgsl.so.27]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000017 (JMPREL)             0x1000398
 0x0000000000000002 (PLTRELSZ)           72 (bytes)
 0x0000000000000003 (PLTGOT)             0x1002670
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000006 (SYMTAB)             0x1000258
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000005 (STRTAB)             0x1000344
 0x000000000000000a (STRSZ)              78 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x10002e8
 0x0000000000000004 (HASH)               0x100030c
 0x000000000000000c (INIT)               0x1001500
 0x000000000000000d (FINI)               0x1001503
 0x0000000000000000 (NULL)               0x0

Relocation section '.rela.plt' at offset 0x398 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000001002688  000100000007 R_X86_64_JUMP_SLO 0000000000000000 __libc_start_main + 0
000001002690  000200000007 R_X86_64_JUMP_SLO 0000000000000000 gsl_cdf_gaussian_P + 0
000001002698  000300000007 R_X86_64_JUMP_SLO 0000000000000000 printf + 0
No processor specific unwind information to decode

Symbol table '.dynsym' contains 6 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main
     2: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND gsl_cdf_gaussian_P
     3: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND printf
     4: 0000000001001500     0 NOTYPE  GLOBAL DEFAULT   11 _init
     5: 0000000001001503     0 NOTYPE  GLOBAL DEFAULT   12 _fini

Symbol table '.symtab' contains 12 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crt1.c
     2: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS main-gsl.c
     3: 0000000001002550     0 NOTYPE  LOCAL  HIDDEN    14 _DYNAMIC
     4: 0000000001001470     0 NOTYPE  GLOBAL DEFAULT   10 _start
     5: 0000000001001486    35 FUNC    GLOBAL DEFAULT   10 _start_c
     6: 00000000010014b0    77 FUNC    GLOBAL DEFAULT   10 main
     7: 0000000001001500     0 NOTYPE  GLOBAL DEFAULT   11 _init
     8: 0000000001001503     0 NOTYPE  GLOBAL DEFAULT   12 _fini
     9: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main
    10: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND gsl_cdf_gaussian_P
    11: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND printf

Histogram for bucket list length (total of 6 buckets):
 Length  Number     % of total  Coverage
      0  2          ( 33.3%)
      1  3          ( 50.0%)     60.0%
      2  1          ( 16.7%)    100.0%

Histogram for `.gnu.hash' bucket list length (total of 1 bucket):
 Length  Number     % of total  Coverage
      0  0          (  0.0%)
      1  0          (  0.0%)      0.0%
      2  1          (100.0%)    100.0%

No version information found in this file.

FWIW, $LD_LIBRARY_PATH is empty.

Ah sorry, try x86_64-linux-gnu, seems like it links to musl by default

zig cc -target x86_64-linux-gnu $(pkg-config --libs --cflags blas gsl) src/main-gsl.c && ./a.out

./a.out: error while loading shared libraries: libgsl.so.27: cannot open shared object file: No such file or directory

ldd ./a.out

linux-vdso.so.1 (0x00007ffd3378a000)
libgsl.so.27 => not found
libc.so.6 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/libc.so.6 (0x00007f7e9084f000)
/lib64/ld-linux-x86-64.so.2 => /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib64/ld-linux-x86-64.so.2 (0x00007f7e90a39000)

Change gsl.dev to just gsl in your buildinputs
You only need gcc, zig and pkg-config in nativeBuildInputs
Libs go to buildInputs

New flake.nix

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-23.11";
    flake-utils.url = "github:numtide/flake-utils";
    zig-overlay.url = "github:mitchellh/zig-overlay";
    zls.url = "github:zigtools/zls";
  };

  outputs = inputs:
    let
      inherit (inputs) nixpkgs zig-overlay flake-utils zls;
      systems = [ "x86_64-linux" ];
    in
    flake-utils.lib.eachSystem systems (system:
      let
        pkgs = import nixpkgs { inherit system; };
        # zig = zig-overlay.packages.${system}."0.11.0";
        zig = zig-overlay.packages.${system}.master;
        zls_pkg = zls.packages.${system}.default;
        gsl = pkgs.gsl;
        blas = pkgs.blas;
        pkgconfig = pkgs.pkg-config;
        gcc = pkgs.gcc;
      in
      {
        devShells.default = pkgs.mkShell {
          packages = [ zig zls_pkg ];
          nativeBuildInputs = [ gcc zig pkgconfig ];
          buildInputs = [ gsl blas ];
        };
      }
    );
}

same results as in Zig cc dynamic C lib runtime error on NixOS - #12 by fleimgruber

I forgot that mkShell does not set the LD_LIBRARY_PATH, you can do it like this:

devShells.default = pkgs.mkShell rec {
  nativeBuildInputs = with pkgs; [ gcc zig zls_pkg pkg-config ];
  buildInputs = [ gsl blas ];
  shellHook = with pkgs.lib; ''
  export LD_LIBRARY_PATH="${makeLibraryPath buildInputs}:$LD_LIBRARY_PATH"
  '';
}

Slightly funny since it seems to set PKG_CONFIG_PATH though?

Anyhow, you might want to use GitHub - Cloudef/zig2nix: Flake for packaging, building and running Zig projects. that makes all these simpler for you. You can put the packages into zig-env’s customRuntimeDeps and customRuntimeLibs respectively.

@Cloudef edit with actual remaining error.

With your devShells snippet it still does not work for me, zig cc -target x86_64-linux-gnu $(pkg-config --libs --cflags blas gsl) src/main-gsl.c && ./a.out

./a.out: symbol lookup error: /nix/store/9h2wc53zx6x91472d7xwk2k8f9rcq5yb-gsl-2.7.1/lib/libgsl.so.27: undefined symbol: cblas_ctrmv

i.e. the same error as in OP. What’s the full flake.nix that you tested with?

Thanks for recommending zig2nix, it was on my list of projects to look into. About passing packages into zig-env: I am struggling with getting the packages (gsl and blas) into scope to pass them to zig-env via the custom* lists. I looked at your example for mach, where the packages are one of the outputs(?) of zig-env itself AFAIU, see here.

You can either do what I do there, or use your own nixpkgs input.
The cblas_ctrmv seems to come from libgslcblas.so which should also be linked by $(pkg-config --libs --cflags blas gsl)

I think I got zig2nix to the point where I see the same results than without it, so I would rather like to understand the cause than using yet another layer. I can share my zig2nix flake.nix if you want to take a look and carry it over the finishing line.

Back to analysis, after your comment I confirmed that libgslcblas.so does show up in the output of pkg-config --libs --cflags blas gsl, but I do not understand why it does not get linked with zig cc (it does get linked with gcc though).

-I/nix/store/bwms0ryhcqf7kyql1bs88a506m3dbvaw-gsl-2.7.1-dev/include -I/nix/store/zd7mm186212ibl659yqbm7nn4miqnrdd-blas-3-dev/include -L/nix/store/9h2wc53zx6x91472d7xwk2k8f9rcq5yb-gsl-2.7.1/lib -L/nix/store/q8fbd9y357n3k6zr2m0gp3hcaj6jwacb-blas-3/lib -lblas -lgsl -lgslcblas -lm

ldd ./a.out

linux-vdso.so.1 (0x00007ffeccb02000)
libgsl.so.27 => /nix/store/0sf6i80ij0s1464qkh965z5piw90v32k-gsl-2.7.1/lib/libgsl.so.27 (0x00007f7513c00000)
libc.so.6 => /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libc.so.6 (0x00007f7513a17000)
libm.so.6 => /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib/libm.so.6 (0x00007f7513ef2000)
/lib64/ld-linux-x86-64.so.2 => /nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/lib64/ld-linux-x86-64.so.2 (0x00007f7513fd6000)

I’m not 100% sure what’s going as I’m not familiar with blas. I wonder if zig has internal blas implementation and simply does not implement that symbol …? Unlike your gcc result, the ldd does not seem to show libblas.so either.

EDIT: Using the Library — GSL 2.7 documentation
It might be opposite. The cblas might be bundled with gcc, which is the reason it gets linked there, but zig does not have one so you need to provide the cblas dependency somehow. Adding blas from nixpkgs to the buildInputs might work.

AFAICS, zig does not have an internal blas implementation.

It is already in buildInputs, see Zig cc dynamic C lib runtime error on NixOS - #14 by fleimgruber

The GSL documentation says the CBLAS implementation used is libgslblas.so which is present under …-blas-3/lib and is found with pkg-config, see my previous message.