Hey everyone, first post here. And a big one, sorry if it feels unconventional, but this is the kind of post I like come across.
Also, I’m not as fluent as I would want to be in english, so sorry about that too.
I have what seems a rare use case, because I haven’t found many issues on gihtub revolving around the core problem I face.
Zig version is master, my host is x86_64-linux-gnu, and target is native.
I need to build a shared library, and then an executable that link it, without linking libc.
Not a big deal you might think, so let’s try it.
// root.zig
pub export fn add(a: i32, b: i32) i32 {
return a + b;
}
// main.zig
const std = @import("std");
extern fn add(i32, i32) i32;
pub fn main() !void {
const result = add(34, 35);
std.log.info("{d}", .{result});
}
// build.zig
const std = @import("std");
pub fn build(b: *std.Build) void {
const target = b.standardTargetOptions(.{});
const optimize = b.standardOptimizeOption(.{});
const lib_mod = b.createModule(.{
.root_source_file = b.path("src/root.zig"),
.target = target,
.optimize = optimize,
});
const lib = b.addSharedLibrary(.{
.name = "libz",
.root_module = lib_mod,
});
b.installArtifact(lib);
const exe_mod = b.createModule(.{
.root_source_file = b.path("src/main.zig"),
.target = target,
.optimize = optimize,
});
exe_mod.linkLibrary(lib);
const exe = b.addExecutable(.{
.name = "exez",
.root_module = exe_mod,
});
// I like to see what's going on, too bad the output is
// reported "like an error", but I can deal with it
b.verbose_link = true;
b.installArtifact(exe);
const run_cmd = b.addRunArtifact(exe);
run_cmd.step.dependOn(b.getInstallStep());
if (b.args) |args| {
run_cmd.addArgs(args);
}
const run_step = b.step("run", "Run the app");
run_step.dependOn(&run_cmd.step);
}
Pretty much standard. Let’s run zig build :
error: ld.lld: undefined reference: __tls_get_addr
Ok. As I have some background in compiling stuff, I know what’s going on: __tls_get_addr is usually provided by libc, and zig doesn’t provide an implementation.
Which, to be fair, is understandable at first sight.
But looking into the source code, I was amazed by all the provided utilities included to interpret ELF and prepare execution. I mean, look at lib/std/os/linux/pie.zig, lib/std/os/linux/tls.zig, or , lib/std/elf.zig. Very impressive.
So, after 3 days of reading the parts of std library and compiler_rt involved into those kind of things, not providing __tls_get_addr feels like a miss.
I’m a brutal programmer, so I will provide an implementation myself just to satisfy the linker:
// root.zig
pub export fn add(a: i32, b: i32) i32 {
return a + b;
}
pub export fn __tls_get_addr(offset: *anyopaque) usize {
_ = offset;
// let's see what happens
return 69;
}
zig build: no errors!
After a free shot of dopamine, it’s time to run the produced executable:
./zig-out/bin/exez:
[1] 13750 segmentation fault ./zig-out/bin/exez
Hmm… almost, but not today.
Debugging time:
* thread #1, name = 'exez', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x0)
frame #0: 0x0000000000000000
This is what I discovered after stepping from the start:
- program entry point is
_startinstart.zig:224 posixCallMainAndExitinstart.zig:224is calledstd.os.linux.tls.initStaticis called, to initialize the executable TLS area in case some threadlocal variables ares used somewhere in the codestd.os.linux.tls.prepareAreais called@memsetis called- program counter goes into the .plt section to call the memset builtin (provided by compiler_rt)
- and… segfault
And it makes sense, kind of. After inspecting the executable image, it seems nothing has been done for the relocations involved in loading a dynamically linked executable.
Like if /lib64/ld-linux.so.2 was not used at all. We can see from readelf -l ./zig-out/bin/exez that no INTERP header was set, and file ./zig-out/bin/exez report no interpreter.
Let’s call the executable through the dynamic linker directly, just to be sure:
/lib64/ld-linux-x86-64.so.2 ./zig-out/bin/exez:
info: 69
It works perfectly.
The linker call for the executable is (with args on different lines for clarity):
ld.lld
--error-limit=0
--entry _start
-z stack-size=16777216
--image-base=16777216
--eh-frame-hdr
-znow
-m elf_x86_64
-o /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/dcb863ec1f0c28a714f96a0d8f6186a3/exez
-rpath /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/ccbb04d27259d135001cdad08ee7e7a2
/home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/dcb863ec1f0c28a714f96a0d8f6186a3/exez.o
--as-needed /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/ccbb04d27259d135001cdad08ee7e7a2/liblibz.so
/home/tibbo/.cache/zig/o/b31258b48dfe6887846bbea61e2106e2/libcompiler_rt.a
No mention of --dynamic-linker /lib64/ld-linux-x86-64.so.2, so it’s understandable that no interpreter is set on the executable.
Let’s try to add it:
ld.lld
--error-limit=0
--entry _start
-z stack-size=16777216
--image-base=16777216
--eh-frame-hdr
-znow
-m elf_x86_64
--dynamic-linker /lib64/ld-linux-x86-64.so.2 // <= HERE
-o /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/dcb863ec1f0c28a714f96a0d8f6186a3/exez
-rpath /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/ccbb04d27259d135001cdad08ee7e7a2
/home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/dcb863ec1f0c28a714f96a0d8f6186a3/exez.o
--as-needed /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/ccbb04d27259d135001cdad08ee7e7a2/liblibz.so
/home/tibbo/.cache/zig/o/b31258b48dfe6887846bbea61e2106e2/libcompiler_rt.a
And then, /home/tibbo/Temp/ZigDynNoLibc/.zig-cache/o/dcb863ec1f0c28a714f96a0d8f6186a3/exez:
info: 69
Oh yeah, nice.
So it’s just a matter of how the args of the call to ld.ldd are computed. In src/link/elf.zig:1883, we see:
...
if (comp.config.link_libc) {
if (comp.libc_installation) |libc_installation| {
try argv.append("-L");
try argv.append(libc_installation.crt_dir.?);
}
if (have_dynamic_linker) {
if (target.dynamic_linker.get()) |dynamic_linker| {
try argv.append("-dynamic-linker");
try argv.append(dynamic_linker);
}
}
}
...
So no libc, no dynamic linker. Let’s try linking libc to check if this is true:
// build.zig
...
const exe_mod = b.createModule(.{
.root_source_file = b.path("src/main.zig"),
.target = target,
.optimize = optimize,
.link_libc = true, // <= HERE
});
...
zig build run:
info: 69
Ok, we found the cause: when not linking libc, no dynamic interpreter is set, so no relocations are made.
At this point, we could discuss why it is the case.
I thought about opening an issue, but to be honest I feel like the issue tracker has became noisy lately, with a bunch of people opening issues with zero investigation and a touch of “I demand my niche use case to be addressed immediately, or I will tweet this zig trend is just propaganda”.
I feel uncomfortable pinging some core contributor, potentially adding to the noise with what seems to be a use case nodbody cares about, just a few days before a release. Even if I think the debate about the rationale behind “no libc, no dynamic linker” is exactly the kind of thing I like in the chaos systems programming has become.
If by chance a core contributor or someone with good mastery of the zig source code see this post and can give a short explaination, I’ll just be happy with it.
And there is something else.
Remember the first segfault? We saw that entry point ends up in posixCallMainAndExit. There is something interesting in this function (start.zig:522):
...
// Apply the initial relocations as early as possible in the startup process. We cannot
// make calls yet on some architectures (e.g. MIPS) *because* they haven't been applied yet,
// so this must be fully inlined.
if (builtin.position_independent_executable) {
@call(.always_inline, std.os.linux.pie.relocate, .{phdrs}); // <= HERE
}
...
Wow, very cool. If the executable is PIE, zig will take care of the relocations itself. Again, very impressive.
So let’s remove the libc linking, and mark the executable as PIE:
// build.zig
...
const exe_mod = b.createModule(.{
.root_source_file = b.path("src/main.zig"),
.target = target,
.optimize = optimize,
// .link_libc = true,
});
exe_mod.linkLibrary(lib);
const exe = b.addExecutable(.{
.name = "exez",
.root_module = exe_mod,
});
exe.pie = true; // <= HERE
...
I will not tell the story of my journey seeking for complete explaination of the .pic and .pie usage through the source code, but it was wild.
Now, let’s run it. zig build run:
[1] 23395 segmentation fault ./zig-out/bin/exez
Too bad. But I’m curious, so let’s debug it:
* thread #1, name = 'exez', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0xbab46)
* frame #0: 0x00000000000bab46
frame #1: 0x00007ffff7f6b43f exez`os.linux.tls.prepareArea(area=(ptr = "", len = 48)) at tls.zig:462:5
frame #2: 0x00007ffff7f6989b exez`os.linux.tls.initStatic(phdrs=[]elf.Elf64_Phdr @ 0x00007fffffffd348) at tls.zig:528:33
frame #3: 0x00007ffff7f68788 exez`start.posixCallMainAndExit(argc_argv_ptr=0x00007fffffffdb30) at start.zig:547:40
frame #4: 0x00007ffff7f6844e exez`start._start at start.zig:271:5
It is not the same thing than the first segfault, when building without marking the executable as pie. This time, relocations has been resolved, but the program counter jumps to an unmapped address when calling @memset.
In fact, it seems relocations are only one part of the job, effectively mapping the library into the process memory must be done too, and must be done before.
I wonder if zig could be taking this responsability… Wait, what is this crazyness in test/standalone/load_dynamic_library/main.zig?
const std = @import("std");
pub fn main() !void {
var gpa: std.heap.GeneralPurposeAllocator(.{}) = .init;
defer _ = gpa.deinit();
const args = try std.process.argsAlloc(gpa.allocator());
defer std.process.argsFree(gpa.allocator(), args);
const dynlib_name = args[1];
var lib = try std.DynLib.open(dynlib_name);
defer lib.close();
const Add = *const fn (i32, i32) callconv(.C) i32;
const addFn = lib.lookup(Add, "add") orelse return error.SymbolNotFound;
const result = addFn(12, 34);
std.debug.assert(result == 46);
}
But does it works without linking libc? Let’s see the source:
// lib/std/dynamic_library.zig
const std = @import("std.zig");
const builtin = @import("builtin");
const mem = std.mem;
const testing = std.testing;
const elf = std.elf;
const windows = std.os.windows;
const native_os = builtin.os.tag;
const posix = std.posix;
/// Cross-platform dynamic library loading and symbol lookup.
/// Platform-specific functionality is available through the `inner` field.
pub const DynLib = struct {
const InnerType = switch (native_os) {
.linux => if (!builtin.link_libc or builtin.abi == .musl and builtin.link_mode == .static)
ElfDynLib
else
DlDynLib,
.windows => WindowsDynLib,
.macos, .tvos, .watchos, .ios, .visionos, .freebsd, .netbsd, .openbsd, .dragonfly, .solaris, .illumos => DlDynLib,
else => struct {
const open = @compileError("unsupported platform");
const openZ = @compileError("unsupported platform");
},
};
...
pub const ElfDynLib = struct {
strings: [*:0]u8,
syms: [*]elf.Sym,
hashtab: [*]posix.Elf_Symndx,
versym: ?[*]elf.Versym,
verdef: ?*elf.Verdef,
memory: []align(mem.page_size) u8,
...
/// Trusts the file. Malicious file will be able to execute arbitrary code.
pub fn open(path: []const u8) Error!ElfDynLib {
const fd = try resolveFromName(path);
defer posix.close(fd);
const file: std.fs.File = .{ .handle = fd };
const stat = try file.stat();
const size = std.math.cast(usize, stat.size) orelse return error.FileTooBig;
// This one is to read the ELF info. We do more mmapping later
// corresponding to the actual LOAD sections.
const file_bytes = try posix.mmap(
null,
mem.alignForward(usize, size, mem.page_size),
posix.PROT.READ,
.{ .TYPE = .PRIVATE },
fd,
0,
);
defer posix.munmap(file_bytes);
...
Mind blowing… zig provides a way to map a dynamic library without the help of libc (I didn’t test if it’s really working though, but the fact that a test exists is a good sign).
It seems to me that most of the work has already been done to be able to do loading/mapping AND relocations on linux.
And judging by the existence of test/cases/pic_linux.zig:
const std = @import("std");
// Eventually, this test should be made to work without libc by providing our
// own `__tls_get_addr` implementation. powerpcle-linux should be added to the
// target list here when that happens.
//
// https://github.com/ziglang/zig/issues/20625
pub fn main() void {}
// run
// backend=stage2,llvm
// target=arm-linux,armeb-linux,thumb-linux,thumbeb-linux,aarch64-linux,aarch64_be-linux,loongarch64-linux,mips-linux,mipsel-linux,mips64-linux,mips64el-linux,powerpc-linux,powerpc64-linux,powerpc64le-linux,riscv32-linux,riscv64-linux,s390x-linux,x86-linux,x86_64-linux
// pic=true
// link_libc=true
I have good hope. I must confess, I’m already working on it, but as I said earlier, I feel adding a PR from a random dev with very specific use cases to the issue tracker is not a thing I want to do, at least not before I have a complete understanding of the side effects of my edits. Zig is a big project, I cannot torture it like it was just my personal window manager or something.
Let’s recap (this concerns linux):
- it’s not currently possible to link a shared library while not linking libc for two reasons:
__tls_get_addris needed- workaround: provide an implementation yourself, and export it
- solution I’m working on these days: implement it for linux in compiler_rt
- question: is this the right thing to do?
- note:
__tls_get_addris needed because std lib use threadlocal vars; if compiling with--release=fastyou may not encounter this problem (you will if you make use of anything that callstd.Thread.getCurrentId(), likestd.Thread.Mutex.lock()in debug mode, because it is caching the thread id in a threadlocal var)
- if libc is not linked, no dynamic linker will be used, and no loading/mapping nor relocations will be made before
main- question: why is that? Am I missing something?
- workaround: I didn’t find any way to force the linker call to pass
-dynamic-linkerwithout modifying the zig source code - solution I’m working on these days: allow dynamic linker even without libc, changes mainly happens in
src/link/Elf.zig- question: is it a good idea?
- currently, if a PIE executable linked to a shared library is produced without linking libc (which is almost as nice as static linking):
- no dynamic linker will be used as per the above (good, one less dependency on the linker path)
- zig can handle relocations, but does not load/map linked dynamic libraries, so execution will fail
- in the hypothesis the loading/mapping become effective, I think distributing updates of a complex program consisting of a PIE libc-free dynamic-linker-free executable using dynamic libraries to organize its components can become a cool thing to do, and is in fact the motivation behind my use case
- question: does someone have a good doc that is not ChatGPT diarrhea explaining precisely how and where (which address) is a dyanmic library mapped into process memory on linux?
Conclusion: in my ideal world (for linux), __tls_get_addr is provided by zig, and the start logic in posixCallMainAndExit handles all the dynamic linker tasks !