Producing relocatable ELF with Zig build system

Help
1

Hi there! I’m trying to use the Zig build system and toolchain to replace my embedded/GameCube C++ project’s build tooling entirely. I’m nearly there, but I need to ultimately build a relocatable ELF library, so that it may be converted to GameCube’s native object format by another tool.

Basically I’d like to run:

zig ld.lld -T ogc.ld -r -e _prolog -u _prolog -u _epilog -u _unresolved -g --gc-sections zig-out/lib/libmod.a -o zig-out/lib/libmod.elf

Note the -r - I can’t seem to find a way to link a relocatable ELF via the build system APIs, for one. I wouldn’t mind needing to shell out to zig ld.lld in a run step explicitly if required, but I’m also not sure how to determine the path of the zig executable to use e.g. the one used to invoke zig build.

Any suggestions would be appreciated, thanks!

2

I don’t know too much about the linker command you use, but you can use b.addSystemCommand with b.graph.zig_exe.

3

It sounds like you’re looking for a Compile step which emits an object file (i.e. a relocatable) rather than an executable or library. Take a look at std.Build.addObject, it should do what you need.

(It’s recommended to avoid using zig ld.lld directly fwiw—that subcommand is an implementation detail and could disappear at any time!)

1 Like
4

Thanks for this, I still get objects/libraries/shared libraries mixed up more than I should.

std.Build.addObject() seems right, can/should I pass the remaining -e, -u, and --gc-sections flags?

5 
obj.link_gc_sections = true;
obj.entry = .{ .symbol_name = "_prolog" };

Might need a compiler enhancement for the -u args. Just so I understand what’s the problem if you leave those off?

Also may I see ogc.ld?

Thanks for bringing this up. AFAIK you’re the first person to try to use Zig for GameCube and actually bothering to share the problems you ran into.

2 Likes
6

Might need a compiler enhancement for the -u args.

Nah, we already expose that functionality too. Here’s the std.Build snippet:

obj.forceUndefinedSymbol("_prolog");
obj.forceUndefinedSymbol("_epilog");
obj.forceUndefinedSymbol("_unresolved");
1 Like
7 
  --force_undefined [name]       Specify the symbol must be defined for the link to succeed

Bit of an odd name and description there, isn’t it?

  -u SYMBOL, --undefined SYMBOL
                              Start with undefined reference to SYMBOL

ld --help not much better…

8

It’s an accurate description (it marks the symbol as initially undefined which means the linker is required to find a definition for the link to succeed—this is important because it means the linker will include objects from static libraries if the objects define those symbols), but I agree a confusing one. I think linker CLI is something Zig can easily improve on compared to traditional linker implementations. A better name for this flag would probably be something like --need-definition for instance.

1 Like
9

Sure! Here’s a buildable subset of the full project if you’re curious: https://codeberg.org/ComplexPlane/wsmod-sample

https://codeberg.org/ComplexPlane/wsmod-sample/src/branch/addLibrary/ogc.ld

ogc.ld is originally from the devkitpro-ppc toolchain, I appended the constructor/destructor symbols to the end.

Anyways, this config does seem to build+link correctly, but I did notice it bloats the final binary by ~10% compared to addLibrary() + zig ld.lld, and maybe ~20% compared to devkitppc-gcc:

mod.link_gc_sections = true;
mod.link_function_sections = true;
mod.link_data_sections = true;
mod.entry = .{ .symbol_name = "_prolog" };
mod.forceUndefinedSymbol("_prolog");
mod.forceUndefinedSymbol("_epilog");
mod.forceUndefinedSymbol("_unresolved");
mod.setLinkerScript(b.path("ogc.ld"));

You can verify this yourself by comparing zig build && ./todo.sh on the main branch vs. addLibrary branch.

10

Pleased to hear you got it working!

I did notice it bloats the final binary by ~10% compared to addLibrary() + zig ld.lld, and maybe ~20% compared to devkitppc-gcc

Hm, and I assume this is a use case where binary size is quite important? I’m not immediately sure why compiling to a static library and then using zig ld.lld affects binary size compared to the addObjecty approach, but unfortunately, this is almost certainly an issue in LLD rather than anything we could realistically improve on our end. If this particular size regression is a blocking issue for you, then the only thing I can think of is that you could experiment with using addLibrary and then separately linking that into an addObject—I have no idea why that would affect anything, but it’s closer to the zig ld.lld invocation you were comparing to. You could also try passing --verbose-link to zig build, which should make the compiler dump out the underlying zig ld.lld commands it’s invoking—not sure how much that’ll help you, but perhaps it’ll come in handy.

11

Also glad to see this all working, really appreciate all the help!

So I did notice that zig build --verbose-link passes -float-abi=soft and -O2 instead of -mhard-float and -Os - not that I’d expect these to affect linking, but I wonder if they’re incorrectly being used during compilation too.

(zig build --verbose weirdly shows “failed command” but seems to have worked fine…)

 zig build --verbose-link
zig ld -dynamic -platform_version macos 26.5.1 26.4 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.4.sdk -e _main -o .zig-cache/o/d19f15731e1388816745b687a010cfb6/build .zig-cache/o/d19f15731e1388816745b687a010cfb6/build_zcu.o -lSystem /Users/alex/.cache/zig/o/f50cc1f46f1960cc57a6c80e296eac2f/libcompiler_rt.a
install
└─ install generated to libmod.elf
   └─ compile obj mod ReleaseSmall powerpc-other-eabi failure
error: ld.lld -r --error-limit=0 -mllvm -float-abi=soft -O2 --entry _prolog -u _prolog -u _epilog -u _unresolved --build-id=none --image-base=65536 -T /Users/alex/Documents/repos/2022/wsmod-sample/ogc.ld --gc-sections --eh-frame-hdr -znow -m elf32ppc -static -o .zig-cache/o/a8b9d6671bd7ee510b7e6a374e82d40b/mod.o .zig-cache/o/7a5d663c693100e9d3a758740a261fee/modlink.o .zig-cache/o/e6f7f2e028fa99dc34bd50d737cbe528/assembly.o .zig-cache/o/de289eb1c0a67550e1186a5008e0b005/gameheaps.o .zig-cache/o/3a5e4dcf3c7b382c4e5c0fd28fbbdbec/logging.o .zig-cache/o/12fee53359ae7d5254f643489b36cf7a/version.o .zig-cache/o/d051f08fecf8de6e84281c8faaaacfbf/patch.o .zig-cache/o/929f445d2d5b02fcde3ed73960ad83f5/sync_texture_scroll.o .zig-cache/o/208aab5b4dfd578b2136991d4ba02e1b/heap.o .zig-cache/o/af38b7f6ea7ef21a6a5b90ca8695cc43/arena.o .zig-cache/o/2c9add6860456975d2ed2835baeaa056/scratch.o .zig-cache/o/fb102c3d56c6badb56a550a078ddb3d5/debugslider.o .zig-cache/o/0059a2ecc2f8fc6ed52cdfc7b2a8dd05/config.o .zig-cache/o/6cd32138ba5d3bb6f919124f5d0536df/world_timer.o .zig-cache/o/2e943013b7365ed6591cf8caa7b08a17/relutil.o .zig-cache/o/26150373e01b3edf6f7806e927a466d6/cxx.o .zig-cache/o/818681264e01cad39ac8963c98859758/pad.o .zig-cache/o/92553dba32e5b539120d93a51a4f0051/relpatches.o .zig-cache/o/9064e8b2f9a1afd8e06af346786733b9/draw.o .zig-cache/o/2a1551fd327daa142038e19461214e8d/mem.o .zig-cache/o/3928684203573cc653cee4a726bb3ac0/rel.o .zig-cache/o/ae54521d901c9778cb337bab3fa4f31d/main.o .zig-cache/o/ece87df31ae19a77471ac25f20a312d1/sj.o .zig-cache/o/8d0bd49d23d2291e8a01ad037d9932ef/segmented_beaten_bar.o .zig-cache/o/ecc5e21ce3fb9ca5d7ea2698d4f3ca10/main_loop_assembly.o .zig-cache/o/d7abca21f19775e4b3720a93d5a8b589/story_mode_char_select.o .zig-cache/o/92696e5813770facfdb2679336d52d99/theme_id_per_stage.o .zig-cache/o/fd40e18ad9228c62028f97da6d56abce/rain_ripple_fix.o .zig-cache/o/9bca78f24163314691900731fbd69510/texscroll_hook_asm.o .zig-cache/o/2ea74c8bb00bc4e17e8b03b25d2dd038/stobj_reflection_fix.o .zig-cache/o/67f4feb0a2fa595dccb5a65b826edb5f/full_debug_text_color.o .zig-cache/o/b7f2b9d97d0abddd60d5981f6685012c/story_mode_music_fix.o .zig-cache/o/0fdaaa9e769b9bd0fbb99a46e36b3b79/music_id_per_stage.o

failed command: /opt/homebrew/Cellar/zig/0.16.0_1/bin/zig build-obj -fentry=_prolog --force_undefined _prolog --force_undefined _epilog --force_undefined _unresolved -cflags -std=c++20 -Wno-write-strings -Wno-address-of-packed-member -fno-exceptions -fno-rtti -g -Wall -mcpu=750 -mhard-float -- -x c++ /Users/alex/Documents/repos/2022/wsmod-sample/src/modlink.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/gameheaps.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/logging.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/version.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/patch.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/sync_texture_scroll.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/heap.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/arena.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/scratch.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/debugslider.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/config.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/world_timer.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/relutil.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/cxx.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/pad.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/relpatches.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/draw.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/mem.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/rel.cpp /Users/alex/Documents/repos/2022/wsmod-sample/src/main.cpp -x none -cflags -std=c99 -g -Wall -mcpu=750 -mhard-float -- -x c /Users/alex/Documents/repos/2022/wsmod-sample/3rdparty/sj.c -x none -cflags -- /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/segmented_beaten_bar.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/main_loop_assembly.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/story_mode_char_select.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/theme_id_per_stage.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/rain_ripple_fix.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/texscroll_hook_asm.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/stobj_reflection_fix.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/full_debug_text_color.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/story_mode_music_fix.s /Users/alex/Documents/repos/2022/wsmod-sample/src/assembly/music_id_per_stage.s -fno-strip -OReleaseSmall -target powerpc-other-eabi -mcpu baseline -I /Users/alex/Documents/repos/2022/wsmod-sample/src -isystem /Users/alex/Documents/repos/2022/wsmod-sample/3rdparty -Mroot --verbose-link -ffunction-sections -fdata-sections --gc-sections --cache-dir .zig-cache --global-cache-dir /Users/alex/.cache/zig --name mod --script /Users/alex/Documents/repos/2022/wsmod-sample/ogc.ld --zig-lib-dir /opt/homebrew/Cellar/zig/0.16.0_1/lib/zig/ --listen=-

install
└─ install elf2rel
   └─ compile exe elf2rel ReleaseFast native failure
error: zig ld -dynamic -platform_version macos 26.5.1 26.4 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX26.4.sdk -dead_strip -e _main -o .zig-cache/o/252e9ee0f8764ad8cd29f0d8d75c22bb/elf2rel .zig-cache/o/72c5a6f3b66cb84876b58ad23d7a51df/elf2rel.o /Users/alex/.cache/zig/o/9ae8e23d14c79b40d124c7a78d0a099f/libc++abi.a /Users/alex/.cache/zig/o/7278820ab2c509d6d662ea5e43f5c535/libc++.a -lSystem /Users/alex/.cache/zig/o/f50cc1f46f1960cc57a6c80e296eac2f/libcompiler_rt.a

failed command: /opt/homebrew/Cellar/zig/0.16.0_1/bin/zig build-exe -cflags -std=c++20 -- /Users/alex/Documents/repos/2022/wsmod-sample/3rdparty/elf2rel/elf2rel.cpp -OReleaseFast -I /Users/alex/Documents/repos/2022/wsmod-sample/3rdparty/elf2rel -Mroot -lc++ --verbose-link --cache-dir .zig-cache --global-cache-dir /Users/alex/.cache/zig --name elf2rel --zig-lib-dir /opt/homebrew/Cellar/zig/0.16.0_1/lib/zig/ --listen=-

TBF this project does have some pretty silly size requirements - we’re injecting code into the middle of some free space in a running existing GameCube game and only have ~150KB for multiple game mod binaries + savestate data.

12

I imagine using software floats instead of hardware will include the floating point emulation code. Dumping the binary will tell you more about what’s exactly in there taking space.

13

I know I’ve veered off topic at this point but just wanted to follow up the size discussion with: it appears that clang, even with -Oz, just generally produces larger codegen than powerpc-eabi-gcc:

gcc:

80009480 <sj_iter_object>:
80009480:    94 21 ff d0     stwu    r1,-48(r1)
80009484:    7c 08 02 a6     mflr    r0
80009488:    80 84 00 0c     lwz     r4,12(r4)
8000948c:    bf a1 00 24     stmw    r29,36(r1)
80009490:    7c 7d 1b 78     mr      r29,r3
80009494:    7c be 2b 78     mr      r30,r5
80009498:    90 01 00 34     stw     r0,52(r1)
8000949c:    7c df 33 78     mr      r31,r6
800094a0:    48 00 00 01     bl      800094a0 <sj_iter_object+0x20>
800094a4:    7f a4 eb 78     mr      r4,r29
800094a8:    38 61 00 10     addi    r3,r1,16
800094ac:    48 00 00 01     bl      800094ac <sj_iter_object+0x2c>
800094b0:    81 21 00 10     lwz     r9,16(r1)
800094b4:    91 3e 00 00     stw     r9,0(r30)
800094b8:    28 09 00 01     cmplwi  r9,1
800094bc:    81 41 00 14     lwz     r10,20(r1)
800094c0:    91 5e 00 04     stw     r10,4(r30)
800094c4:    81 41 00 18     lwz     r10,24(r1)
800094c8:    91 5e 00 08     stw     r10,8(r30)
800094cc:    81 41 00 1c     lwz     r10,28(r1)
800094d0:    91 5e 00 0c     stw     r10,12(r30)
800094d4:    40 81 00 4c     ble     80009520 <sj_iter_object+0xa0>
800094d8:    38 61 00 10     addi    r3,r1,16
800094dc:    7f a4 eb 78     mr      r4,r29
800094e0:    48 00 00 01     bl      800094e0 <sj_iter_object+0x60>
800094e4:    81 21 00 10     lwz     r9,16(r1)
800094e8:    91 3f 00 00     stw     r9,0(r31)
800094ec:    2c 09 00 01     cmpwi   r9,1
800094f0:    30 69 ff ff     addic   r3,r9,-1
800094f4:    81 41 00 14     lwz     r10,20(r1)
800094f8:    7c 63 49 10     subfe   r3,r3,r9
800094fc:    91 5f 00 04     stw     r10,4(r31)
80009500:    81 41 00 18     lwz     r10,24(r1)
80009504:    91 5f 00 08     stw     r10,8(r31)
80009508:    81 41 00 1c     lwz     r10,28(r1)
8000950c:    91 5f 00 0c     stw     r10,12(r31)
80009510:    40 a2 00 14     bne     80009524 <sj_iter_object+0xa4>
80009514:    3d 20 00 00     lis     r9,0
80009518:    39 29 00 00     addi    r9,r9,0
8000951c:    91 3d 00 10     stw     r9,16(r29)
80009520:    38 60 00 00     li      r3,0
80009524:    39 61 00 30     addi    r11,r1,48
80009528:    54 63 07 fe     clrlwi  r3,r3,31
8000952c:    48 00 00 00     b       8000952c <sj_iter_object+0xac>

clang:

000091f4 <sj_iter_object>:
    91f4:    7c 08 02 a6     mflr    r0
    91f8:    94 21 ff d0     stwu    r1,-48(r1)
    91fc:    90 01 00 34     stw     r0,52(r1)
    9200:    80 84 00 0c     lwz     r4,12(r4)
    9204:    93 81 00 20     stw     r28,32(r1)
    9208:    7c bc 2b 78     mr      r28,r5
    920c:    93 a1 00 24     stw     r29,36(r1)
    9210:    7c dd 33 78     mr      r29,r6
    9214:    93 c1 00 28     stw     r30,40(r1)
    9218:    7c 7e 1b 78     mr      r30,r3
    921c:    48 00 00 01     bl      921c <sj_iter_object+0x28>
    9220:    38 61 00 10     addi    r3,r1,16
    9224:    7f c4 f3 78     mr      r4,r30
    9228:    48 00 00 01     bl      9228 <sj_iter_object+0x34>
    922c:    80 a1 00 10     lwz     r5,16(r1)
    9230:    80 61 00 1c     lwz     r3,28(r1)
    9234:    90 bc 00 00     stw     r5,0(r28)
    9238:    80 81 00 18     lwz     r4,24(r1)
    923c:    90 7c 00 0c     stw     r3,12(r28)
    9240:    80 61 00 14     lwz     r3,20(r1)
    9244:    80 bc 00 00     lwz     r5,0(r28)
    9248:    90 9c 00 08     stw     r4,8(r28)
    924c:    90 7c 00 04     stw     r3,4(r28)
    9250:    28 05 00 02     cmplwi  r5,2
    9254:    3b 80 00 00     li      r28,0
    9258:    41 80 00 64     blt     92bc <sj_iter_object+0xc8>
    925c:    38 61 00 10     addi    r3,r1,16
    9260:    7f c4 f3 78     mr      r4,r30
    9264:    48 00 00 01     bl      9264 <sj_iter_object+0x70>
    9268:    80 a1 00 10     lwz     r5,16(r1)
    926c:    80 61 00 1c     lwz     r3,28(r1)
    9270:    90 bd 00 00     stw     r5,0(r29)
    9274:    90 7d 00 0c     stw     r3,12(r29)
    9278:    80 7d 00 00     lwz     r3,0(r29)
    927c:    80 81 00 18     lwz     r4,24(r1)
    9280:    80 a1 00 14     lwz     r5,20(r1)
    9284:    28 03 00 00     cmplwi  r3,0
    9288:    90 9d 00 08     stw     r4,8(r29)
    928c:    90 bd 00 04     stw     r5,4(r29)
    9290:    41 82 00 20     beq     92b0 <sj_iter_object+0xbc>
    9294:    28 03 00 01     cmplwi  r3,1
    9298:    40 82 00 20     bne     92b8 <sj_iter_object+0xc4>
    929c:    3c 60 00 00     lis     r3,0
    92a0:    38 63 00 00     addi    r3,r3,0
    92a4:    38 63 00 51     addi    r3,r3,81
    92a8:    90 7e 00 10     stw     r3,16(r30)
    92ac:    48 00 00 10     b       92bc <sj_iter_object+0xc8>
    92b0:    7c 7c 1b 78     mr      r28,r3
    92b4:    48 00 00 08     b       92bc <sj_iter_object+0xc8>
    92b8:    3b 80 00 01     li      r28,1
    92bc:    57 83 07 fe     clrlwi  r3,r28,31
    92c0:    83 c1 00 28     lwz     r30,40(r1)
    92c4:    83 a1 00 24     lwz     r29,36(r1)
    92c8:    83 81 00 20     lwz     r28,32(r1)
    92cc:    80 01 00 34     lwz     r0,52(r1)
    92d0:    38 21 00 30     addi    r1,r1,48
    92d4:    7c 08 03 a6     mtlr    r0
    92d8:    4e 80 00 20     blr

Seems like a decent amount of the increase is clang inlining full function prologs/epilogs instead of using the setgpr/restgpr jumps of gcc, and not using stuff like stmw.

Final size diff is 44K vs 55K, which isn’t abysmal but enough to be a deciding factor for me. Oh well :frowning: