ELF Linker Improvements
Author: Matthew Lugg
37 Likes
That’s pretty cool
1 Like
Missed opportunity to flex your Tetris skills
12 Likes
Would be cool for live-reloading demo!
2 Likes
Clipping out of bounds wasn’t enough?
3 Likes
I was wondering why incremental compilation didn’t work for me when linking C libraries. The days of waiting 600ms for a rebuild like some peasant are numbered!
2 Likes
they sadly don’t exist, I am shockingly bad at Tetris
1 Like
Amusingly, your clean build time is on the same order of magnitude as the incremental rebuild time for some codebases 
only big ones though, don’t worry. This happens because there’s a small amount of overhead on each rebuild which is still proportional to the total size of your codebase—so, incremental rebuilds of the full Zig compiler do currently take 200-300ms each on my laptop. Of course, I’m planning to improve on this in the future—I’d like rebuilds of the compiler to be sub-100ms at least, that’s the point where it starts to feel like magic to me!
But yeah, hopefully these improvements will unblock your use case 
let me know how that goes when you try it out!
9 Likes
First of all: very nice!
Secondly:
Why only x86? I always thought the ELF file layout stays the same, independently of target architecture. You just put a different object file in there.
The linker itself isn’t x86_64-specific, but right now, x86_64 is the only target for which we have a stable self-hosted code generation backend. Other architectures still go through LLVM at the moment, so you wouldn’t get any benefit from this linker. (I don’t think its non-incremental performance is anything revolutionary—it might even be worse than the old linker right now, it’s not a use case I’ve been focusing on too heavily. I will of course still benchmark it at some point and make improvements if necessary—we want this to be a good non-incremental linker too!)
Also, the non-x86_64 code-paths in the linker are mostly unimplemented right now—however they shouldn’t be too difficult to implement when I get around to it, I just haven’t bothered to look into other architectures’ target-specific details yet (relocation types, PLT layout, etc).
8 Likes
Does the new linker solve the issue of fatal linker error: unhandled relocation type R_X86_64_PC64 at offset 0x1c I keep seeing on (arch) Linux? I was working around this with -Doptimize=ReleaseSafe, but last Friday even that failed – I assume the optimize option was not being correctly passed down the build pipeline to a vendored C library I am linking against.
2 Likes
Huh, looking at the linker source code, it sounds like you have a… quite strange object file being linked in. Seems like it must have a R_X86_64_PC64 relocation in a non-alloc section, which is pretty nonsensical. I’m not quite sure how the object in question wants that relocation to be resolved… I think it actually would compile okay on the new linker right now, but only because the new linker is missing various error checks.
If you’ve not already, would you mind opening an issue about this? Even when we eventually replace the old linker implementation, I’d like this quirk to be tracked so that I can look into it and find out what the linker is actually meant to do here.
6 Likes
Considering how long it takes to change one value in one header in a large C++ codebase believe me even < 1s for a 1M loc codebase is already absolutely mental by multiple orders of magnitude.
3 Likes
Have you considered that the c++ tooling is absolutely mentally slow instead :3
3 Likes
I am pretty sure this issue already covers it. I added a comment noting that this is now also happening for -Doptimize=ReleaseSafe.
Biggest thing for me is that typing zig build --watch -fincremental -fnew-linker --error-style verbose_clear is getting a bit long to type. Going to need to setup some aliases.
Some tips:
-fnew-linker is redundant in your command
ZIG_BUILD_ERROR_STYLE env var exists
-fincremental will become default eventually
Ctrl+R exists
16 Likes
# Zig Build Watch Incremental
alias zbwi='zig build --watch -fincremental --error-style verbose_clear'
3 Likes