The updates to the raylib build.zig are most often done by many different individuals, I think it would help if we discuss some things and track different goals for what should be possible to do.
I turn this post into a wiki entry so we can collect the goals here and discuss further ones via normal discussions.
Regarding 1. the change [build] remove examples/build.zig, incorporate into main build.zig · raysan5/raylib@ac17de5 · GitHub by @marler8997 changed the examples, which is fine but had a negative side effect, when I now try to build a raylib+emscripten project I get this error message:
thread 188284 panic: Emscripten building via Zig unsupported
/home/sze/.cache/zig/p/raylib-5.5.0-whq8uPuyNARF5oIcIW5D0fNHVw_Zg6dB6q0DhTSjKGr-/build.zig:507:9: 0x1582bd2 in addExamples__anon_205295 (build)
@panic("Emscripten building via Zig unsupported");
So I think we need to change this somehow so that the example build step doesn’t trigger its panic in an eager way, so that we can use the for emscripten built raylib library in an raylib-application build, without the example build steps causing trouble for us.
I think for 2. we maybe could add something like this:
- raylib.addIncludePath(dep.path("upstream/emscripten/cache/sysroot/include"));
+ const sysroot_include = dep.path("upstream/emscripten/cache/sysroot/include");
+ raylib.addIncludePath(sysroot_include);
+ raylib.installHeadersDirectory(sysroot_include, "sysroot_include", .{});
So that the application creating an emscripten build can add the sysroot include header files for its emscripten build.
Additionally I think it would make sense to expose the base path to the emcc (and other) commands as a named LazyPath. So that raylib application builds can easily use the existing raylib emscripten dependency to also do the linking for the application and possibly even use emrun to run the debug server.
My change just moved the examples, that @panic for emscripten already existed before it was moved.
Adding support for emscripten sounds fun. What’s the zig command you’re running to build/run an example via emscripten?
I think the difference is that before it was in a separate build.zig that wasn’t executed? (not completely sure)
Oh I see…yeah that should definitely be fixed then! What’s the zig build command to build raylib for emscripten?
For my zig15game I had something along the lines of:
zig build -Dtarget=wasm32-emscripten --sysroot "/home/sze/development/workspace/c/emsdk/upstream/emscripten"
But that is currently still using a local emscripten install, I think with the tweaks described in the comments above, it should be able to reuse the emscripten dependency of raylib (I had a poc but that needs to be restored after shuffling between different commits, but it was essentially those changes).
It looks like the build already automatically downloads the emscripten SDK? I just ran zig build -Dtarget=native-emscripten and it seemed to work?
I created a commit to address the build issue here: build.zig: fix building raylib for emscripten · marler8997/raylib@8d03636 · GitHub
Does it look good to you?
I was thinking that it could be something like that, thanks for adding it, looks good, will test later (have something else I have to do right now).
Cool I created a PR: build.zig: fix building raylib for emscripten by marler8997 · Pull Request #4910 · raysan5/raylib · GitHub
Would be cool to make the examples actually work though.
You mean emscripten build for examples?
So this How to compile raylib wasm game works now using your fix, using a local emscripten install.
Next I will try to do 2. so that it instead reuses the emscripten install from raylib.
If you want a different opinion, I don’t think emsdk/Emscripten should be depended on as a package in the first place, for multiple reasons:
emsdk install latest && emsdk activate latest will download files to the package directory, and emcc will modify and copy files to the Emscripten cache directory as needed. This means that if there are multiple concurrent build processes, they might trample on each other and produce nonsensical errors.EM_CONFIG environment variable, passing --cache or --em-config to emcc or specifying the option in ~/.emscripten. Assuming the headers are always located in emsdk_dep.path("upstream/emscripten/cache/sysroot/include") is wrong and will not work for users who have overridden the path in a different way (e.g. if they have already independently installed and set up the Emscripten toolchain globally).cache/sysroot/include only works out of the box because emsdk builds/installs the sysroot to the cache automatically. If the cache is empty, the Zig part of the build won’t find any headers and will fail until the user runs embuilder build sysroot (or an emcc command that does the same thing).I understand the convenience of depending on emsdk as a package, but it’s a flaky hack at best and the moment anything goes wrong it will be a very confusing experience for users that will be difficult for them to fix on their own (especially the kinds of inexperienced users that might want to use raylib).
I think the more sound approach is to instruct users to install Emscripten globally whichever way they prefer (emsdk is recommended, but you could also use a system package manager, or build the toolchain on your own) and then have them explicitly pass the path to the sysroot via --sysroot "$(em-config CACHE)/sysroot". It’s slightly more effort for the user invoking zig build, but this way there are fewer things that can go wrong because the user is in full control of Emscripten installation and the build script won’t need to make assumptions that won’t always hold.
I think you are conflating your personal preference for not managing emscripten as a zig dependency, with the hackiness of how it is currently used as a dependency (by raylib and other projects that have used that hacky solution).
I think nobody who looks at it finds it pretty or reliable in theory, I think the only reason why it is being used, is that it works surprisingly well in practice and solves an immediate need for enough people, so it keeps spreading. It provides a solution for a use case and happens to work well enough, often enough. (Yes I want better solutions, but I don’t think this means emscripten can’t be managed as a Zig dependency)
My Ideal solution would be that people can either use an immutable emscripten zig dependency or detect/provide an override to use some system dependency. I don’t see why any one of the two should be the only option.
From what I have heard about Zig’s buildsystem, it was supposed to give people the means to make and enable these choices, but I also think that some of those features aren’t implemented yet, things like version resolution/overrides, reproducible, isolated and sandboxed package builds, etc.
From a perspective of “what should be” a lot of your points make sense, but practically speaking I don’t think that we are there yet, for example if
then it should be a compile error, when they aren’t, but currently you still can do funky stuff, which maybe you shouldn’t do, but you can.
And these kind of hacks exist, because they bridge gaps for things, that don’t exist yet. I think a lot of game developers that have spent a lot of time learning and implementing all kinds of stuff, don’t want a workflow where they have to setup a whole bunch of things manually and have to learn about details that they shouldn’t have to know.
When you can simply zig build for desktop and even for other operating systems, but then for web you first have to understand a whole bunch of details to pick your preferred way of installing emscripten, that seems silly to me.
I think it is way better to have a suboptimal way that happens to work most of the time and then make it so that people can opt-in to something where they need to know more about the details.
And I think the hackiness is because emscripten is an awkward dependency, it seems too big and clunky, but once you have gotten it to give you what you want “make it compile for web”, you no longer want to spend time on making it better and would rather do something that feels rewarding, like adding content to the game, instead of trying to understand how emscripten works internally and how it could be packaged better.
Having good workflows matters and forcing people to make choices they don’t care about doesn’t help.
So I think what you describe should eventually be one of the things that people can do, but I also think there should be a default/just-compile-the-damn-thing option where people can be blissfully unaware of emscripten’s existence, until their hard drive contains 40 gigs of different emscripten versions and it causes them to clear that cache, or somebody got annoyed enough to improve the situation.
Personally I don’t think “install emscripten manually on your system” is a satisfying solution, but I do agree that it should be one of the supported options, at least when there is some versioning flexibility.
For example games that require fully deterministic simulation, would not want any system packages and instead build everything from immutable packages for the parts related to the deterministic simulation.
I think there are different workflows and use-cases, forcing one, over all the others, doesn’t work, the raylib’s build.zig manages to be useful for people, because people try to enable their use-cases without destroying other peoples existing use-cases.
Saying
doesn’t make much sense in the context of how raylib’s build.zig currently grows organically, because currently it is just that whatever happens to work for the people who update the build.zig, is supported and everything else might be, or not be, supported.
And this just happens to be the thing that raylib’s build.zig had already assumed.
The idea behind this topic was to get people talking, so that things can improve over time, through incremental improvements, but we can’t just completely change things and mess up all existing workflows, just because it doesn’t work perfectly every time.
Peoples use-cases should be kept intact and only broken when it really can’t be avoided.
With the disclosure that I’m not super familiar with Emscripten, the way I would expect this to work is you’d have a default build that uses a package, and then you’d use the system integration options if you want to use a system-wide installed emscripten. Does this sound like a good solution to you?
I will admit to having made some “works for me” changes (added an Android target) a couple weeks ago, definitely would be less stressful if I knew what not to do and what approaches are preferred.
This is no longer the recommended way to use the Emscripten SDK, the Emscripten team has gone to great lengths in the last couple of years to ensure that multiple installed Emscripten SDKs can work side by side without ‘polluting’ the global system state, and such ‘non-global’ installs (which was called ‘embedded installs’) are now the default (and I think the only supported way in the future).
IMHO for Zig projects, the best location for the Emscripten SDK would be the local Zig cache of the top-level project, that way each Zig project can have its own version requirement for the Emscripten toolchain (which is important, because breaking changes between Emscripten toolchain versions are not all that uncommon) - or alternatively I think the global Zig cache already also works since the Emscripten SDK is distributed via git, e.g. you get a unique commit hash for each new version - however one problem is that the emsdk is just a bunch of helper files to install a specific Emscripten toolchain version - and by default those toolchain files are installed in a subdirectory of the emsdk install location (that’s why it was called ‘embedded install’ before). So using the emsdk to install older toolchain version may bring back version conflicts if the emsdk directory is in the global Zig cache (or in any global location).
…I would rather see a common Zig build system strategy to integrate ‘external’ toolchains and build tooling (e.g. the requirements for using the Android, iOS or Emscripten SDKs overlap a lot, and all of those also overlap with the idea to move the Clang toolchain into a separate packege (e.g. the LLVM divorce) - and at least for Android and Emscripten there’s a lot more to consider than just calling the Zig build system with --sysroot (for instance the custom link-/post-link steps which on Emscripten produce the wasm+js+html output files, and on Android a signed APK bundle.
My main point is that the Emscripten toolchain as a host system dependency managed by the user will basically always work (and if it somehow doesn’t the user has the tools needed to resolve any issues). Fetching Emscripten as a dependency the way raylib does only works by happenstance when the system and/or cache is in a specific state. If a completely unrelated package that depends on emsdk can inadvertently and permanently (until the state of the cache is wiped clean or otherwise fixed) break the build for any raylib project that uses the same revision of emsdk, simply by running specific emcc commands from its build script, it’s not a very good or stable solution.
At the least, the core raylib maintainers who may be on the receiving end of issues created by users confused as for why their builds inexplicably fail should be informed of the problems with and flakiness of the current approach.
The only reason you can simply zig build for Windows, Linux and macOS is because Zig itself ships libcs for those platforms and has put great care and effort into making sure they work. If you want to compile for iOS, you need Xcode as an external dependency. If you want to compile for some game console, you need its proprietary SDK. You wouldn’t fetch and install Xcode or other similar SDKs via the Zig build system. Needing to manage Emscripten in a similar way shouldn’t be controversial.
Maybe you misunderstood what I meant by “global”, what I meant was that the user manages their Emscripten environment. The officially recommended workflow is to clone the emsdk repo and run the install/activate commands. After that, any time you need to use Emscripten you just run source path/to/emsdk_env.sh to set all environment variables and add emcc and all other tools to PATH for the current shell session (alternatively, you can configure your shell to do this automatically). Any build scripts or programs you run from that shell session can then obtain any information they need about the Emscripten environment or run any of the Emscripten tools.
Doing this you technically don’t even need to concern the user with passing --sysroot to zig build, you could instead run b.run(&.{ "em-config", "CACHE" }) from the build.zig itself and join the result with b.pathJoin(&.{ cache, "sysroot", "include" }) to get the system header search path. emcc will already be in PATH so you only need b.addSystemCommand(&.{"emcc"}) to invoke it.
FWIW in my ‘fips’ cmake wrapper (Introduction | Fips Docs) I have commands for setting up the Emscripten, WASI and Android SDKs as ‘project local installs’.
It’s only not done for iOS SDK because it cannot really be installed via the command line but is managed via Xcode.
IMHO if an SDK installation is ‘automatable’ it’s better if the build system takes care of the installation into a local place where it doesn’t collide with other projects, so that the user only needs to git-clone a project and run zig build (instead of having to follow a readme which explains how to setup build prerequisites, it’s already bad enough that this is needed on Linux for installing system library ‘dev-packages’ for GL, X11, Wayland, ALSA etc… (which isn’t specific to Zig of course, but is a Linux specific hassle).
In my opinion, SDKs, toolchains or custom build tools (like texture atlas generators) are also just build dependencies.