Why sandbox the build runner (wasm)?

Explain
1
  1. Why is wasm sandboxing the zig build system being considered?
  2. What are the disadvantages to wasm sandboxing?
  3. Why choose wasm? Are there alternatives?
  4. Does this mean ZSF will create their own wasm interpreter / emulator?
  5. When / how will network access be allowed during a build?
  6. Will systemCommand go away or still be allowed?
  7. Where do you draw the line between what runs on wasm and what runs native? I can imagine a native compiler will be faster than one running on top of wasm?
2

It largely depends on how it is implemented. And Iā€™m not sure what that will be. But here are my thoughts anyway.

Security can not be a big motivation, a build system needs to be very powerful and most of that power comes from being able to run tools/scripts. The ability to do so defeats the security benefits of Wasm sandboxing.

It could/probably will restrict what the build.zig will be able to do directly, forcing you to outsource certain things to a run artifact/command. This is already preferred, but it is very easy to not do outsource it, which immediately makes it quite hard to integrate with caching. I am mostly thinking of file system and network access.

Since it would be a rewrite of the build system, it is an opportunity to do other changes.

To directly respond to the questions:

The only reason I can think of is the restriction I mentioned.

The disadvantage is that it will introduce a dependency on a wasm emulator. I can see them eventually having their own, but it will likely be an existing one at first.

Their are alternatives to wasm, but none are as well-supported in the various senses of ā€˜supportā€™. I suppose JVM is an exception to that, but I donā€™t think that would seriously be considered.

See what I said about restrictions and security above.

I do not expect them to run the compiler in wasm too, I think that would be quite arbitrary. The compiler is already run as a separate process(es), the build system just constructs the commands to it and other things.

4 Likes
3

I think running in wasm/sandboxed mode could be made optional, I think the main benefit of having a sandboxed build mode would be for projects that run their end product in wasm too. (Another benefit might be for reproducible builds)

I havenā€™t investigated that much, but I think there are increasingly more providers of cloud services that want to switch to some sort of wasm/wasix as an alternative to things like docker, so those mostly have an interested in being able to run some untrusted code that somehow customizes what their service is doing. And if zig has a sandboxed build mode they can build the code for you instead of you uploading the wasm artifact (This doesnā€™t seem like a very important distinction to me though? Maybe it is if the cloud provider needs to add its own internal code to the thing they end up deploying on your behalf, but I imagine they just could patch/modify the wasm artifact?)

Similar I think it could be useful for any kind of tool that wants to run arbitrary untrusted plugin code as wasm/wasix plugins, so possibly editors, graphics, modeling/sculpting, messaging applications, or maybe even game engines. I think those would likely have two kinds of plugins where one runs native code and the other sandboxed code, the former needs to be vetted, the latter either uses the APIs correctly or it doesnā€™t (and thus is a bit simpler to allow users to do whatever they want).

I think some people have hope that wasm could be an alternative to more bloaty virtualization/sandboxing techniques, but I am not familiar enough with them to know whether that will actually work out. I definitely would love if I could just create a single wasm thing and then easily deploy that thing so that it works everywhere, instead of having to create an individualized thing for every annoying walled garden out there (especially for games which usually donā€™t care about target platform ui conventions). (And once you have that, I could imagine wanting to run some user supplied code on servers etc. to let them customize stuff)

So overall I think my preferred way would be if you could run the buildsystem in sandboxed and native mode with most of the code and apis staying the same, where maybe the native mode could allow some things that are more complicated to do in sandboxed mode, in a more direct way.
But I am not sure whether this is what the core team envisions.

3 Likes
4

Hereā€™s my understanding based on the discussions Iā€™ve read in the past:

The Zig build process is run in two phases. The first phase (ā€œconfigureā€) executes the build function in your build.zig to construct the build graph (i.e. available build steps and their dependencies in terms of other steps). The second phase (ā€œmakeā€) takes the build graph and actually executes the steps using their make functions (e.g. compiling and linking an executable).

In the original issue for sandboxing, itā€™s clear that the Wasm sandboxing being proposed is for the configure phase, not the make phase (although the make phase would be subject to some sort of controllable restrictions on what it can do). So, as for your questions:

  1. Because, as it is today, even running zig build --help can execute arbitrary Zig code on your computer, since it runs the configure phase of the build to collect available steps (and not only the Zig code of the project you have checked out, but that of any dependencies as well).

  2. One disadvantage is that it will remove the ability to create custom build steps (where you implement your own make function). However, I personally have never run into a situation where a custom build step was needed, rather than addRunArtifact or addSystemCommand. Also, if this sandboxing is applied before the self-hosted Wasm backend is fully ready, it means regressing to using LLVM to compile build.zig again, which is slower.

  3. Wasm is an existing standard (not reinventing the wheel) and is relatively straightforward. Also, Zig already uses Wasm as part of the bootstrapping process (stage1.wasm).

  4. Presumably, yes. Zig already has a Wasm to C transpiler written in C which is used during the bootstrap process, and at least one Wasm runtime in Zig already exists, which could possibly be used for inspiration or as a base. (Edit: and apparently Andrew has also written his own already: GitHub - andrewrk/zig-wasi: Minimal WASI interpreter)

  5. Network access would certainly be possible during the make phase of the build, as part of a run artifact or system command step. If thereā€™s a real use-case for network access during the configure phase (aside from dependency fetching, which is handled by the compiler already), I assume it could be enabled through some flag allowing it via WASI.

  6. I havenā€™t seen anything indicating that any existing build steps (aside from custom steps, as noted above) would be unavailable. System commands are important for many builds, so I seriously doubt they would be removed; however, the Wasm sandbox would prevent executing system commands during the configure phase.

  7. As noted above, only the configure phase of the build is planned to be sandboxed. The make phase would still run natively, so in particular the Zig compiler would still be running natively at full performance.

11 Likes
5

It doesnā€™t have to. A sandboxed configuration stage could have a capabilities architecture, where any custom build steps have to be configured: with a command-line flag, if run directly, or as a build option if run as an import.

Not clear if thatā€™s even necessary, I havenā€™t once wanted to roll my own build step, and Iā€™ve gotten pretty weird with the build system by now. The options we have areā€¦ less than perfectly discoverable, I would go so far as to say, underdocumented, but they cover the bases quite well.

But say it was worthwhile, Wasm (WASI specifically) is designed with this kind of thing in mind. It could be added, and nothing new could break out (if done properly, which is feasible but never easy) without an explicit opt-in provided.

2 Likes
6

Yes, youā€™re right, that bit isnā€™t specifically related to the Wasm sandbox, as the original issue notes that

Perhaps a middle ground, here, would be to run the make() steps directly in the WASI code. But eventually the idea would be that make() steps happen by a separate build runner, not by the wasm guest.

Losing custom build steps is really more a consequence of the planned ā€œseparate processes for the configure vs make stepsā€ architecture, since the make step process wouldnā€™t be running any user-defined code in itself (unless itā€™s via a run step or system command, of course).

2 Likes