SPIR-V Backend Progress Devlog

News
, ,
1

Congrats to @ali for his first devlog entry!

50 Likes
2

Back to back devlogs! We are getting spoiled.

9 Likes
3

Following up with some handy links for those who want to play with shaders in Zig.

Iā€™m not sure how up to date this one is, but here are some examples:

also, @ali said

gota7 has been helping me to make sure real world examples work. these are pretty up to date i believe https://codeberg.org/7Games/zig-sdl3/src/branch/master-gpu/gpu_examples/shaders/zig

15 Likes
4

Iā€™m really happy to hear this. Last I checked, the SPIR-V support hasnā€™t included Metal/macOS. Has this changed?

5

Metal doesnā€™t consume SPIR-V format so thereā€™s nothing we can do in terms of supporting it.
Perhaps i misunderstood what you meant?

6

I hope itā€™s fine that Iā€™m asking this here and not in a dedicated question thread but what does Zig gain from SPIR-V? I read that it is an intermediate representation for shading languages. Does this mean Zig is getting improved type safety with shading applications or something else?

7

(Iā€™m not massively well-versed in GPU stuffā€”Iā€™m pretty confident everything below is correct, but apologies for any errors!)

Although itā€™s an IR, as far as the Zig compiler is concerned, SPIR-V is an instruction format, just like x86_64 machine code or WASM bytecode (which is actually also an IR). Like you said, the purpose of SPIR-V is for shadersā€”the way todayā€™s graphics APIs generally work is that you give them a compiled shader in a format like SPIR-V (SPIR-V is used by Vulkan, and is also supported by modern OpenGL extensions) which the graphics driver then compiles down to whatever hardware-specific ISA it needs. So, the Zig compiler supporting SPIR-V allows you to write GPU shaders in Zig. This is useful for a few reasons:

  • Most shading languages are, frankly, not goodā€”for instance, I believe GLSL is still pretty common, despite it essentially being a subset of C with some hacked-on extensions and incredibly limited tooling. Using Zig for your shaders has the immediate benefit that, to Zig users, itā€™s probably nicer than any existing shader language!

  • A key part of GPU programming is transferring data between the CPU and the GPU. When you do this, you write that data to a buffer on the CPU, and transfer it to the GPU, where it is read from a buffer. If the source language is the same between your CPU and GPU code, this becomes very convenient, because you can keep all of your shared data structures in a ā€œcommonā€ source file which is imported from both your CPU and GPU codeā€”so you donā€™t need to duplicate your type definitions and remember to keep them in sync!

  • Kind of a sub-point of the above: modern GPU programming often has a focus on minimizing the amount of data transferred between CPU and GPU by using dense memory layouts. This could involve bit-packing data, and thatā€™s something which Zig is quite good at thanks to packed struct and packed union types! So this is one specific case where Zig can significantly improve the ergonomics of dealing with data coming into the shader, whereas existing shader languages would (AFAIK) all require manual bit manipulation to pull out the bits you need.

20 Likes
8

I was silently hoping that I would one day be able to write shaders in Zig, this is really really cool!
Is this already available to users? I think itā€™s better if I just try to use this myself instead of bombarding everyone with questions :sweat_smile:

9

spirv-cross may interest you. It supports SPIR-V to GLSL, MSL, and HLSL. I canā€™t vouch for how good it is, but itā€™s perhaps worth exploring.

1 Like
10

Ohh, what great news! In a half-abandoned project by now I made a Vulkan rendergraph thingy. Back then the shader support wasnā€™t there yet, but canā€™t wait to get back to it, to try out throwing out spirv-reflect and have some inbuilt reflection. Thank you, @ali!

1 Like
11

This is a very interesting read! Iā€™d love to dabble with this SPIR-V work in my free time, but I lack the prerequisite knowledge to get started. Any reading materials or exercises youā€™d recommend for a newbie to get started hacking away at shaders in Zig?

1 Like
12

If you havenā€™t messed with shaders at all before, Iā€™d suggest doing an intro tutorial to them, just in GLSL. there are lots and lots, but Iā€™d be happy to find you some links if youā€™d like.

After that, maybe the next step would be to take one of the shaders you wrote as part of that tutorial and using the links @andrewrk posted as a guide, make that same shader but in Zig instead.

3 Likes
13

what does Zig gain from SPIR-V?

Potential advantages are that you can remain in the same language and directly share type definitions between the CPU and GPU side, and maybe even share code (e.g. common helper functions).

Making the compiler understand both the GPU- and CPU-side has amazing potential beyond ā€œjustā€ sharing code, when this works properly we could basically get rid of at least 75% of the API surface of current 3D APIs (most of that API surface only exists to describe the ā€˜shapeā€™ of data thatā€™s transferred from the CPU to the GPU, but that shape can be easily inferred by the compiler when it could see ā€˜both sidesā€™).

For a new language which does the whole CPU/GPU integration right, check https://minc.dev/, specifically this integrated rendering demo where both CPU- and GPU-side code lives in the same file:

6 Likes
14

I am personally betting on this being a very big deal for game development. The glue code between CPU/GPU is such a pain in the butt that is just accepted as the cost of doing business today. Everyone rewrites or copy/pastes the same data structures and algorithms around because thereā€™s not a good way to reuse shader code.

Writing your shaders and games in one language with an advanced build pipeline and package manager will be one of those obvious in hindsight, table stakes things once people who are complacent with the current standard realize what theyā€™ve been missing

4 Likes
15

I use SPIRVCross in sokol-shdc and can vouch for its robustness :wink:

AFAIK SPIRVCross is also used in Valveā€™s Proton btw (the SPIRVCross author has or is working on VKD3D-Proton).

The only downside is that SPIRVCross doesnā€™t support WGSL output (and most likely never will), so for that a separate solution is needed (in sokol-shdc I link with an older version of Googleā€™s Tint).

PS: Apparently MoltenVK also uses SPIRVCross.

3 Likes
16

Additionally, some languages could conceptually compile to SPIR-V, but at the end of the day you have to hand write and maintain, or codegen, the binding code for Vulkan and so on. I believe Zig comptime brings an edge here, since the meta programming is basically best in class, thereā€™s no costly tooling or compiler level support required, it can be done in user space

2 Likes
17

Apart from SPIRVCross (which is also used by the older Vulkan-on-Metal solution MoltenVK) maybe the new SPIRV-to-MSL code from KosmicKrisp (the new Vulkan-on-Metal ā€˜driverā€™) can also be salvaged, e.g.

SPIRV-to-NIR: Making sure you're not a bot!

NIR-to-MSL: Making sure you're not a bot!

ā€¦the problem with both SPIRVCross and this KosmicKrisp thingie is that they create MSL (e.g. textual source code) instead of the Metal bytecode format (which I think isnā€™t documented by Apple).

(this SPIRV-to-MSL translation would probably need to be the job of a build-system tool though)

Also D3D would have the same problem, it needs a translation layer from SPIRV to HLSL or DXIL (D3D will switch to a SPIRV-flavour though, so in the future it might be easier to directly support D3D).

1 Like