Massively Parallel Programming in Zig (More GPU Support Incoming)

News
1

Well, these are exciting times we live in. In the 0.11.0 release notes, you’ll see this: 0.11.0 Release Notes ⚡ The Zig Programming Language

In my opinion, this is an important step to take. This space really needs some fresh thinking and better syntactical support/design in general.

According to the release notes, we can see that:

For now, these are only wired up to work when compiling Zig to AMD GCN machine code via LLVM, that can be used with ROCm. In the future they will be added to the LLVM-based NVPTX and self-hosted SPIR-V backends as well.

To be clear, this is AMD GCN, so PTX (think NVIDIA) is not yet supported: PTX Compiler API

Thoughts, opinions, celebratory noises about getting some standardized support for what is ostensibly one of the most important components of modern computing hardware?

3 Likes
2

Here are some hopes I have for this new direction (these are mostly CUDA concerns):

  1. Standardized integer types for kernel grid dimensions and len/size type parameters. CUDA is very inconsistent with their integer types. For instance, in Cutensor, sometimes parameters are signed 32 bit ints… sometimes they are unsigned 64 bit… sometimes you get both for size types on the same function call… meanwhile their block objects contain different types than that still. I hope they bring more consistency here.

  2. This is a big ask upfront, but I hope they look into cooperative kernels from the very beginning with CUDA when they get there. GPU programming is already infested with indirection, and having to indirectly dispatch to a kernel that they has to be wrapped with yet another call to launch it as a cooperative group is an annoying swamp of boiler plate.

  3. Support for error enums in kernels. Kernels are notorious for janky error handling. This is why you see the same error-handling macros appearing in so many GPU centric libraries.

1 Like
3

Can you target WGPU with SPIR-V? Because my impression is, that WGPU and WGSL are getting more and more attention.

4

Interesting. I did a preliminary search and I’m mostly finding forum posts and a few GitHub issues… I did find this though:

Shader Support - wgpu supports shaders in WGSL, SPIR-V, and GLSL. Both HLSL and GLSL have compilers to target SPIR-V.

2 Likes