Representing GPU device pointers on the CPU host side

Brainstorming
,
1

I’m making an abstraction over Vulkan based on No Graphics API and I’ve been debating how to represent gpu side pointers on cpu, here’s a small snippet of what it looks like right now in my user code so you get the idea:

const Data = extern struct {
    particles: *anyopaque, // gpu pointer, read by the shader
    count: u32,
};

const particles_gpu = try device.rawAlloc(count * @sizeOf(Particle), .of(Particle), .gpu);
defer device.rawFree(particles_gpu);

const data_gpu = try device.rawAlloc(@sizeOf(Data), .of(Data), .default);
defer device.rawFree(data_gpu);

// deviceToHostPointer gives you a cpu mapped pointer
const data_cpu: *Data = @ptrCast(@alignCast(device.deviceToHostPointer(data_gpu)));
data_cpu.* = .{ .particles = particles_gpu, .count = count };

// pass the gpu address to the shader
cmd.dispatch(device, data_gpu, .{ count / 64, 1, 1 });

and one line of pointer arithmetic:

const second_half = @ptrFromInt(@intFromPtr(particles_gpu) + (count / 2) * @sizeOf(Particle));

currently gpu pointers are represented as an *anyopaque but that’s just a placeholder. Gpu pointers are used a lot with this kind of api so I want it to be ergonomic. Here’s the options I have though of:

1. Struct with a u64 and methods

pub fn GpuPtr(comptime T: type, ...alignment and other pointer attributes etc) type {
    return struct {
        addr: u64,

        pub fn add(p: @This(), n: u64) @This() {
            return .{ .addr = p.addr + n * @sizeOf(T) };
        }
        // sub, alignCast, cast, slice type, ...
    };
}

You cant accidentally deref it so that’s good but I have to reimplement everything
pointers do as methods. And you lose a lot of the ergonomics of zig pointers.

2. Just keep using pointers

You keep all the ergonomics. But reads and writes are a bug. And you can pass cpu pointers to functions that expect gpu pointers and vice versa very easily. This doesnt work for 32bit hosts like wasm32.

3. Keep using pointers, but wrap the destination type to make it somewhat opaque

pub fn GpuOpaque(comptime T: type) type {
    return extern struct {
        bytes: [@sizeOf(T)]u8 align(@alignOf(T)),
    };
}

// *u32 -> *GpuOpaque(u32)

GPU pointers become *GpuOpaque(T). Deref still compiles but it’s harder to mess up. This doesn’t work for 32bit hosts like wasm32.

Do you guys know better options I don’t know about?

What if we could use physical_storage_buffer on the cpu side?

One possibility at the language level would be to make *addresspace(.physical_storage_buffer) T representable in CPU code (but deref would be a compile error). This would also work for 32bit hosts if 15232’s last requirement is implemented

Here’s the what the code would look like approximately:

const Data = extern struct {
    particles: [*]addrspace(.physical_storage_buffer) Particle, // gpu pointer, read by the shader
    count: u32,
};

const particles_gpu: []addrspace(.physical_storage_buffer) Particle = try device.alloc(Particle, count, .gpu);
defer device.free(particles_gpu);

const data_gpu: *addrspace(.physical_storage_buffer) Data = try device.create(Data, .default);
defer device.free(data_gpu);

// deviceToHostPointer gives you a cpu mapped pointer
const data_cpu = device.deviceToHostPointer(data_gpu);
data_cpu.* = .{ .particles = particles_gpu.ptr, .count = particles_gpu.len };

// pass the gpu address to the shader
cmd.dispatch(device, data_gpu, .{ count / 64, 1, 1 });

and the line of pointer arithmetic:

const second_half = particles_gpu.ptr + count / 2;
2 Likes
2

Vulkan-Zig tends to use enum(u64) { _ } i think, if i remember correctly?

Also, just in case it jangles some nerve endings in an interesting way, you can do things like

pub const MyPointerType = opaque {
    pub fn hasMethods();
};

similar to *anyopaque, it is a compile error to read from or write to *MyPointerType, but you can pass them around, cast them, etc, and they are allowed declarations, which *anyopaque obviously can’t do.

1 Like
3

Vulkan-Zig tends to use enum(u64) { _ }

Vulkan-Zig uses a plain u64: pub const DeviceAddress = u64; (for DeviceAddress specifically, for vulkan handles it does use non-exhaustive enums)

enum(u64) { _ } would be roughly similar to 1. Struct with a u64 and methods, it’s a fine solution but you dont get any of the conveniences of pointer types

*opaque {} and *anyopaque really are no goes because they dont support most of the convenience features of regular pointers and they don’t work for 32bit hosts.

thx for the suggestions though :+1:

4

I’ve internally debated this issue internally several times and can’t come to a conclusion.

I think a special address space which prevent pointer dereference, but still allows slicing would be the coolest thing.

2 Likes
5

I know the API requires pointers, but is there any value to having a slice-like “ptr + len” structure to track these GPU allocations? It can then be the foundation of safety checks in the GPU memory.

6

As they aren’t actually pointers in a meaningful sense to CPU code, I’ve always just passed them around as ints (like file handles). What ergonomics break with that method for you?

7

Well it is the same difference as using usize addresses instead of pointers except that there is no deref so its not quite as bad. No alignment safety no slices no type confusion safety no pointer arithmetic no pointer coercion…