`File.reader().readAllAlloc()` and `File.readToEndAlloc()` give different behavoir

So I’m currently learning OpenGL, and I chose to learn it with Zig. I struggled with a strange issue for hours until I finally found the fix.

Here’s the function I use to create a shader:

    pub fn init(allocator: Allocator, vertex_path: []const u8, fragment_path: []const u8) !Shader {
        const vertex_file = try std.fs.cwd().openFile(vertex_path, .{});
        const fragment_file = try std.fs.cwd().openFile(fragment_path, .{});
        defer vertex_file.close();
        defer fragment_file.close();

        const vertex_stat = try vertex_file.stat();
        const vertex_source = try vertex_file.readToEndAlloc(allocator, vertex_stat.size);
        defer allocator.free(vertex_source);

        const fragment_stat = try fragment_file.stat();
        const fragment_source = try fragment_file.readToEndAlloc(allocator, fragment_stat.size);
        defer allocator.free(fragment_source);

        // vertex shader
        const vertex = gl.CreateShader(gl.VERTEX_SHADER);
        defer gl.DeleteShader(vertex);

        gl.ShaderSource(vertex, 1, &.{vertex_source.ptr}, null);
        gl.CompileShader(vertex);
        try checkCompileErrors(vertex, .vertex);

        // fragment Shader
        const fragment = gl.CreateShader(gl.FRAGMENT_SHADER);
        defer gl.DeleteShader(fragment);

        gl.ShaderSource(fragment, 1, &.{fragment_source.ptr}, null);
        gl.CompileShader(fragment);
        try checkCompileErrors(fragment, .fragment);

        const id = gl.CreateProgram();
        gl.AttachShader(id, vertex);
        gl.AttachShader(id, fragment);
        gl.LinkProgram(id);
        try checkCompileErrors(id, .program);

        return .{
            .id = id,
        };
    }

Here’s the vertex shader that was causing the issue:

#version 330 core

layout (location = 0) in vec3 aPos;

void main() {
    gl_Position = vec4(aPos, 1.0);
}

Originally, I read the shader file using the reader() interface like this:

        const vertex_stat = try vertex_file.stat();
        const vertex_source = try vertex_file.reader().readAllAlloc(allocator, vertex_stat.size);
        defer allocator.free(vertex_source);

When I ran my program like this, I get this shader compile error

2025-03-21T22:10:31,725129870+01:00

And the only way to “fix” it was to add spaces after each line of the GLSL code, which made no sense.

However, when I switched to using readToEndAlloc() instead without the reader() interface, like this:

        const vertex_stat = try vertex_file.stat();
        const vertex_source = try vertex_file.readToEndAlloc(allocator, vertex_stat.size);
        defer allocator.free(vertex_source);

The shader compiled perfectly.

What really confused me is that the fragment shader compiled fine even when using the reader() interface. Here’s the fragment shader that worked:

#version 330 core
out vec4 FragColor;

void main()
{
    FragColor = vec4(0.5, 0.5, 0.5, 0.0);
}

If anyone knows why this happened, please let me know. I spent a lot of time trying to track this down, and I’d love to understand what was going on under the hood. Sorry for the long post.

1 Like

If length is NULL, each string is assumed to be null terminated.

gl.ShaderSource(vertex, 1, &.{vertex_source.ptr}, null);

I suppose you missed NULL terminator (you can also pass length array instead). If this is the case, undefined behavior is the reason it gives different results.

2 Likes

Oh wow, for the longest time I thought files were null-terminated. I didn’t even consider that. Now i feel stupid…

Huh, The tutorial I’m following did not add the length in his cpp code.

        const char* vShaderCode = vertexCode.c_str();
        vertex = glCreateShader(GL_VERTEX_SHADER);
        glShaderSource(vertex, 1, &vShaderCode, NULL);
        glCompileShader(vertex);
        checkCompileErrors(vertex, "VERTEX");

I didn’t know that std::String::c_str() appends \0 at the end of the string:

Get C string equivalent

Returns a pointer to an array that contains a null-terminated sequence of characters (i.e., a C-string) representing the current value of the string object.

This array includes the same sequence of characters that make up the value of the string object plus an additional terminating null-character (‘\0’) at the end.

I got CPlusPlus`ed.

1 Like

How did you arrive at the notion of passing vertex_source.ptr to gl.ShaderSource rather than just vertex_source?

Picking apart a slice type is generally not a good idea unless you know precisely what you’re doing. Most of the time there exist appropriate, automatic coercions between slice types and pointers. Both *T and []T, for example, can decay to [*c]T which is what’s often used for interop with unvarnished C code.

In your case, though, I imagine the actual argument type of gl.ShaderSource is something like [*][*:0]u8, which is an (unsized) array of pointers to zero-terminated byte buffers. The best Zig type to use here would therefore be a zero-terminated slice, i.e. [:0]u8, instead of just []u8. Using File.readToEndAllocOptions would allow you get such slice in return, and it would neatly coerce to the type required by OpenGL.

1 Like

How did you arrive at the notion of passing vertex_source.ptr to gl.ShaderSource rather than just vertex_source?

I believe gl is a @cImport, that’s why pointer is requested.

The best Zig type to use here would therefore be a zero-terminated slice, i.e. [:0]u8

I don’t think so, we can pass length array and use plain slice which is a lot more common. Good tip on readToEndAllocOptions though.

Tbh, I used it cuz it compiled :sweat_smile:

The type of the funtion is:

fn ShaderSource(shader: uint, count: sizei, strings: [*]const [*]const char, lengths: ?[*]const int) callconv(APIENTRY) void

So when i used File.readToEndAllocOptions

const vertex_source = try vertex_file.readToEndAllocOptions(allocator, vertex_stat.size, null, @alignOf(u8), 0);

I had to cast it for it to compile.

gl.ShaderSource(vertex, 1, &.{@ptrCast(vertex_source)}, null);