Slice and Array not pointing at the same data

MassKlaus · February 6, 2025, 11:25pm

I’ve run into a rather confusing issue where an array and it’s respective slice seem to be pointing at two different things. I am writting a CHIP-8 emulator and I was trying to do some conversion from u8 array to u16 array, in the middle I decided to use a slice rather than an array but started getting errors through the code so I slipped in a few asserts

Here is the assert line:

    const upper: u16 = self.memory_s[self.execute_cursor];
    const lower: u16 = self.memory_s[self.execute_cursor + 1];

    std.log.info("{} {}", .{ self.execute_cursor, self.execute_cursor + 1 });
    std.debug.assert(self.memory[self.execute_cursor] == self.memory_s[self.execute_cursor]);
    std.debug.assert(self.memory[self.execute_cursor + 1] == self.memory_s[self.execute_cursor + 1]);

The memory_s is assigned to on init and then never changed again

This is the error

C:\Programs\zig\zig-windows-x86_64-0.14.0-dev.3062+ff551374a\lib\std\debug.zig:518:14: 0x7ff79d3b71ed in assert (Chip8.exe.obj)
    if (!ok) unreachable; // assertion failure
             ^
D:\Coding\Chip8\src\emulator.zig:389:21: 0x7ff79d3b57f7 in currentInstruction (Chip8.exe.obj)
    std.debug.assert(self.memory[self.execute_cursor] == self.memory_s[self.execute_cursor]);

The repo in question in the specidied branch oddError:

At first I believed it’s just another mess up on my part so I went around the code trying to ensure that It’s not broken. But the pointer address of the slice not being the same as the array while debugging had me thinking this could be compiler related. So to be on the safe side, I decided to mention it here first.

Zig version: 0.14.0-dev.3062+ff551374a

castholm · February 7, 2025, 12:12am

The problem is in Emulator.init:

pub fn init(program_bytes: []const u8) !Emulator {
    // ...
    var emu: Emulator = .{
        // ...
    };
    // ...
    emu.memory_s = &emu.memory; // points to the current stack frame
    // ...
    return emu; // this returns a copy of 'emu', including the `emu.memory` array
}

// ...

fn foo() void {
    const emu = try Emulator.init(bytes);
    // 'emu.memory' resides on the 'foo' stack frame, but
    // 'emu.memory_s' points to data on the 'init' stack frame,
    // which no longer exists and will be overwritten by junk
}

In summary:

emu is initialized and stored on the init function’s stack frame. This includes the emu.memory field (important reminder: arrays are values in zig, they don’t decay into pointers like in C).
A pointer pointing to emu.memory is then stored to emu.memory_s, meaning that emu.memory_s is pointing to the current stack frame.
Finally, emu is returned to the caller (the foo function), meaning that its data is copied by value to the result location. After this, the init stack frame is no longer active and any data residing on it invalidated.
The emu inside the foo function’s memory_s field is pointing to data that is no longer valid and which may be overwritten by any random junk data.

I would advice against defining self-referential structs (meaning structs with fields that point to the struct instance itself) as they are very error-prone and easy to make mistakes with when struct instances are copied around without updating pointers.

The memory_s field is pretty redundant; just pass &emu.memory directly instead.

The problem you’re experience is kind of similar to a few recent threads (Seg faults using dir.Walker through interface - #2 by castholm and Local Random Number Generator sometimes core dumps2 - #3 by castholm), so if anything is unclear I would also suggest that you check both of those threads and my answers.

MassKlaus · February 7, 2025, 12:16am

I must admit, I fully assumed that returned memory would be at the same address as if this was heap allocated. Thanks a lot!

The reason I was doing this was to test for why the u16 pointer wasn’t behaving as expected.

That’s one more thing for me to keep in mind.

Again, thanks a lot!

castholm · February 7, 2025, 12:22am

Relevant issues/proposals that might have saved you from the confusion if implemented :

github.com/ziglang/zig

returning a pointer to a local should be a compile error

opened 05:13AM - 27 Jun 20 UTC

ghost

proposal

Maybe duplicate, but I couldn't find it anywhere. Consider the following code…: ``` zig const Point = struct { x: f32, y: f32, }; // Using #1717 syntax -- this proposal does not depend on #1717 const init = fn (_x: f32, _y: f32) Point { var inst = Point { .x = _x, .y = _y }; return inst; }; ``` No problems here, copy elision (should) put `inst` in the right place. Now consider this alternative: ``` zig const init = fn (_x: f32, _y: f32) *Point { var inst = Point { .x = _x, .y = _y }; return &inst; }; ``` Someone who doesn't know all the details of RLS might do this. Problem is, `frame` is now explicitly stack-allocated, in a frame which is invalidated as soon as `init` returns. This is comptime-detectable undefined behaviour. Zig should not compile this code. (There are reasons not to allow this at comptime either -- just look at the headache that is https://github.com/ziglang/zig/issues/5718#issuecomment-650491695. Also, breaking from blocks permits a similar problem.)

github.com/ziglang/zig

Proposal: Pinned Structs

opened 06:49PM - 13 Jan 21 UTC

SpexGuy

proposal accepted

This proposal supersedes #3803 and #3804 as the plan going forward for pinned st…ructs, [as discussed in the design meetings](https://docs.google.com/document/d/1GFQxcoXgzmOFThjFLpjkVASopa33el2f5esvk_OZlCo/edit#heading=h.uqpv9ul2nyey). ## Motivation A common pattern in Zig is to embed a struct in a larger struct, pass pointers to the inner struct around, and then use `@fieldParentPtr` to retrieve the outer struct. This is central to our current favored approach to runtime polymorphism. However, it's easy to accidentally make a value copy of inner structs like this, causing hard to debug memory corruption problems or segfaults. A similar problem exists with types that may contain pointers to data within themselves. As an example, consider an array with an inline stack buffer. Copying this value would yield a broken object, with a pointer pointing into the previous instance. Finally, at the extreme end of this is async frames. Async frames may contain pointers into themselves, and so cannot safely be copied. Where possible, the compiler should give an error if you try to copy any of these types. This is also important for the current plan for Result Location Semantics (see #2765), which is very implicit. In the case of these types, it's extremely important to know if RLS is working (and to fail the compile if not). ## Proposal Introduce the `pinned` keyword. This keyword can appear as a modifier to an anonymous type declaration, similar to `packed` and `extern`. `pinned` may coexist with `packed` or `extern`, and is compatible with `struct` and `union` types. It is not allowed (or ever needed) on `opaque`, because opaque values cannot be copied. It is also not allowed on `enum` or `error`, because those types cannot contain self pointers. If a type is `pinned`, it means that the address of the type is part of its value, and the compiler is not allowed to copy an instance to a new location. But braced initialization is still allowed, and the address may be forwarded through result location semantics. Any composite type which contains a field which is pinned is also considered pinned, and the same restrictions apply. This includes error unions and optionals, so #2761 will also be important for working with these types. Frame types will also be considered pinned. It's important to note that, despite some superficial similarities, `pinned` is not related to RAII. Pinned will not enforce that your deinitialization code is run, nor will it prevent you from overwriting fields that needs to be cleaned up, or even whole structs with braced initializers. It only prevents you from accidentally relocating an existing object. ## Examples ```zig const CAllocator = pinned struct { allocFn: fn(*CAllocator, usize) *c_void, freeFn: fn(*CAllocator, *c_void) void, }; const CountingAllocator = struct { // CountingAllocator is pinned because of this field allocator: CAllocator = .{ .allocFn = allocFn, .freeFn = freeFn, }, count: usize = 0, inner: *CAllocator, // we can return "by value" here because of RLS, // this is not actually a copy. pub fn init(inner_allocator: *CAllocator) CountingAllocator { // this is braced initialization of the result location, // which is allowed. return .{ .inner = inner_allocator }; } pub fn allocFn(super: *CAllocator, size: usize) *c_void { const self = @fieldParentPtr(CountingAllocator, super, "allocator"); self.count += 1; return self.inner.allocFn(self.inner, size); } pub fn freeFn(super: *CAllocator, ptr: *c_void) void { const self = @fieldParentPtr(CountingAllocator, super, "allocator"); self.count -= 1; self.inner.freeFn(self.inner, ptr); } }; fn foo() void { // this is fine, it's not a copy. The result location is forwarded to init. const alloc = CountingAllocator.init(); // compile error: copies pinned type from temporary const alloc2 = CountingAllocator.init().allocator; // this is ok (for now), the stack temporary is passed // as the result location and then its address is taken, // no copy occurs const alloc3 = &CountingAllocator.init().allocator; // braced initialization is fine const alloc4 = CountingAllocator{ .inner = alloc }; // but value copies are not, this is a compile error const alloc5 = alloc4; // this is fine, it just forwards the result location to bar. var frame: @Frame(baz) = bar(); } fn bar() @Frame(baz) { // this is fine, it forwards the result location. return async baz(); } fn bar2() @Frame(baz) { // this is fine with #2765, frame is the result location. var frame = async baz(); return frame; } fn bar3() @Frame(baz) { // this is not ok, NRLS does not apply to conditionals // like this so each of these returns is generating a value // copy of the frame, which fails because it is copying pinned. var frame1 = async baz(); var frame2 = async baz(); if (something) return frame1; return frame2; } ``` ## Other Notes Though we have chosen here to put `pinned` on type declarations, for some uses it is actually a property of values. For example, an allocator with no attached state would not need to be pinned. However, introducing `pinned` as a new CV-qualifier seems like a much more invasive change, and it's not clear that it's worth it.