Manual Indexing into an Allocation

Help
1

So, I added a basic example to show the concept, but my desire is to store data into a u8 buffer. No need for sentinels or anything as things are fixed length.

However, when I try to memcpy data over, it errors on

error: type '*u8' is not an indexable pointer
        @memcpy(&storage[i], &examples[i]);

which is just a bit surprising for me. In a sort of C sense, I would’ve expected this to work. copying the number of bytes available in the source to the dest, only encountering errors if the source exceeded the dest in length.

const std = @import("std");

pub fn main() !void {
    const GPA = std.heap.GeneralPurposeAllocator(.{});
    var gpa: GPA = .init;
    var allocator = gpa.allocator();
    const storage = try allocator.alloc(u8, 10);
    defer allocator.free(storage);
    @memset(storage, 0);

    const examples: [5][]const u8 = .{
        "aa",
        "bb",
        "cc",
        "dd",
        "ee",
    };

    var i: usize = 0;
    while (i < examples.len): (i += 1) {
        @memcpy(&storage[2 * i], &examples[i]);
    }
    std.debug.print("{s}\n", .{storage});


}
2

This change should work as expected:

@memcpy(storage[2 * i..].ptr, examples[i]);

zig build run
aabbccddee

From the language reference on memcpy:

dest must be a mutable slice, a mutable pointer to an array, or a mutable many-item pointer

source must be a slice, a pointer to an array, or a many-item pointer

… at least one of source and dest must provide a length, and if two lengths are provided, they must be equal

Your example’s mistakes are that dest is a single-item pointer, and that src is a pointer to a slice, not the slice itself.

In my example (noting there are many other ways to make it work), dest provides a many-item pointer to the target destination, and source is a slice containing the length so that memcpy knows how many items need to be copied.

Edit: fix some URLs

2 Likes
3

Applying your concept of .ptr instead of & worked for my more complex example, i.e. my actual problem.

1 Like
4

I would call it suboptimal solution. Getting raw pointer from slice causes you to lose bounds checking provided by @memcpy.

Here is example of error which you can encounter using such method. I changed length of storage slice from 10 to 9.

const std = @import("std");

pub fn main() !void {
    const GPA = std.heap.GeneralPurposeAllocator(.{});
    var gpa: GPA = .init;
    var allocator = gpa.allocator();
    const storage = try allocator.alloc(u8, 9); // 9 instead of 10
    defer allocator.free(storage);
    @memset(storage, 0);

    const examples: [5][]const u8 = .{
        "aa",
        "bb",
        "cc",
        "dd",
        "ee",
    };

    var i: usize = 0;
    while (i < examples.len) : (i += 1) {
        @memcpy(storage[2 * i ..].ptr, examples[i]);
    }
    std.debug.print("{s}\n", .{storage});
}

> zig build run
aabbccdde

It works successfully even tho we accessed out of bounds of an array.

Instead you can write memcpy line like this:
@memcpy(storage[2 * i ..][0..2], examples[i]);

With that original program will work correctly but one with 9 instead of 10 will panic with index out of bounds as we want it to.

zig build run

thread 31617 panic: index out of bounds: index 10, len 9
/home/andrewkraevskii/Documents/projects/sandbox/src/main.zig:21:34: 0x11ae077 in main (main.zig)
        @memcpy(storage[2 * i ..][0..2], examples[i]);
                                 ^
/home/andrewkraevskii/.cache/zig/p/N-V-__8AAC_uTRUrhIpzwcTOMDh5tBuMQQ3cDzGRmhAegCJd/lib/std/start.zig:678:59: 0x11ae6a0 in callMain (std.zig)
    if (fn_info.params.len == 0) return wrapMain(root.main());
                                                          ^
???:?:?: 0x7f4160c2a1c9 in ??? (/lib/x86_64-linux-gnu/libc.so.6)
???:?:?: 0x7f4160c2a28a in ??? (/lib/x86_64-linux-gnu/libc.so.6)
???:?:?: 0x11d3cb4 in ??? (???)
3 Likes
5

Agree that my solution is flawed. But I don’t think a run-time panic is necessarily better. Since the language gives us the tools to do either, I think it’s up to the author to determine how they want to manage program safety (silent failure vs. panic).

Practically, I’d avoid builtins for these simple scenarios and prefer something more robust from std:

    const storage = try std.mem.concat(allocator, u8, &examples);
    defer allocator.free(storage);
    std.debug.print("{s}\n", .{storage});
1 Like
6

The bounds checking does not invoke a panic, it invokes a runtime checked illegal behavior. In short, it’s closer to if (a.len != b.len) unreachable; than if (a.len != b.len) @panic("");. In long, this means that it does not always panic; it only panics in a safe build optimization level, like Debug or ReleaseSafe. If you build the code with ReleaseFast or ReleaseSmall, it will silently fail instead of panicking.

Returning an error instead of a panic may be a good idea for your problem, but they are semantically different things. Returning an error means: “user behavior or environmental factors may cause us to have to stop here, and it should be up to the caller to handle this case”. Panicking means: “it is a programmer error for this condition to occur, and the bug should be patched”. In this case, it means that the caller must ensure they only pass slices of the same length, which seems like a reasonable thing to assert that the programmer does.

3 Likes