Allocations overriding each other?

Help
1

Let me first say my background is primarily in GC’d languages (but have some minor experience in Rust) so coming to Zig has been quite the learning experience. I am still fairly new to the language and still looking around learning the language doing some practical exercises.

Now I feel like I’ve hit a point where reading the documentation isn’t helping me and my skills are not advanced enough to properly debug what exactly is going on and I need some help.

I am attempting to write a very basic CLI tool that will eventually need to read from a JSON config file on the file system located at the user home dir. So I started writing a very basic config lib that does exactly that as an extra exercise.

My basic idea was to create a GPA that serves as the general allocator which gets used everywhere. Then in my config lib I just wrap it in an Arena to make cleaning up stuff more straight forward.

As for the config itself, the idea is to read from filesystem on init and persist the data back to the file on deinit. So essentially only in-memory data is mutated at runtime without excessive filesystem reads and writes.

    var gpa = std.heap.GeneralPurposeAllocator(.{
        .verbose_log = true,
    }){};
    const allocator = gpa.allocator();
    defer _ = gpa.deinit();

    var persister = try configr.Persister.init(allocator, "genmoji");
    const config = try configr.Config(Config).init(allocator, &persister, .{});
    defer config.deinit();

    std.debug.print("before {s}\n", .{config.data.model});

    ...

    const app = cli.App.init(
        allocator,
        "genmoji",
        package.version,
        &params,
        &commands,
    );

    std.debug.print("before {s}\n", .{config.data.model});

    const usage = try app.help();
    const result = try app.parse();

    std.debug.print("before {s}\n", .{config.data.model});
    config.data.model = "gpt-3.5-turbo";
    std.debug.print("after {s}\n", .{config.data.model});

steps [2/6] zig build-exe genmoji Debug native... LLVM Emit Object... info(gpa): small alloc 82 bytes at u8@104980000
info(gpa): small alloc 591 bytes at u8@10498c000
info(gpa): small resize 591 bytes at u8@10498c000 to 747
info(gpa): small resize 747 bytes at u8@10498c000 to 751
info(gpa): small resize 751 bytes at u8@10498c000 to 757
info(gpa): small resize 757 bytes at u8@10498c000 to 771
info(gpa): small resize 771 bytes at u8@10498c000 to 772
info(gpa): small resize 772 bytes at u8@10498c000 to 786
info(gpa): small resize 786 bytes at u8@10498c000 to 799
info(gpa): small resize 799 bytes at u8@10498c000 to 822
info(gpa): small resize 822 bytes at u8@10498c000 to 826
info(gpa): small alloc 2103 bytes at u8@104994000
info(gpa): small resize 2103 bytes at u8@104994000 to 2136
info(gpa): small resize 2136 bytes at u8@104994000 to 2148
info(gpa): small resize 2148 bytes at u8@104994000 to 2149
info(gpa): small resize 2149 bytes at u8@104994000 to 2165
info(gpa): small resize 2165 bytes at u8@104994000 to 2176
info(gpa): small resize 2176 bytes at u8@104994000 to 2183
info(gpa): small resize 2183 bytes at u8@104994000 to 2184
info(gpa): small resize 2184 bytes at u8@104994000 to 2189
info(gpa): small resize 2189 bytes at u8@104994000 to 2190
info(gpa): small resize 2190 bytes at u8@104994000 to 2210
info(gpa): small resize 2210 bytes at u8@104994000 to 2212
info(gpa): small resize 2212 bytes at u8@104994000 to 2218
info(gpa): small resize 2218 bytes at u8@104994000 to 2276
info(gpa): small alloc 6654 bytes at u8@10499c000
info(gpa): small alloc 8 bytes at u8@1049a4000
info(gpa): small alloc 8 bytes at u8@1049a4008
info(gpa): small alloc 20 bytes at u8@1049dc000
info(gpa): small free 8 bytes at u8@1049a4000
info(gpa): small alloc 38 bytes at u8@1049f0000
info(gpa): small free 20 bytes at u8@1049dc000
info(gpa): small free 8 bytes at u8@1049a4008
info(gpa): small resize 38 bytes at u8@1049f0000 to 31
info(gpa): small alloc 8 bytes at u8@1049fc000
info(gpa): small alloc 20 bytes at u8@104a34000
info(gpa): small resize 20 bytes at u8@104a34000 to 19
info(gpa): small free 8 bytes at u8@1049fc000
before gpt-4-turbo-preview
info(gpa): small alloc 105 bytes at u8@104980080
info(gpa): small alloc 64 bytes at u8@1049f0040
info(gpa): small alloc 112 bytes at u8@104980100
info(gpa): small free 64 bytes at u8@1049f0040
info(gpa): small free 105 bytes at u8@104980080
before thread 180635 panic: reached unreachable code

As you can see from the error log above there are a total of 3 debug prints trying to print the config value. Notice that only one of the before logs has been logged before something happens what I can only imagine memory conflicts between allocations. But then weirder stuff starts happening when I shift the debug logs around and drop the first two and only keep the ones around the actual mutation of the data.

    std.debug.print("before {s}\n", .{config.data.model});
    config.data.model = "gpt-3.5-turbo";
    std.debug.print("after {s}\n", .{config.data.model});

steps [2/6] zig build-exe genmoji Debug native... LLVM Emit Object... info(gpa): small alloc 82 bytes at u8@100dc8000
info(gpa): small alloc 591 bytes at u8@100dd4000
info(gpa): small resize 591 bytes at u8@100dd4000 to 747
info(gpa): small resize 747 bytes at u8@100dd4000 to 751
info(gpa): small resize 751 bytes at u8@100dd4000 to 757
info(gpa): small resize 757 bytes at u8@100dd4000 to 771
info(gpa): small resize 771 bytes at u8@100dd4000 to 772
info(gpa): small resize 772 bytes at u8@100dd4000 to 786
info(gpa): small resize 786 bytes at u8@100dd4000 to 799
info(gpa): small resize 799 bytes at u8@100dd4000 to 822
info(gpa): small resize 822 bytes at u8@100dd4000 to 826
info(gpa): small alloc 2103 bytes at u8@100ddc000
info(gpa): small resize 2103 bytes at u8@100ddc000 to 2136
info(gpa): small resize 2136 bytes at u8@100ddc000 to 2148
info(gpa): small resize 2148 bytes at u8@100ddc000 to 2149
info(gpa): small resize 2149 bytes at u8@100ddc000 to 2165
info(gpa): small resize 2165 bytes at u8@100ddc000 to 2176
info(gpa): small resize 2176 bytes at u8@100ddc000 to 2183
info(gpa): small resize 2183 bytes at u8@100ddc000 to 2184
info(gpa): small resize 2184 bytes at u8@100ddc000 to 2189
info(gpa): small resize 2189 bytes at u8@100ddc000 to 2190
info(gpa): small resize 2190 bytes at u8@100ddc000 to 2210
info(gpa): small resize 2210 bytes at u8@100ddc000 to 2212
info(gpa): small resize 2212 bytes at u8@100ddc000 to 2218
info(gpa): small resize 2218 bytes at u8@100ddc000 to 2276
info(gpa): small alloc 6654 bytes at u8@100de4000
info(gpa): small alloc 8 bytes at u8@100dec000
info(gpa): small alloc 8 bytes at u8@100dec008
info(gpa): small alloc 20 bytes at u8@100e24000
info(gpa): small free 8 bytes at u8@100dec000
info(gpa): small alloc 38 bytes at u8@100e38000
info(gpa): small free 20 bytes at u8@100e24000
info(gpa): small free 8 bytes at u8@100dec008
info(gpa): small resize 38 bytes at u8@100e38000 to 31
info(gpa): small alloc 8 bytes at u8@100e44000
info(gpa): small alloc 20 bytes at u8@100e7c000
info(gpa): small resize 20 bytes at u8@100e7c000 to 19
info(gpa): small free 8 bytes at u8@100e44000
info(gpa): small alloc 105 bytes at u8@100dc8080
info(gpa): small alloc 64 bytes at u8@100e38040
info(gpa): small alloc 112 bytes at u8@100dc8100
info(gpa): small free 64 bytes at u8@100e38040
info(gpa): small free 105 bytes at u8@100dc8080
info(gpa): small alloc 9 bytes at u8@100e90000
info(gpa): small alloc 65 bytes at u8@100dc8180
info(gpa): small alloc 75 bytes at u8@100dc8200
info(gpa): small alloc 67 bytes at u8@100dc8280
info(gpa): small alloc 143 bytes at u8@100eb0000
info(gpa): small alloc 8 bytes at u8@100eb8000
info(gpa): small alloc 40 bytes at u8@100e38080
info(gpa): small alloc 49 bytes at u8@100e380c0
info(gpa): small alloc 51 bytes at u8@100e38100
info(gpa): small alloc 101 bytes at u8@100dc8300
info(gpa): small alloc 298 bytes at u8@100ef0000
before genmoji
after gpt-3.5-turbo
...
Segmentation fault at address 0x16f598000
Panicked during a panic. Aborting.

It seems that in-memory value got overwritten somewhere along the way.

I really want to understand what is going on here but, as I’ve said before, I feel like I’ve hit a point where help from smarter people would probably get me on my way again. Sorry for the long post and appreciate any help or explanations or optimizations you can give. Please criticize all the things! :pray:

2

One of the first issues I struggled a lot with was figuring out why my deferred config.deinit() was never called. Turns out defer is only called when the scope fully completes. So it took me a while to realise that when I do std.process.exit(0) somewhere in the underlying flow the scope in practice never completes and the defer is never called.

In most languages when I write CLI tools I would handle errors where they happen and exit the flow early with a proper message to stdout. So definitely good to know Zig really wants you to return all your errors to the top level and handle them accordingly so defer’s can run when they need to run. It adds an extra level of paying attention to the flow of code I usually wouldn’t even bother thinking about.

3

In configr.Persister.init you are calling alloc = arena.allocator(); and then returning a copy of alloc. The call arena.allocator() returns an Allocator instance containing a pointer to the original arena variable. Using alloc after the init function returns is undefined behavior because the original arena variable has gone out of scope.

You do make a copy of arena and store it in your structure, but the allocator has a pointer to the original variable not the copy.

Edit: to fix this is you probably should not store alloc in Persister and just call arena.allocator() whenever you need a corresponding allocator.

Edit2: I can’t see anywhere that you actually use the allocator member of Persister, so maybe this isn’t the cause of your current problems.

2 Likes