Getting error(gpa): Allocation size 280 bytes does not match free size 448

Help
,
1

Hi guys, so I’m trying to write a scanner/lexer and for some reason I’m getting this issue where the allocation size does not match the free size. I’m not sure how to go about finding the cause of this. My project is here: Zrox/src at main · Dappstr/Zrox · GitHub

Screenshot 2024-07-22 003954
Screenshot 2024-07-22 003954941×743 36.3 KB

I’m pretty sure I’m defer-ing every time I allocate something.

Such as here (token.zig):

    pub fn to_string(self: *const Self, allocator: std.mem.Allocator) ![]const u8 {
        var buffer = std.ArrayList(u8).init(allocator);
        defer buffer.deinit();

        try buffer.appendSlice("Token { type: ");
        try buffer.appendSlice(@tagName(self.type));
        try buffer.appendSlice(", lexeme: ");
        try buffer.appendSlice(self.lexeme);
        try buffer.appendSlice(", Literal: ");
        try append_literal(&buffer, self.literal);
        try buffer.appendSlice(", line: ");
        try std.fmt.formatInt(self.line, 10, .lower, .{ .precision = 10 }, &buffer.writer());
        try buffer.appendSlice(" }\n");
        return try buffer.toOwnedSlice();
    }

And here (scanner.zig):

    pub fn deinit(self: *Self, allocator: std.mem.Allocator) void {
        self.tokens.deinit();
        keywords.deinit(allocator);
    }

And here (main.zig)

fn run(source: []u8) !void {
    const stdout = std.io.getStdOut().writer();

    var scanner = Scanner.Scanner.init(allocator, source);
    defer scanner.deinit(allocator);

    var tokens = try scanner.scan_tokens();
    defer tokens.deinit();

    const token_slice = try tokens.toOwnedSlice();

    for (token_slice) |token| {
        const token_str = try token.to_string(allocator);
        defer allocator.free(token_str);
        try stdout.print("{s}\n", .{token_str});
    }
}

fn run_file(path: []const u8) !void {
    const file = try std.fs.openFileAbsolute(path, .{ .mode = .read_only });
    defer file.close();

    const file_size = try file.getEndPos();
    const buffer: []u8 = try allocator.alloc(u8, file_size);
    defer allocator.free(buffer);

    const bytes_read = try file.read(buffer);
    _ = bytes_read;
    try run(buffer);
    if (had_error) return Err_Parsing.General_Error;
}

I’m looking for some guidance to help me try and find out why my allocation and free sizes aren’t matching.

2

Hey,
the stack trace in your screenshot tells you, which allocations are the problem (the ‘self.add_token_with_literal’ call).
Maybe this can help you to figure out the problem.

3

You actually forgot one place:

const token_slice = try tokens.toOwnedSlice();

toOwnedSlice transfers ownership to the caller, so you need to free it.

Furthermore it seems that scanner.scan_tokens() is making a copy of it’s internal ArrayList. This is a problem, because now you have two instances of the ArrayList which point to the same slice. Normally something like this would cause a double free, but since you forgot one deinit, it causes an invalid free because toOwnedSlice modifies the size of the allocation, but the scanner.tokens ArrayList doesn’t know about this.

3 Likes
4

There’s a few things going wrong.

scan_tokens is returning std.ArrayList(Token.Token), which means it is returning a copy of the ArrayList. In other words, if you did this in main:

var tokens = try scanner.scan_tokens();
defer tokens.deinit();

std.debug.assert(&tokens == &scanner.tokens);

The assertion would fail. This means that when you do:

const token_slice = try tokens.toOwnedSlice();

it only affects the copy of the ArrayList in main, while scanner.tokens still contains a pointer to the old slice. So, when scanner.tokens.deinit() is called in Scanner.deinit, it is trying to free the old slice.

Secondly, you are attempting to free the tokens ArrayList twice:

    var scanner = Scanner.Scanner.init(allocator, source);
    // Scanner.deinit frees the tokens ArrayList
    defer scanner.deinit(allocator);

    var tokens = try scanner.scan_tokens();
    // This (in theory) also frees the same ArrayList
    defer tokens.deinit();

There are two possible fixes:

  • Make scan_tokens return a pointer to the ArrayList. IMO this should only be done if you have a reason to alter the contents of the ArrayList after getting it from scan_tokens.
  • Make the scan_tokens return type []const Token.Token and return self.tokens.items instead. This comes with the caveat that further modifications to the ArrayList may invalidate the returned slice.

For both solutions, it should be noted that the lifetime of the returned value is the same as the lifetime of the Scanner, and that the return of scan_tokens should not be freed by the caller.

A third possible fix would be to copy the self.scanner.items slice and return the copy. Then it would be up to the caller to free the returned slice.

Side note: this deinit call is incorrect. A StaticStringMap initialized at comptime does not use heap-allocated memory, so it will attempt to free static memory and fail. In other words, there’s no need to call deinit when using initComptime.

2 Likes