Provide my own malloc and free to C dependency?

Help
,
1

Is it possible for me to provide my own allocator to a c dependency so I can detect leaks?

2 Likes
2

Some C libraries allow you to customize malloc and freefunctions, some do not. If your dependency doesn’t, the best thing to do is to patch the code. Other methods include providing your own libc, LD_PRELOAD (global effect) on Linux and hooking.

Leak detection requires check after all memory is supposed to be freed, so I don’t think an elegant solution is available unless the library provides an allocator changing interface.

2 Likes
3

I was thinking about how could this be achieved the most pleasant way without patching, and came to this solution:

build.zig

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    const exe = b.addExecutable(.{
        .name = "main",
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
        .link_libc = true,
    });

    exe.addCSourceFile(.{
        .file = b.path("src/lib.c"),
    });

    exe.root_module.addCMacro("malloc", "lib_malloc");
    exe.root_module.addCMacro("free", "lib_free");

    b.installArtifact(exe);

    const run_cmd = b.addRunArtifact(exe);
    run_cmd.step.dependOn(b.getInstallStep());
    if (b.args) |args| {
        run_cmd.addArgs(args);
    }

    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);
}

src/main.zig

const std = @import("std");

var lib_allocator: std.mem.Allocator = undefined;

export fn lib_malloc(size: usize) ?*anyopaque {
    const total_size = @sizeOf(usize) + size;
    const ptr = lib_allocator.alignedAlloc(u8, @alignOf(usize), total_size) catch {
        // you should set errno here, auxiliary C function will work
        return null;
    };
    @as(*usize, @ptrCast(ptr)).* = total_size;

    return ptr.ptr + @sizeOf(usize);
}

export fn lib_free(ptr: ?*align(@sizeOf(usize)) anyopaque) void {
    if (ptr == null) return;

    const to_free = @as([*]align(@sizeOf(usize)) u8, @ptrCast(ptr.?)) - @sizeOf(usize);
    const total_size = @as(*usize, @ptrCast(to_free)).*;

    lib_allocator.free(to_free[0..total_size]);
}

extern fn do_work() void;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();

    const allocator = gpa.allocator();
    lib_allocator = allocator;

    do_work();
}

src/lib.c

#include <stdlib.h>

void do_work(void) {
    void *data = malloc(8); // leak
    (void)data;

    void *data2 = malloc(8);
    free(data2); // no leak
}

It reports leak correctly:

 $ zig build run
error(gpa): memory address 0x78a2c0a7c000 leaked:
/tmp/tmp.w7qI6cwFrg/src/main.zig:7:43: 0x10351e4 in lib_malloc (main)
    const ptr = lib_allocator.alignedAlloc(u8, @alignOf(usize), total_size) catch {
                                          ^
src/lib.c:4:18: 0x1034831 in do_work (/tmp/tmp.w7qI6cwFrg/src/lib.c)
    void *data = malloc(8); // leak
                 ^
/tmp/tmp.w7qI6cwFrg/src/main.zig:34:12: 0x10348b8 in main (main)
    do_work();
           ^
/usr/lib/zig/std/start.zig:524:37: 0x1034da0 in main (main)
            const result = root.main() catch |err| {
                                    ^
???:?:?: 0x78a2c0864487 in ??? (libc.so.6)
???:?:?: 0x78a2c086454b in ??? (libc.so.6)

It works by defining malloc and free macros, but I think this solution is not elegant. Real world case will also export malloc, free, calloc, realloc, GNU stuff like aligned_alloc, malloc_usable_size, memalign, etc. Don’t forget about reallocarray, strdup, and others. I think the “hack” presented here would fit small codebases, but for something more complex consider patching code.

EDIT: fix small mistake

6 Likes
4

Looks like my dependency has some aliases for malloc already, perhaps there is some kind of DEFINE mechanism or something to customize it…

/*------------------ Memory ------------------*/
static inline void *z_malloc(size_t size) { return malloc(size); }
static inline void *z_realloc(void *ptr, size_t size) { return realloc(ptr, size); }
static inline void z_free(void *ptr) { free(ptr); }

Edit: nevermind, this has nothing to do with that. Its just for the library maintaining comptability with other implementations: zenoh-pico: GitHub · Where software is built

5

There’s no configuration mechanism, zenoh_memory.h is included unconditionally in zenoh.h. You can use define “hack” as there’s only 3 functions to mock (malloc, realloc and free) or replace zenoh_memory.h with Zig build system.

1 Like
6

The library is actually written in rust, and uses the rust global allocator, so there isn’t really any guarantee that I can replace its allocator (rust could choose to use something other than libc, though I think its unlikely).
I guess I’m really just searching for a means to ensure that I drop all the the things I am supposed to drop when using this library.

7

The Rust global allocator can be replaced at link time, with enough fiddling about it seems you could use Zig export functions to provide the FFI interface using the allocator of your choice.

No idea if this is actually practical for your use case, but it seemed worth mentioning.

3 Likes
8

What are the benefits of using your own malloc and free over using something link Valgrind to detect leaks?

9

Valgrind doesn’t work on Windows, where I work most of the time.

1 Like
10

As pachde said, built-in memory leak checking is useful on Windows, but I believe it is more like using Zig allocator features, including arena, scoped allocations, write tests with failing allocator, ability to adjust allocator settings (for example see DebugAllocatorConfig), use more performant allocators.

1 Like
11

Also, zig is its own language. Sometimes people learn zig first and not C, so they don’t have experience with the C tooling :slight_smile: (me)

1 Like
12

Those are all great points, thanks guys!

I honestly had no idea Valgrind doesn’t work on Windows, that’s good to know.