Options for keeping return memory location stable

[joshua]

Hello.
In my application I have a struct, and I need to pass a pointer to the instance to a function (specifically it’s setting some callback in a C library).
The C library keeps that pointer around to use later when I trigger the callback.

I’d like to initialise the struct, pass the pointer to it to the c library, and then return the struct, without changing the memory location so the C library doesn’t break.

My understanding is that the following is broken, because the &b passed into the C library refers to a stack location that’s invalid after return.

fn init() Foo {
    const b = Foo{};
    function_from_c_lib(&b);
    return b;
}

fn main() void {
    const d = init();
}

I have 2 questions (assuming everything above is correct).

1. is this behaviour defined somewhere in the language reference? I couldn’t find a “Semantically returned values are memcpy’d” mention.

2. What are typical ways people get around this? Is it just to change init to take a *Foo and have the caller deal with creation (whether on their stack or on the heap)?
   It’s how I think I would do it in C, but I just haven’t seen many zig libraries have to do this, so I was wondering if I was missing something.

   Or , can I force it to be semantically inlined with inline fn?

Thanks for any help.

[Justus2308]

Hey, welcome to Ziggit!

As of now there are no real guarantees that this will work AFAIK. The C library might refer to a dangling stack pointer after init() returns.

Note that this will probably work anyway because of Result Location Semantics.

A common pattern to get around this (as you’ve suggested):

fn init(foo: *Foo) void {
    foo.* = .{};
    function_from_c_lib(foo);
}

pub fn main() void {
    var foo: Foo = undefined;
    foo.init();
}

This is used by std.Thread.Pool for example.

There are plans to make this more ergonomic by introducing pinned structs which cannot be copied by value and would allow you to do something like this:

const Foo = pinned extern struct {
    [...]
    
    /// Because `foo` is pinned you are guaranteed to get
    /// proper RLS here. The returned `Foo` will have the
    /// same address as the one created in this scope.
    pub fn init() Foo {
        var foo = Foo{};
        function_from_c_lib(&foo);
        return foo;
    }
};

pub fn main() void {
    const foo = Foo.init();
}

(I’m not 100% sure that this would work but if it wouldn’t you would at least get a compilation error. Without the pinned qualifier it would compile anyway but still be broken, as it is the case now.)

[floooh]

The C library keeps that pointer around to use later when I trigger the callback.

This is really not good C library design to begin with IMHO, e.g. everything would be fine if the call to function_from_c_lib(&b) just accesses &b but doesn’t store the pointer - and without a big fat ‘WARNING’ comment on that C function I would expect that it didn’t store the pointer for use after the function has returned but instead copies anything out of b it needs for later.

But if that’s the ‘contract’ between the C library and the code using that library, then you need to make sure that the memory behind the pointer doesn’t go away - and in your example that’s not the case.

In general, the solution to making such values ‘sticky’ is to allocate them on the heap instead of the stack. E.g. pass an allocator into init(), allocate b in the allocator, and also don’t return a Foo value, but a pointer to a Foo.

Then the owner of that allocator needs to make sure that b survives long enough to outlive the C library’s usage of &b - but if you are the author of that C library I would really recommend to change the library implementation to not require thinking about ownership and lifetime for anything that goes into or comes out of the library (and the same advice applies to any Zig modules - since Zig doesn’t help with any ‘ownership+lifetime questions’ I would avoid such questions to begin with by passing things around by value instead of by reference as much as possible).

[joshua]

Thank you both, I went for the “caller passes in a *Foo to init()” option.

It is not my library, specifically if you’re curious it’s libtls (from libressl).

The issue is in this function:

 int
 tls_accept_cbs(struct tls *tls,
   struct tls **cctx,
   ssize_t (*tls_read_cb)(struct tls *ctx, void *buf, size_t buflen, void *cb_arg),
   ssize_t (*tls_write_cb)(struct tls *ctx, const void *buf, size_t buflen, void *cb_arg),
   void *cb_arg);

You’re supposed to pass a pointer *cb_arg which will in turn be passed to every invocation of your callback. In my initial example I wanted to pass a *Foo.

I’m not sure they have any other choice than to store the pointer, since they need to use it at arbitrary times.

I suppose you could pass in the specific pointer you want to thread through on every call that can trigger the callback.
e.g: instead of

ssize_t
tls_write(struct tls *ctx, const void *buf, size_t buflen);

they could have

ssize_t
tls_write(struct tls *ctx, const void *buf, size_t buflen, void *cb_arg);

But… most of the time you use tls_write you are not using the callback backend at all, so you’d have to make that last parameter NULL, I suppose it’s all a tradeoff

[floooh]

That’s like a userdata pointer which is quite common for C libraries with callbacks. And yeah, in that case there’s no better way and you’ll need to make sure that the data behind the pointer survives long enough.