This feels an important point about where Fil C currently is. I’m a bit surprised to find out about this in a Ziggit thread and not in the talk, nor Philip numerous tweets nor the issue. I don’t get how this can be considered “safe”.
This is typically the kind of bug which are exploited. It’s a little bit better than being able to write on the rest of the stack but still.
At least here Zig cover us here, but I think the Codeberg issue would gain by clarifying what kind of new checks they can enforce using Fil-C inspired technics, which are already enforced by Zig and thoses that won’t be enforced.
How asking for a list of safety checks is not technical ?
For the records, I’ve been under a positive impression of Fil C, and I even encouraged you to pursue this direction in a previous thread a few weeks ago.
Also I’m using Zig for applications where I allocate up front big buffers and use smaller parts for different requests/users. So the kind of example shared above is important to me. Most of my pointers are internal pointers computed based on stored offsets. Will Zig FilC mode help me here? I thought yes, but I’m now confused
to me, in the part of your post quoted by @andrewrk you don’t appear to be asking for a list of safety checks. indeed, you just express surprise and concern.
if i were overly sensitive to a hot-button issue, i might take the quoted part of your post as the kind of “well actually” that exhausts me when talking about memory safety with Rust users who used to use C++ but don’t also use Zig.
the quoted part does not have the technical reason for your concern, which i’m grateful to you for providing in this most recent reply.
Since Zig has explicit control flow, wouldn’t it be possible to implement Perceus reference counting instead of a runtime garbage collector? That could reduce the performance cost of the mode to 2-3x run time in the worst case scenario.
I recall one of his previous talks covered this - though I forget which one. He was talking about which libraries were more or less difficult to port Fil-C, and there was one that compiled fine with no changes, but was (1) an attack surface, and (2) heavily used arenas, which made the Fil-C checks moot within each arena. He didn’t like that, so he rewrote it to use individual allocations instead.
I’d say it just goes to show that a program can only be ‘safe’ if the programmer cares to make it so. The compiler can certainly make it easier (or harder), but that’s it.
C has an inherent problem, in that the pointer type allows access to EVERYTHING! so there’s no way to say a particular de-referencing is “bad” until you go outside of the allocation. In the case of an arena allocator, the Fil-C runtime could provide one that constructed inviscaps the same way malloc() does, but there’s no way for a custom one to do anything.
In Zig, we have pointer types with scope - slices and single-item-pointers. The compiler can, and already does, put runtime bounds-checks around their use. What Fil-Zig would be able to do if fall back to allocation-level checks for many-item-pointers and C pointers.
The aux allocation is only used if you are storing pointers into the memory allocated by the arena - same as any other heap or static memory. The only time you don’t need an aux allocation is when you store the pointers on the stack, because the compiler simply stores the lowers on the stack as well.
That’s not the problem with arenas. The problem is that from the perspective of Fil-C’s allocator, an arena allocator is only one allocation, with one set of capabilities. That means if an attacker gets control of one pointer returned by the arena allocator, it can look at the rest of the arena, and the Fil-C runtime will not see an issue with that.
I guess you could add instrumentation to the arena allocator to detect this… but then it starts to look less and less like an arena allocator. Also, I personally don’t think it’s a bad thing for a programmer to be able to ‘defeat’ the Fil-C checks, with sufficient effort, when there are clear performance benefits and no attack surface. Keep in mind you can only do this with your own allocations, not pointers you receive from other people’s code or syscalls.
The problem is that from the perspective of Fil-C’s allocator, an arena allocator is only one allocation, with one set of capabilities
I’m not sure if the filc runtime has an API for that, but I think it is alwas safe to split a large (lower, upper) capability into two, such as (lower,midpont) and (midpoint,upper).
Or just anoint the allocator interface with capability creation superpowers instead of/in addition to malloc.
But even without that it might be okay. In a recent presentation, the filc author defined memory safety as “an attacker doesn’t get access to the whole memory”. So I think if an attacker can get access to the whole arena, but not anything outside of it, then that still satisfies the definition.
If you restrict yourself to call “Fil-C allocator” then yes, but I was thinking how the std.heap.ArenaAllocator could contribute to FilC guarantees by inserting invisicaps too.
When you arena.alloc(T, n) it’s pretty clear to the compiler if T contains ptr or not, so the arena can allocate aux memory at the same time and correctly set the ‘aux’ word for this allocation.
For me “attacker gains control of the full arena” is a big NO.
I’m not sure if the filc runtime has an API for that, but I think it is alwas safe to split a large (lower, upper) capability into two, such as (lower,midpont) and (midpoint,upper).
Unfortunately no, I did look for such a function, a few weeks ago, but it doesn’t exist. It’s not possible with the way invisicaps are implemented: The lower is used by the Fil-C runtime to access the upper and aux word stored before each allocation, so if you wanted to make a lower inside the allocation, you would need to store its upper and aux word… in the previous section of memory. Oops.
You could try to redesign Invisicaps to handle that, but you won’t be able to get around the fact that each time you split a buffer, you need to store the new ‘upper’ word somewhere, which means some sort of heap allocation… and now you’ve lost the main reason for using Arenas anyway.
While it would be nice to have Arenas ‘just work’, I think it’s perfectly fine to just use Filip’s solution of ‘replace the arena with malloc and let the garbage collector sort it out’. This is not hard to do even in a C project… but imagine how convenient it would be if there was a programming language that let you change allocators as easily as changing the argument to a function?
I think that’s a fairly limited definition. If you can access everything in an arena you can cause some havoc for sure.
Depends what is in the Arena. If a webserver, for example, is using one big Arena to handle every connection, then it’s very bad; an attacker can read/write other user’s data. If it uses a single Arena per connection, then it an only read and write it’s own data. Much more difficult to exploit.
… Unless the webserver does something silly, like storing access permissions in the same Arena. The attacker uses a buffer overflow to overwrite the access permissions, and now it can read/write anything the webserver daemon can.
The common factor is mixing data from different privileges; server & user, or user & other user. If you keep them separate, Fil-C will zealously enforce that separation. If you mix them, Fil-C will assume you have a good reason for doing so.
In any case, I think people are overthinking arenas:
They’re easy to replace with a GPA, especially in idiomatic zig code.
If you really need the performance benefits from Arenas? Put your performant code in a Yolo-C (or Yolo-zig) library and transfer data via sockets or pipes. Keep it small and simple, so you can unit-test and fuzz it into submission.
If you really need the automatic cleanup that Arenas provide? No worries, Fil-C has a garbage collector.
I could see a Fil-C aware Arena Allocator being a useful option in the future, but I don’t think it’s something that requires consideration now, in the early development of a Fil-C target.
alloc(T, n): if T doesn’t contains ptr, just insert a “null” aux ptr in front of the allocation.
alloc(T, n): if T does contains ptr: make two allocations, tmemset` the second one to zero, insert an “aux” ptr to in in front of the first one , and return it.
free: check the aux pointer and also free it
resize: seems fine too, logic being a bit more complicated because both slices need to resize.
I’m not 100% sure if extra precautions are needed to prevent attacker to modify the aux section, normally the lower bound ptr seems enough.
Of course for all of this to make sense you still need to insert the FilC checks which can’t be done in userland
I did some investigation of Zig’s type system (I did not know it before!) and how Zig’s allocators work.
The most straightforward integration between Zig and Fil-C that is:
Zig compiles to LLVM configured the Fil-C way. Maybe literally the Fil-C fork of LLVM.
_T and [_]T and friends lower to LLVM ptr type, and so get the Fil-C instrumentation.
Using custom allocators is fine. For arenas, fixed buffers, page allocators etc., you get a Fil-C capability that encompasses the entire allocation.
What you get from this:
UAF cannot be used to type-confuse an integer with a pointer in an interesting way.
Fil-C provides fallback bounds checks at coarses granularity than Zig does. Fil-C’s bounds checks are guaranteed no matter how you write your code; Zig’s are guaranteed only if you don’t race or use [*]T directly.
Interoperability with Fil-C.
This feels useful, but also somewhat anticlimactic.
So I wonder: is it possible to make the allocator interface special in this way:
all calls to alloc just call zgc_alloc in Fil-C mode
all calls to resize/remap just call zgc_realloc (though resize might need a new intrinsic on the Fil-C side)
calls to free just call zgc_free or do nothing
The safest and possibly most performant way to use Fil-C is to just lean into the GC.
You could imagine some kind of builtin_malloc annotation on allocators that achieves this (Andrew’s suggestion to me).
Another issue is distribution. Fil-C isn’t just an llvm plugin; it’s a customized llvm and customized clang. Those patches are not upstream. So this project would mean that Zig would have to have a story for building my llvm fork (or using binaries I provide?).
Finally, I wanted to step back and give a braindump of what I think Zig’s actual memory safe issue is, so we can talk about attacking that head-on, if Fil-ABI wasn’t the best choice for some reason:
Use-after-free is useful to attackers for a specific reason:
It’s an opportunity to type-confuse an integer with a pointer. Attacker causes program to free an object that has an int at offset 40 while still having a way to access that object, then the attacker causes the program to allocate another object with a pointer at offset 40 in the same location, and then the attacker controls an integer (using the old object) that the program uses as a pointer (in the new object). The reverse also works (free an object with a pointer, allocate new object with an integer).
It’s an opportunity to mess with the allocator’s innards. Often, freeing an object means that the allocator will place its meta-data (like free lists) inside the freed memory. So, by accessing the dangling pointer, the attacker can cause the allocator to vend objects at addresses of their choosing (the next alloc call returns a pointer of the attacker’s choosing).
Hence, in a language that defends against everything but use-after-free, the thing you want to do is add P^I (pointer-integer exclusivity). Simplest version of this is that you have a bitmap somewhere that tells you, for each word in memory, whether that word genuinely gets to be a pointer, or should be an integer instead. This might not work in Zig. It almost works in tagged unions, because you could say that changing which field is active edits the bitmap. It completely falls apart if you use extern unions - but maybe the answer there is that you don’t get checking on extern unions? That might be adequate.
Note that on platforms that have PAC (aka ptrauth), you could implement P^I by signing all pointers. This totally works, even with unions, but it’s depressingly slow, at least last time I checked.
I’ve often wondered if there’s some other trick that could be deployed to just give you the P^I property without pulling in all of what Fil-C does.
I might have an idea (I don’t have any project on codeberg, but on github and at Work), I’ve read the conversation, and I think there might be a good middle ground that avoids adding a language builtin. Maybe we could add a new calling convention for each of the operation that fill_c needs and then we could do something like this :
And for the other operation, there would be a fil_free, fil_remap etc. Then for the case of the Arena, the arena could have special code that register each subsequent suballocation in some ways ? The way i see it this avoids adding a new builtin, and those “calling convention” would just be a way to tag those functions such that the compiler can wire them properly, under the hood they would be lowered to either auto or something else.