The links I just shared dive deeper into these topics, but the answer is that you can analyze assembly code, detect memory accesses, and decorate them with the same checks that you would put around a pointer dereference in the language proper.
4 Likes
Hopefully zig could do a lot better here because it doesnāt depend on libc. But yeah thereās no way around having some amount of trusted code unless thereās hardware support. Even then something has to be the capability arbiter of global resources.
Take a DMA peripherals for example. All your best laid plans fall apart when random hardware can just take over the memory bus and access the entire physical address space. (One of the reasons I find the over indexing on affine type based approaches unsatisfactory).
That being said, I donāt think it would be particularly hard to add verifiable code in front of a DMA to manage capabilities before writing addresses/lengths to DMA constructs. This is an area where affine types get extremely hairy because the DMA really has ownership of any address+length in its mmio/scatter gather lists until manually cleared. (Unless you want to somehow model all the ways random peripherals on your target can trigger DMA transactionsāgood luck).
This is the point where folks will usually put their fingers in their ears and say something like āla la la at least itās marked with an unsafe blockā. But these are precisely the sources of memory unsafety bugs that slip through and could use the extra help.
3 Likes
I think this is a bit missing the point, to me Fill-C is the right move why ?
Because one itās optional, not all software should care about any kind of safety, so itās good that the 80% of users donāt have to pay the cost of additional constraint such that the remaining 20% of user can benefit from the additional safety.
The second and arguably the best yet, is the fact that it is an ABI, any language could essentially be compiled to this ABI, and this solves real problems, thereās so many libraries, like ffmpeg, libgit, glfw, raylib, and many many more that are written in C or C++ and arenāt going anywhere anytime soon, so if now itās just a matter of compiling to a specific ABI, to bring many more guarantees, itās really good.
6 Likes
Thanks, that makes sense, especially the ability to reuse C code that just doesnāt care.
I do think the idea of having a compilation mode that only permits a āsubsetā of zig I described is also possibly not a bad idea either, mainly because it seems like itās something thatās easy to implement, either on the compiler or more likely with an external tool. (example a formatter that complains if you donāt use this subset + a library that uses that subset to help you do things)
edit:
I also want to clarify I never meant to say that I want these changes to be forced into the language In Case I didnāt clarify that enough, I would never want that, I use these things all the time, the suggestion was an ability to opt-in.
1 Like
I would add to this also that if CHERI hardware ever becomes a common thing, then you will be able to just build your code normally and get optimal behavior from the hardware without having to modify any of your code.
4 Likes
Oh, didnāt think about that either!
1 Like
Yes I also agree, CHERI is very exciting, I always though it should be a hardware job to enforce invariant as much as possible. Not to say software shouldnāt help, but software enforcement is only meaningful if the platform it is running on is also enforcing the same rules, otherwise, even if your software is āsecuredā the OS could become the target, and overcome the safety that your language guarantees
This is verging off topic, but, to emphasize this, there is one neat theoretical fact that deserves to be as wildly know as the Riceās Theorem: If you have a program, and its big-O time complexity is expressible as a primitive recursive function, the program itself can be expressed as a PRF.
Like, if you wrote a thing, and it runs faster than 2^(2^(2^n), then you donāt need a full Turing Machine for such a problem.
Like Riceās Theorem, it is a bit of a vacuous statement, as you essentially equip a TM with a āfuelā counter that forcibly halts the machine after it runs out of fuel, but it is still neat that, for any problem which you hope to solve before the heat death of the universe, you donāt actually need non-termination demons.
4 Likes
There is always a slight performance impact. Mostly itās down to the increased size of pointers, as they become double size in CHERI, but also thereās extra CHERI specific instructions that are needed around things like function calls to manipulate caps. The compilers arenāt very good at hiding the cost yet. Donāt think I can share numbers, but with the benchmarks Iāve seen itās a few percentage points penalty, not a few hundred percentage points like Fil-C.
Iām aware of a Cambridge paper thatās due to be submitted this month that details the performance penalty in some specific cases on a few cores.
3 Likes
This is a bit science-fictiony, but what excites me most about CHERI story is that you can replace virtual memory with capabilities, getting rid of page table and TLB, which should free up extra hardware resources to offset the cost of bigger pointers. Skeptical if this can fly, but it does sound elegant.
5 Likes
Yeah, itās a really nice idea. I think weāre exploring something along those lines for an RTOS. As each capability passed to a process effectively defines the sandbox it can work in, you donāt need to have separate virtual address spaces for processes.
Iām not sure youād get rid of virtual memory entirely though. Having a virtual address space larger than physical RAM is a very useful thing. Itās mainly the memory protection features that become redundant.
2 Likes
Virtual memory is useful for quite a few more things than just protecting processes from each other.
It also letās operating systems lie about which pages actually are in the physical RAM (for example if certain code parts about your executable are never used, it may not actually be in RAM but on disk, freeing things up for other stuff), it letās the OS set up shared memory under the hood (e.g. sharing the code of a shared library between multiple processes instead of each one having their own copy) and quite a few more things.
Sure, in case of bare metal software (e.g. a microcontroller) none of this are probably implemented, but for a general purpose OS, you still want virtual memory.
1 Like
how will be fill-c compiler itself safe if written in c or c++ or any other unsafe code?
compile the fil-C source code using a non-fil-C compiler, the resulting compiler (letās call this stage 1 compiler) is unsafe but its output will be safe. Now use the stage 1 compiler to compile the source code again, the resulting compiler (letās call this stage 2 compiler) will be both safe by itself and producing safe output.
4 Likes
You should watch Pilzoās talk, the entire talk is super interesting but around the 25 minute mark he explains how you can reach memory safety starting from non memory safety
2 Likes
Unsafe compiler can output a safe program.
For example, literally every strict functional programming language is safe, but their interpreters are written in C. Another example is that nowadays almost every programming language, safe or not, are dependent on LLVM or GCC. Their machine codes are emitted from LLVM or GCC. LLVM and GCC are written in C, and they are very unsafe, and they are very buggy sometimes. But it doesnāt mean they are not usable, or their emitted results are bad.
When we talk about memory safety, we talk more about exploitability, or something hides for decades without anyone ever checked it. For example, a hacker found a use after free can promote the user account to be a superuser in a very rare circumstance. Letās go back to the examples of the second paragraph. A hacker can fuzz a normal unsafe compiler or interpreter by generating a lot of super large and super unusual codes, then let the compiler compile or interpret the codes. Then the hacker found out his account became admin account. This means the compiler or interpreter is unsafe. However, the hacker canāt do the same thing when running the program that is emitted by the compiler or when interpreting a piece of safe functional codes for normal daily using.
When in daily uses, the users are mostly likely already reported usability bugs and those were fixed in a reasonable time. (I am not saying they are not important though)
2 Likes
sorry from second para i could not understand, also second and third para relation i could not get.
sure i will watch , thanks
Sorry. I fixed the second and third paragraph a little, and added one more example. And see if you can understand better now.
even though you have safe code , compiler not affected if binary can run withou any other support like linker. but interpreter can be effected as it is in runtime with safe code? now i can understand better thanks
1 Like