Rewriting Bun in Rust

Media
21

Pre-merge, this took 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input token reads — around $165,000 at API pricing.

Not strictly related to Bun’s relationship with Zig, but spending $165,000 to make Bun 2% - 5% faster is bananas. I know Jarred sees other benefits, not just raw performance, but these are difficult to quantify short-term. Only time will tell, I guess.

And I can almost guarantee it cost more than $165,000 when this PR was merged. He states that was the bill pre-merge (whatever that means).

2 Likes
22

For a while there was a project that iirc aimed to provide a Rust implemention of Lua, which is a language similar to Javascript in relevant-to-the-discussion ways. It appears to have been scrapped in favor of Rust bindings to the C implementation, so I would bet that GC language implementations in Rust is an underexplored subject.

I guess (although I don’t know) that Bun doesn’t implement the GC portion of JavaScript; that’s (probably) a C/C++ dependency (JavascriptCore), so interactions with it already have to be marked unsafe, which might be the solution for a hypothetical GC implementation in Rust anyway.

1 Like
23

Something that I used to see mentioned a lot in Rust circles is that the borrow checker generally encourages data oriented design and clear and simple memory relationships, otherwise one ends up in borrow checker hell.

RAII & the global allocator can and do cut against that sometimes since one can clone their way out of trouble, but overall the principle held when I was still writing Rust.

I feel like Zig can feel surprisingly similar to Rust in encouraging those practices but through different means like the limitations of defer and the friction of passing Allocator and Io around.

It’s funny that the cultures of both languages can feel so at odds on this forum because I think the spirit of Rust 10 years ago reminds me a LOT of zig now.

9 Likes
24

Honestly, unnecessary mad. If Bun found other tool makes things easier and mistakes harder to happen it is fair. What should Zig do is how to make things easier in Zig, more safety features that is enforced not based on a guide, instead of being mad

1 Like
25

LLMs are going to destroy the best practices around Rust’s memory safety and make Rust code in enterprise as bad as existing C and C++ code, with unsafe splattered everywhere and filled with undefined behavior. Then when newcomers to the Rust language take a look at existing Rust code in the wild, they end up copying the terrible practices of LLMs instead of the best practices of experienced Rust developers.

Worst case scenario for Rust fanboys. Rust becomes widespread in the industry at the cost of memory safety and just becomes known as the LLM slop language.

4 Likes
26

It also is not just Bun that is using LLMs to rewrite code to Rust. The JavaScript package manager pnpm is using LLMs to rewrite their entire codebase over to Rust, and so is the React compiler. The entire JavaScript ecosystem seems to have an obsession with using LLMs to rewrite programs over to Rust.

Like rewriting to Rust is fine but do what the Fish shell devs did and take time to migrate individual components to Rust and checking to see that there are absolutely zero regressions, don’t just hand the entire codebase over to LLMs who then cause massive amounts of regressions and undefined and unsafe code in the codebase.

2 Likes
27

I think this is how it generally works in languages with linear types, like Austral.

3 Likes
29

i just want to add: IMHO, i don’t think they (or we) have realized the ramifications of this. no one wants to read ai slop. in fact, it’s actually less motivating and the developer loses incentive to want to grasp/understand that piece of code. “why bother, when it’s just generated and not human written?”

if we really wanted to, we can skim through the code and be like oh this does that, etc. however, i still think there’s a much deeper issue at play here. the feeling of wanting to understand something is degraded when we subconsciously know the code isn’t written by a human. the same way AI art is vastly different than a real painting

from my experience, it’s similar to the uncanny valley feeling you get from those ai images. but it’s with code

10 Likes
30

ArenaAllocator in Zig is a tool to help managing allocating and freeing memory in a group manner. It’s not just like what many articles tell you that Arena will allocate a lot of times and only free once. This gives people an impression that they will have one Arena on the top of their program, then only free when the program shutdown. That’s not true and that’s not how most Zig programmers use Arena.

People usually let their structs own their own arena. Each struct has its own ArenaAllocator. They have a function .init(whatever_child_allocator) !WhatEverStruct… So in this case, if their struct alive, all the heap data for the struct, doesn’t matter if they are in fields or not, will be alive. When their struct’s lifetime is ended, they call deinit(), in the deinit() function, the Arena will deinit(), and all the memory pages associated with the struct return back to the system. By using this way, you reduce the abusing of defer allocator.free(blabla), because you group the data together. You still defer deinit() all the data when you don’t need them, but you write less defer deinit() or free() codes, so you will less likely to forget. This is not a longer lived leaking. Sometime, you just delayed the free for nano seconds…

Arena Allocator in Zig can also be reset with retain capacity. The memory is not free but they will be reused. This is particular helpful when your program has a main loop (or multiple infinite loops in multi threading program), and your structs with Arena are being used in the loop. By the end of each loop, you reset(.retain_capacity) the Arena instead of deinit() it. In the next loop, the Arena will reuse the memory, and it only alloc more memory if the next loop need more memory then the previous loop. For example, you develop a website that allow users to post article. Everytime an user click the Submit button, you need some memory to store the article for some tasks, formatting, rendering, saving to the disk. After that, your Arena resets the memory instead of free them. So when next user post another article, if the article is shorter, it will never need to alloc, so no heap syscall is needed. This is very convenient and helpful when you are sure you need to frequently do something (a loop), with an almost known max size of memory. Like the article, you expect every user will not write a supper long article. But what if some users still write some super long articles, but most users don’t. Can we have a way to return the pages back to the system? Yes. The reset function of the ArenaAllocator also has a option called retain_with_limit: usize. This means that you can only retain a certain amount of the memory, and free the rest. For example, you estimate that most of the users’ articles will not be more than 4096 bytes, so after every submit, you call arena.reset(.retain_with_limit: 4096). If occasionally a crazy guy write 5096 bytes, it will release the 1000 bytes back to the system. So in this way, it guarantees that your memory will not continually grow.

These are 2 major examples that how people usually use Arena here.

In Rust community, those data driven people using their own Generational Arena or IndexTree to fool the borrow checker is essentially doing the similar thing (this is why they also call it “arena”). Or Rc and Arc people just make Rust to a GC language. Even by then, they still need to group something together to reduce the IndexTree and/or Rc… And functional people doing HttpClient.add_header().add_body().add_url().add_extra_header().add_port().set_timeout().send().unwrap()… this just abusing the clone(), how many times you call the functions, how many full struct clone(), which also mean alloc the whole struct and free the old whole struct syscall, will happen. When your struct includes heap fields, such as String, Vec, HashMap, etc., you can’t copy the struct in Rust, and you need to add the clone to derives. When you do this kind of functional style programming, they are essentially just like a struct with an allocator, then allocator will create a new struct then destroy the old one

I think Rust and Zig and even C/C++ will cross the path eventually if people really want to write some good programs. They are basically doing the same thing. After doing a lot of C and Zig, when I see people’s Rust codes, I know what is under the hood… (many times it is in a good way though, not necessary to be negative.)

8 Likes
31

I have written arena allocators in C before and understand how they work very well. There are still plenty of pitfalls unfortunately.

For example, you can still shoot yourself in the foot when data needs to escape the original arena and it includes pointers:

const std = @import("std");

const Inner = struct {
    value: u64,
};

const Outer = struct {
    id: u32,
    inner: *Inner,
};

fn makeDanglingObject(
    backing_allocator: std.mem.Allocator,
    output_arena: std.mem.Allocator,
) !*Outer {
    var temporary_arena_state =
        std.heap.ArenaAllocator.init(backing_allocator);
    defer temporary_arena_state.deinit();

    const temporary_arena = temporary_arena_state.allocator();

    // Both objects initially live in the temporary arena.
    const inner = try temporary_arena.create(Inner);
    inner.* = .{
        .value = 0x1234_5678,
    };

    const outer = try temporary_arena.create(Outer);
    outer.* = .{
        .id = 42,
        .inner = inner,
    };

    std.debug.print(
        "inside function: outer={*} inner={*} value=0x{x}\n",
        .{ outer, outer.inner, outer.inner.value },
    );

    // The returned Outer itself lives in the caller-provided output arena.
    const result = try output_arena.create(Outer);

    // Shallow copy: result.inner still points into temporary_arena.
    result.* = outer.*;

    return result;

    // temporary_arena_state.deinit() runs here.
    // result remains alive in output_arena, but result.inner is dangling.
}

pub fn main() !void {
    // var debug_allocator: std.heap.DebugAllocator(.{
    //     .safety = true,
    //     .never_unmap = true,
    //     .retain_metadata = true,
    // }) = .init;
    // defer _ = debug_allocator.deinit();

    // const gpa = debug_allocator.allocator();
    
    // Not detected with debug allocator because addresses are never re-used
    
    const gpa = std.heap.page_allocator;

    var output_arena_state = std.heap.ArenaAllocator.init(gpa);
    defer output_arena_state.deinit();

    const output_arena = output_arena_state.allocator();

    const object = try makeDanglingObject(gpa, output_arena);

    std.debug.print(
        "after return:    outer={*} inner={*}\n",
        .{ object, object.inner },
    );

    // 💥
    std.debug.print(
        "uaf:             id={d} value=0x{x}\n",
        .{ object.id, object.inner.value },
    );
}

Like yeah, obviously escaping pointers beyond the lifetime of its allocator is going to have problems, but I’m trying to demonstrate that it can be trivial to accidentally shoot yourself in the foot still. I’ve seen it firsthand with production-grade code used on billions of devices I’ve worked on which used arena allocators, and was responsible for hardening it :slight_smile:

I imagine that at the scale of a JS runtime, this would be extremely challenging to debug. On my machine I couldn’t even get this to reliably reproduce a segfault either.

Arenas historically (as I learned them) are useful for just ensuring memory is pre-allocated, spatially close, and for avoiding heap fragmentation. You don’t need heap metadata for the weird 8-byte allocation, and for some scenarios make a lot more sense than the system allocator. Maybe I’d just need to see someone redesign a JS runtime with arena allocators in mind to see how they solve the problem more clearly.

More traditional arena allocators obviously exist in Rust: bumpalo 3.20.3 - Docs.rs. I hadn’t really heard of indextree before but that makes sense.

Arena allocators make lifetimes easier but they aren’t a magic bullet. I’m really curious who has been popularizing them more recently as I’ve been seeing it brought up more and more often.

3 Likes
32

“Rust supports cross-language link-time optimization between C/C++ and Rust, which enables inlining across programming languages (how cool is that!!).”

Doesn’t Zig also support this?

33

yes, a zsf told them about it when they complained too :3

it used to be enabled by default, but there were bugs so it was disabled, but you could still manually enable it

4 Likes
34

I haven’t played around with it myself recently, but I remember this video where Andrew demonstrates LTO between C and Zig (in this case it’s Zig code being inlined into a C function).

35

Great explanations and I learned a lot. Thanks IceGuye.

1 Like
36

The blog post, among many others, shows that the AI bann within the Zig project is anachronistic and outdated. It will actually harm the project, not help.
Using AI myself at work and other open source projects, it is not only an incredible productivity boost, but also a quality boost. You just need to now how to use it. I.e., let it write unit tests and let another agent review the changes and of course: review the code yourself. The keyword is “harness”.

1 Like
37

you are completely missing the point of the ban

8 Likes
38

As I mentioned in some other thread it was Anthropic’s PR stunt. Some good read here: Zig Creator Calls Spade a Spade, Anthropic Blows Smoke

4 Likes