MLX.zig: Llama 3.2 in Zig

Showcase
,
1

MLX.zig is a Zig language binding for Apple’s MLX framework.

What it does

MLX.zig enables building and running LLMs directly in Zig without external build tools. The repository includes:

  1. A pure Zig build system that handles all C/C++ dependencies
  2. Direct bindings to MLX’s C API for optimal performance
  3. A working transformer-based language model implementation
  4. An LLM tokenizer using PCRE2 for text processing

Example usage

// Initialize components
var tokenizer = try Tokenizer.init(allocator, null);
var transformer = try Transformer.init(allocator, null);

// Encode and generate text
const input_ids = try tokenizer.encodeChat(allocator, "You are a helpful assistant.", "Hello world");
const output_ids = try transformer.generate(input_ids, 20);

// Decode the result
const output_text = try tokenizer.decode(output_ids);

Acknowledgement

The build system is based on Erik Kaunismäki’s zig-build-mlx approach.

4 Likes