High-performance AI/LLM implementation idea - thoughts?

Hi Ziggit!

I’ve been exploring how Zig’s performance characteristics could work with large language models, specifically the DeepSeek architecture. Had this idea: what if we could leverage Zig’s compile-time metaprogramming and SIMD features to create a really efficient LLM implementation?

The result I came up with from exploring this / a bit of drafting is DeepZig V3 - a proposal for implementing DeepSeek V3 in Zig: https://github.com/Triex/DeepZig-V3

Key ideas:

  • Mixture-of-Experts (MoE) with compile-time expert selection
  • SIMD-optimised attention mechanisms
  • Cross-platform deployment (including WASM)
  • Zero-allocation inference paths where possible
  • Zig’s error handling for model operations

I’m really curious what others think about such a direction. Has anyone else been looking at ML/AI implementations in Zig? I feel like there’s huge potential here that hasn’t been fully explored yet.

Zig’s not my specialty, so I’m mostly putting the idea out there for now.

Also open to collaboration/joint learning or exploration if anyone finds this interesting! I feel the Zig community could build some really impressive AI infrastructure if putting heads together.

What do you all think - viable direction, or am I overthinking this? :sweat_smile:

1 Like

Are you an idea guy or somebody who actually wants to do the work?

Was that repo created this way?

  1. you fork deep seek
  2. use an LLM to generate a new readme
  3. you do some manual edits and shuffle a bit of things around
  4. you add a new logo

If so, why should we be hyped about it, when it looks like a bunch of ai slop / vaporware?

Have you written a single line of Zig code, have you run a Zig program at all?
Why is all your code inside a readme instead of some .zig files?

It doesn’t look to me like you invested any real effort in this, beyond asking an LLM to generate you a new readme.

We aren’t interested in interacting with ai slop here, we want to talk to people, we aren’t here to read a bunch of vaporware generated by LLMs.

If you have a genuine interest than create some runnable poc project, or hand write some original thoughts and your reasoning about what you want to do and how, instead of presenting us with LLM blabber about how it plans to rewrite deepseek in Zig.

13 Likes

My goodness. AI… and blockchain?

That’s, that’s. Both things!

Sorry, couldn’t resist. Welcome to the forum.

You’ll find that we like programming computers, and we know the difference between that, and hype, AI-generated logos, ‘documentation’, and other slop and/or social signalling. Speaking for myself, I’ve found the interest and talent for programming computers to correlate, sharply and negatively, with the kind of hype-beast impression which your Github presence radiates.

But. If you want to learn Zig, welcome aboard. You’ll find that it’s a language which respects and rewards craftsmanship, careful design, and a deep and genuine understanding of how computers actually work.

If you want to write a blockchain in Zig, you can do that. There are companies using it quite successfully for machine learning as well.

But Grok ain’t gonna do it for you.

7 Likes

You’re totally right, and I apologise for the poor first impression…

I got excited about the potential intersection of Zig ++ AI efficiency and jumped straight to “here’s my idea” - without doing the actual work of properly diving into Zig first. That was backwards.

I guess I should have started by simply asking “has anyone explored ML implementations in Zig?” and actually written some Zig code before “proposing” anything

Would it be more appropriate to delete this post and come back when I’ve actually built something minimal in Zig? Or maybe start with some basic questions about Zig’s SIMD capabilities for learning purposes?

I’m genuinely interested in the language - just went about this the wrong way

2 Likes

I would like to add up through my zig journey, to try to go quick and disturb nobody I tried maybe 4 times to ask questions to chatGPT about zig and how can I tell it’s the first time really I saw chatGPT really struggled with a programming language.

Since zig is kind of a newborn language but also a living one through its golden community (us), LLM didn’t have time to train on us and also can’t really understand on github which is new and legacy zig code.

Using LLM here force me to go back to my old habits of searching answers (searching forum post manually and github issues) and this make me realize how much good it was to be bring in a world where it’s actual people that know things and not just a kind of semi omniscient LLM.

Like you said zig is really a language really outside of those LLM sphere, of course you can build AI products with it but still you are the one who found a katana and decided yourself to take it, master the art of swing to build your idea and decided to sharp it by talking and finding new ideas with the community.

5 Likes

I think ZML is an LLM inference stack written in Zig. Maybe you can check them out after learning a bit about the language.

8 Likes

This is one of my favourite things about this forum. There’s a lot of people who are deeply involved in the language and working with hard programming challenges in general. I feel like I’ve learned a lot about the mechanics of creating a programming language and language design in general from spending time here.

11 Likes

@ben Thanks for the ZML pointer! Planning to study their implementation after I actually learn more language/memory management fundamentals.

@gungun974 Your point about LLMs struggling with Zig def resonates. I’ve experienced this too - tried to get help with basic concepts and got outdated or wrong responses. Forced me back to reading actual docs and forum posts, which honestly felt refreshing. The katana analogy is accurate - you don’t pick up a precision instrument and immediately start proposing new techniques.

@tsdtas Exactly this. There’s real value in learning from people wrestling with actual problems rather than statistical patterns. I need to spend more time lurking and learning before contributing.

Going to step back and focus on fundamentals instead of jumping ahead to big ideas or project management. Thanks for the course correction, everyone.

5 Likes

Maybe it is good to develop a deep learning framework using zig, similar with burn/candle in rust .