Anyone using Apple CoreML ? With zig?

Help
1

Ok, so Apple has this new Mac mini with an M6 chip, and the most interesting bit is that it’s doubled the number of NPU cores.

But the hardware is hidden away, and only accessible through the CoreML api.

So I’m thinking .. there is some potentially cool hardware there that could make for some interesting uses. Like a game opponent that isn’t completely useless for example.

Anyone managed to experiment with touching CoreML in zig ? If so, would like to hear about potential pain points before diving off into another side quest.

Thx

2

I can share a bit of my experience working with Apple Libraries in C/Rust, which applies to Zig as well since they’re either Obj-C runtime or FFI based.

I think you have two ways to go about this:

  1. Swift ↔ Zig communication over FFI, you can either wrap CoreML with some Swift code that you call from Zig by using c calling conventions[1].
  2. You could optionally use the Objective-C runtime to interface with CoreML[2].

The problem with CoreML, is that it heavily depends on Apple’s own tooling. Take this guide for example, it defaults to using Xcode integration with the .mlmodel file that makes everything magically work. But I think working with these files in Zig will require you to check how these .mlmodel bundles work and what exactly is in them, what libraries the auto-generated code depends, how you’ll link and bridge things together[3].

Not sure how much it helps, but working with Apple libraries can be a bit challenging, let me know if you have any other questions I hope I can help with useful answers.

Good to keep in mind that I have only worked with CoreML once, a long time ago back when it was introduced, and back then I worked exclusively on getting it to work on Swift apps.

  1. This guide shows how you can call Swift from C, the Swift compilation steps are the same but instead of doing the clang step you’d have to play with Zig’s build system to link all the system frameworks that your swift code depends on, that would likely be Foundation, CoreML and the libraries in between. ↩︎

  2. You can check Mitchell Hashimoto’s zig-objc project. And this guide might be specially useful if you’d like to work around Apple’s Xcode magic. ↩︎

  3. After a quick search, I found a short article that might shed a light on this. ↩︎

2 Likes
3

Thank you for that, it’s very helpful information.

Yeah I guess doing the whole bridge through swift / objc is the best way to make use of any Apple APIs.

Means I can experiment first just using throwaway Swift code to work out what the npu is good for first, and whether it works the way I think it might work.

I think the painful bit will be working out how to build those Apple specific models. Will have a look through labelstudio and other ML tools and see if I can’t build a CV pipeline first .. or something.