I’m not new to programming, but I’m new to Zig, and I’m looking for an open source project to join or start.
I love music, and although my main profession has been as a web software engineer, I’ve also got several years’ experience in live audio engineering for small concerts and speaking engagements. I want to work on software in the music or audio space, and I think Zig’s performance characteristics and potential for use in embedded environments make it a good fit (and also I just like Zig hah).
Is anyone currently looking for new contributors on a project in this space?
If not, I’m brainstorming the idea of building a FOSS digital audio mixer, inspired in part by this project (part 1, part 2) done for FOSDEM 2024, except more oriented to live operation than remote control. Having mixed on a handful of digital consoles in the past, I can say that they are A) very cool and B) proprietary as all hell. I would love to see (or be!) an open source competitor to tools like these.
I don’t want to work alone, especially not on a project this ambitious. Does anyone have opinions about these ideas? Any feedback, or suggestions for existing projects I could join, or any interest in joining me for a digital mixer-like project, would be much appreciated!
P.S. I am not interested in working on a project which allows LLM contributions. I have to review enough slop as it is at my dayjob
I’ve spent some of my weekend poking at the CLAP format, in hopes of eventually reviving my old arpeggiator plugin project and rewriting it from ugly Java-esque C++[1] to a nicer, more data-oriented Zig. It’s not ready for outside contributors just yet, but once (if?) I’ve got something working going, I’d be happy to start accepting patches from outside contributors.
(I also don’t want it to be CLAP-exclusive, it just seems to be the nicest API to start with)
Something I’ve been thinking about (and not trying very hard I must admit, by lack of time and skills) is programming a daisy seed with zig. This embedded platform is quite popular in the realm of DIY hardware digital synths/samplers.
I have a project that’s on hiatus that could make good use of some help! I’d be willing to pick it up again in the next couple of weeks if you’d like to collab.
It’s a declarative shading language geared specifically for live coding, that is, writing audio-reactive shaders live on stage. This is something I currently do with a JavaScript tool called hydra. My goal is to implement an alternative in zig which runs on desktop and uses Vulkan.
Being able to get good low-latency audio input with some additional processing is essential for audio reactivity: multi-channel input, FFT, beat detection (MIDI as well).
programming a daisy seed with zig. This embedded platform is quite popular in the realm of DIY hardware digital synths/samplers.
I can see why! I was looking at the Teensy 4.1 with audio shield as a potential embedded audio processing option, but the daisy seed looks way more tailored to the use case - 32bit quality, no audio shield required, plus look at all that RAM
This is neat indeed! I don’t use youtube music, but I’ll probably take a look at your source code anyway - I really like the TUI design, and even though you call the spectrum visualizer “fake” I still think it’s super cool
I’ve never seen anything like this before! I’m not familiar with shaders beyond “it’s code that makes visual effects I think”. Could you explain a bit more about what this project is?
writing audio-reactive shaders live on stage
This makes me imagine: you’re on stage with a computer, showing the video output on big displays, and you’ve got some music or a microphone or something jacked in which affects the video? And you’re live coding the shaders that represent the audio as visual effects?
Is that anything like what the actual use case is? If so, that is ridiculously cool and I simply must know which stages people do this on so I can go there haha
The group I am part of doesn’t record anything but here are some examples in 2 styles of performance:
We usually perform as duos, one doing music and one doing viz, but there’s many different fun combinations that people do.
As for my project (image blocks), it’s 3 parts:
Shader compiler & render graph
Signal processing (audio reactivity)
Editor integration
I’ve got the graphics part covered I think, but need help on the audio stuff.
The two fun things about writing a language for live coding is that the ratio of times you write vs read code are totally different than for code as engineering, and it’s important to have low latency from code-change to result.
This is why most live coding tools for music end up embedded in some sort of scripting language.
If you are interested in giving it a try, I will follow it closely - and hopefuly participate if I can.
Also, my first project learning zig was an attempt at recreating something like FAUST. It relied a lot on comptime to build an audio processing graph the way faust does. But in the end it was more verbose than Faust, and as unreadable
If you want to play with embedded electronics, for my day job I’ve created an audio system using an ESP32-S3 and a bunch of I2S DAC/amplifier chips, to control 16 speakers at the same time. Design is proprietary of course, but I’ve been thinking of making my own version for some time, using an ESP32-P4 instead.
Design a custom board, open-source and orderable from JLCPCB.
Write a hardware abstraction library to read from SD-cards, and write an audio stream to the I2S DAC’s.
In zig, specifically; the esp-idf already has an audio development framework, written in C.
Get some simple examples working.
From there you can go ham, do whatever you like with the data in between. The ESP32-S3 and P4 have dedicated instructions for FFT and such, so you can get a lot done with them.
I offer this because the audio development boards commonly available are… not very good. They use really old or weak chips when there is much better stuff available. I can design a much better board if it will encourage people to make cool stuff with it.
That would be an impressive project, but maybe a bit too involved, at least for me. For prototyping stuff, I’d rather use widely used hardware I can buy for $30 than a more powerful but niche project I have to source and assemble myself. but again, that’s just me!
you seem knowledgeable on the subject of audio processing boards - what do you think of the daisy seed3?
Regarding the seed3 board, I’d say it’s pretty good, but not perfect.
Long
It’s got all the essentials for a good audio development platform:
I2S and I2C interface for digital connection to external amps or microphones.
Yes the board has ADC’s and DAC’s, but you’'ll always get better results with an external chip because the sensitive analog audio lines don’t need to go near the micro-controller, clock crystal, and USB lines.
Honestly, way too much. It has ~1024 kB internally, and 64MB off-chip. In my experience, you can run an audio pipeline off of less than 20kB - including reading from fatFS, decode, ALC and I2S out.
You won’'t need the 64MB off-chip unless you want to store a big audio file in memory instead of an SD-card for some reason.
SDMMC for reliable, high-bandwidth access to an SD-card.
Plenty of GPIO’s and ADC’s
Good for buttons and potentiometers. You can use the ADC as a single audio input if you have to.
DACs for crude audio output.
Again, not as good as a dedicated chip, but you can do a lot worse than 12-bit.
SPI and I2C for interfacing with external modules.
I2C is good for small amounts of data (temperature sensor), SPI for large amounts (SPI-to-Ethernet adapter).
What it’s missing is a second I2S interface. While it is technically possible to use a single I2S interface for input and output simultaneously, it requires both devices to run at the same clock frequency and it becomes a massive pain to set it up properly. Much easier to have two I2S ports and dedicate one to your microphone(s) and the other to your codec(s).
If you’re only doing one or the other though, then it’s not a problem.
thanks for the detailed answer! regarding the 64MB of RAM, the use case is sampling / live looping / granular synthesis. for those use cases reading/writing from the sd card won’t cut it.
also, some processing are write-heavy (for example delay lines), which would destroy a sd card quickly, or parallel non-linear reads, which would be problematic even for a fast SDMMC…
