New image processing library: Zignal

Hi everyone!

I work for a company where we use Zig in production.

In particular, we built a virtual makeup try on that runs on WebAssembly for the Korean makeup manufacturer Ameli. You can try it directly for yourself here. Don’t worry, we don’t store any images, since all the processing is done locally on the end device.

My employer, B factory, has agreed to letting me release the foundation library I developed for the virtual makeup. The library features:

I still haven’t uploaded all the features, but I couldn’t wait to share it with you.
Notably, the drawing and blurring parts are missing, since their current implementations are not generic enough to be in the library.

Here’s the library git repo: GitHub - bfactory-ai/zignal: Image processing library in Zig

Be sure to check the examples’ directory, where you can try the face alignment demo for yourself.

A little bit about me: I am a contributor to the dlib C++ library: http://dlib.net/
Some of the functionality in Zignal has been ported from dlib, from where I drew a lot of inspiration.
Other features were developed for Zignal first, and then ported back to dlib:

Finally, I am thrilled to share this and get some feedback: I am sure there are things that can be done better; do not hesitate to give me some advice! It’s still a work in progress.

62 Likes

Small update, I am porting back functionality from our internal code, now you can already blur images with Zignal. The demo has been updated accordingly

It supports box-blur because it’s really fast because it computes the integral image. It works on grayscale images with SIMD and Color images (with any kind of pixel type).

It’s an updated version of what I had here @floatToInt, @intToFloat gone - #29 by adria

But the thing I am most excited about is the new logo/mascot for the library.

I called her Liza:

  • it sounds feminine like the venerable Lenna - Wikipedia
  • it has a Z, so it feels Ziggy
  • it’s a substring of lizard :exploding_head:

15 Likes

good logo/mascot, nice

4 Likes

That’s very impressive! I will definitely be taking a look.

2 Likes

Another small update: I’ve set up GitHub Actions to host the documentation and examples, so that they can now be run from the browser.

4 Likes

Hi everyone.

Following up on my New image processing library: Zignal post from a few months ago, I’m excited to announce that Zignal has reached its first tagged release!

Here’s the TL;DR of this release:

  • 12 color spaces with seamless conversions (RGB, HSV, HSL, Lab, Oklab, XYZ, etc.)
  • Full linear algebra suite with SVD and PCA implementations
  • 2D drawing API with antialiased primitives and Bézier curves
  • Geometric transforms and convex hull algorithms
  • WASM-first design with interactive browser examples
  • Complete image I/O with native PNG codec and JPEG decoder (had to read the zigimg at some point, since I was getting stuck, parts of my understanding of JPEG was wrong)

Here are the full release notes if you’re interested: Release zignal 0.1.0 · bfactory-ai/zignal · GitHub

What pushed me to tag a release was the fact that I wanted to provide python packages, though.

I’ve spent the last week or so trying to get it to work with Python via native bindings using the Python C interface.
I know Ziggy Pydust exists, but I wanted to learn how to do it myself: write the native Python bindings, see which patterns repeat and can be automated with Zig’s comptime reflection, and slowly build my best effort python utils.
Before, I was always compiling a generic shared library with ctypes to bind it to Python.

Right now, I am building the wheels myself via CI, because I am using Zig 0.15-0.dev, and as far as I know, it’s not available in PyPI. At some point I’d like to add the ziglang dependency, so users can build the library themselves and get full native optimizations. It’s also nicer on the PyPI, since we won’t be uploading any binaries.

Let me know if you find this useful!

12 Likes

Sorry for spamming here, but I just tagged 0.2.0 with a pretty nice feature: terminal image display.

I am a in image processing guy, but I love working on the terminal (foot + kakoune + tmux)

I like having my program running on a split pane with zig build run-whatever --watch so whenever I make changes I can see if the program runs. However, working with images made me have to switch to an image viewer. Well, no more!

With Zignal, you can now do:

const std = @import("std");
const zignal = @import("zignal");
const Image = zignal.Image;
const Rgba = zignal.Rgba;

pub fn main() !void {
    var debug_allocator: std.heap.DebugAllocator(.{}) = .init;
    defer _ = debug_allocator.deinit();
    const gpa = debug_allocator.allocator();

    var image: Image(Rgba) = try .load(gpa, "../assets/liza.jpg");
    defer image.deinit(gpa);
    std.debug.print("{f}", .{image.display(.auto)});
}

That’s foot displaying the image with sixel. By default, the .format method on Image will progressively degrade, depending on what your terminal supports:

  • .kitty: it works on Ghostty
  • .sixel: it works on Foot (the default settings use an adaptive palette of 256 colors with dithering)
  • .ansi_blocks: it works on GNOME Terminal (▀)

But it also supports
.ansi_basic using spaces with background (image is stretched, but doesn’t require unicode characters
.braille for monochrome graphics

Here’s how you can customize the printing format:

    std.debug.print("{f}", .{image.display(.ansi_basic)});
    std.debug.print("{f}", .{image.display(.ansi_blocks)});
    std.debug.print("{f}", .{image.display(.{ .sixel = .default })});
    std.debug.print("{f}", .{image.display(.{ .kitty = .default })});
    std.debug.print("{f}", .{image.display(.{ .braille = .default} })});

Have fun!

16 Likes

Don’t apologize, awesome update! Look forward to test this out :slight_smile:

4 Likes

New version: 0.3.0

Highlights

  • font rendering of bitmap fonts (BDF/PCF, with automatic decompression)
  • improved the python bindings to be more on par with the main library
  • improved the documentation generation for the python bindings

zignal-python-demo

12 Likes

Zignal 0.5.0 is out!

While there are some new features, most of the effort was focused on exposing most of the Zig functionality into the Python bindings.
I would say, however, that a significant effort was put into the image convolution.
In a microbenchmark for sobel, I got these results:

  1. Zignal: 6.82ms - Fastest!
  2. Pillow: 7.12ms (1.04x slower)
  3. OpenCV: 7.78ms (1.14x slower)
  4. scikit-image: 14.53ms (2.13x slower)
  5. scipy: 28.72ms (4.21x slower)

Full changelog here: Release zignal 0.5.0 · bfactory-ai/zignal · GitHub

12 Likes

Zignal 0.6.0 has been released:

Highlights

Image Processing

Edge Detection algorithms

  • Shen Castan
  • Canny

Binary Image Operations

  • Otsu and adaptive mean thresholding
  • Morphological operations: erosion, dilation, opening, closing

Order-Statistic Filters

  • Median, min and max filters for edge-preserving blur

Image Enhancement

  • Histogram equalization
  • Automatic contrast adjustment

PNG & JPEG

  • Added baseline JPEG encoder
  • Fixed many bugs in the PNG encoder

Since it’s an image processing library, here’s a visual tease:

Code to generate that image:

const std = @import("std");

const zignal = @import("zignal");

const Image = zignal.Image;
const Rgb = zignal.Rgb;
const Canvas = zignal.Canvas;

pub fn main() !void {
    var debug_allocator: std.heap.DebugAllocator(.{}) = .init;
    defer _ = debug_allocator.deinit();
    const gpa = debug_allocator.allocator();

    var image: Image(Rgb) = try .load(gpa, "../assets/liza.jpg");
    defer image.deinit(gpa);
    var canvas: Canvas(u8) = undefined;

    var scaled = try image.scale(gpa, 0.5, .bilinear);
    defer scaled.deinit(gpa);
    std.debug.print("{f}\n", .{scaled.display(.{ .auto = .{} })});

    var edges: Image(u8) = try .init(gpa, scaled.rows, scaled.cols * 3);
    defer edges.deinit(gpa);
    const font = zignal.font.font8x8.basic;

    var sobel = edges.view(.{ .t = 0, .l = 0, .r = scaled.cols, .b = scaled.rows });
    try scaled.sobel(gpa, &sobel);
    canvas = .init(gpa, sobel);
    canvas.drawText("Sobel", .point(.{ 0, 0 }), @as(u8, 255), font, 3, .fast);

    var shenCastan = edges.view(.{ .t = 0, .l = scaled.cols, .r = 2 * scaled.cols, .b = scaled.rows });
    try scaled.shenCastan(gpa, .heavy_smooth, &shenCastan);
    canvas = .init(gpa, shenCastan);
    canvas.drawText("Shen Castan", .point(.{ 0, 0 }), @as(u8, 255), font, 3, .fast);

    var canny = edges.view(.{ .t = 0, .l = 2 * scaled.cols, .r = 3 * scaled.cols, .b = scaled.rows });
    try scaled.canny(gpa, 1.4, 75, 150, &canny);
    canvas = .init(gpa, canny);
    canvas.drawText("Canny", .point(.{ 0, 0 }), @as(u8, 255), font, 3, .fast);

    std.debug.print("{f}\n", .{edges.display(.{ .auto = .{} })});
    try edges.save(gpa, "edges.png");
}

Note how we only allocate the final image once (edges) and then access different views of it and directly output the resulting edge detections there.

Or using the Python bindings (trades ease-of-use for flexibility, each operation returns a new image, if you gotta go fast you don’t use Python, anyway):


import zignal

image = zignal.Image.load("../../assets/liza.jpg")
scaled = image.resize(0.5)
print(f"{scaled:auto}")

font = zignal.BitmapFont.font8x8()

sobel = scaled.sobel()
shen_castan = scaled.shen_castan(smooth=0.5, window_size=9, high_ratio=0.95)
canny = scaled.canny(sigma=1.4, low=74, high=150)

canvas = zignal.Image(scaled.rows, scaled.cols * 3).canvas()

sobel.canvas().draw_text("Sobel", (0, 0), 255, scale=3)
shen_castan.canvas().draw_text("Shen Castan", (0, 0), 255, scale=3)
canny.canvas().draw_text("Canny", (0, 0), 255, scale=3)

canvas.draw_image(sobel, (0, 0))
canvas.draw_image(shen_castan, (scaled.cols, 0))
canvas.draw_image(canny, (2 * scaled.cols, 0))
print(f"{edges:auto}")

canvas.image.save("edges.png")
12 Likes

Version 0.9.0 has been released!

This was the first code scandal in Zignal, named colorgate.

Before this release, Zignal used to have predefined backing scalars for each colorspace

  • Rgb/Rgba were u8
  • The rest (Hsv, Lab, Lch, Xyz, …) were f64

Also each color type had named methods to convert to all the other color spaces: .toRgb(), .toXyz()

After this change, all color types are generic on the backing scalar: we can now have

  • Rgb(u8), where each channel ranges from 0-255
  • Rgb(f32) or Rgb(f64) where each channel is assumed to be in the 0-1 range.
  • The rest of color spaces are also generic on float types: Hsv(f32), Hsv(f64)…
  • A new Gray(u8/f32/f64) type to make the API more uniform

As a result code that used to look like

const image1: Image(u8) = ...;
const image2: Image(Rgb) = ...;

Now looks like

const image1: Image(Gray(u8)) = ...; // Image(u8) is still supported, though
const image2: Image(Rgb(u8)) = ...;

Also, color conversions, which used to be:

const rgb: Rgb = .black;
rgb.toHsv().toRgb();

Are now:

const rgb: Rgb(u8) = .black;
rgb.as(f32).to(.hsv).to(.rgb).as(.u8);

While it’s a bit more verbose, we can now specify the backing scalar, which we are forced to do when moving from an integer-based color space into a floating-point-based one.

After this change, Zignal color operations can now compile into SPIR-V (which only supports f32).

This was one of the achievements of the first Osaka Zig Day.

7 Likes

Thanks for sharing all the updates! Some of my only C++ days were using OpenCV back in 2014 for a robotics competition. I’m interested to hear your thoughts on the dev experience overall for building a library like this in Zig. I’m a data scientist and want to dip my toes back in systems programming but can’t seem to pick a language: Odin vs Zig vs Mojo vs Rust… so many good ones to choose from!

1 Like

want to dip my toes back in systems programming but can’t seem to pick a language: Odin vs Zig vs Mojo vs Rust… so many good ones to choose from!

Can totally relate to this feeling! I bounce between Rust and Zig still, but Zig feels more explicit than Odin, and I don’t like pythonic syntax. YMMV of course, but I personally found Zig to scratch the systems programming itch of mine perfectly. Rust is also a terrific language, but the complexity gets to me after a while. With Zig, I understand every line.

3 Likes

I am biased towards Zig, of course.

I’ve been working with machine learning and image/video processing for over 10 years now, all of it in C++. Basically, the choice of Zig comes down to this quote from the main site:

“Focus on debugging your application rather than debugging your programming language knowledge.”

Having worked mainly in C++, Rust didn’t appeal to me; I didn’t want to work with yet another complicated, ever-evolving language. This is basically the moment I knew Zig was made for people like me: https://www.youtube.com/watch?v=43X9ia-qpds [07:00]

At that time, I was contributing a lot to the deep learning part of dlib (where layers in neural networks are stacked by nesting templates). C++ was introducing concepts, and I thought it would be nice to restrict the template <typename T> which caused all kinds of weird error messages when we used the wrong T. Rust had traits, but Zig had comptime and reflection. As showcased in the video, Zig could replicate the functionality by adding just a handful of extra lines [08:22]. That was an eye-opening moment for me.

I loved how Zig’s reduced but well-chosen feature set could be combined to generate more “features”.

I must say that I didn’t know about Odin until much later. I was mainly using Zig to write WebAssembly stuff, and I think Odin wasn’t great at that a couple of years ago. I don’t think I want to switch to a less explicit language (Odin has an implicit context). I like the “pure” explicit approach of Zig.

Also, the cherry on top is the Zig build system. I built a virtual makeup try-on library that runs in Wasm for the company I work for. I never had to think about other platforms, and just by changing the -Dtarget flag, I could get it running natively and on Android with no code changes.

But, to each their own. Just try to build a couple of projects in each language and see which one clicks the most for you. The main goal is to have fun!

12 Likes

Digging a bit deeper into Zig and seeing that there is no operator overloading. How has it been developing a numerical library in a language without it? It seems like it could make things quite verbose (and possibly confusing) when you have multiple operations chained together.

I thought it was going to be a bit annoying, coming from C++, but it was fine, actually.

I tackled the problem by using chainable operations, the only challenge was that when chainable operations can fail (due to memory allocation) then it’s annoying since there’s no .! (analog to .? or .*). So I was using tested (try (try (try...))) at the beginning.

After trying some stuff, I came up with a more ergonomic API for the zignal’s Matrix struct. You can just chain the operations, and the errors are stored internally and propagated through the chain, and only returned when calling .eval()

const result = try matrix
    .transpose()
    .dot(other_matrix)
    .inverse()
    .eval();

Here, all lines can fail for various reasons, and the error will only be reported by the .eval() call.
It’s not perfect, but it’s the best I could think of. Open to suggestions.

I am not sure it would be a good idea to have a .! as a shortcut to catch unreachable. Since catch unreachable is very explicit about what’s going on, since .! would be too easy to abuse (like Rust’s .unwrap()).

7 Likes

ah @joshualeond you can check the full implementation here:

There’s also SMatrix for comptime known dimensions (the S could be for static or “small” matrix). Which doesn’t have to deal with memory allocations, so the API is nicer. But seeing that Builtin Matrix type · Issue #4960 · ziglang/zig · GitHub was accepted, I might remove that struct completely when it gets merged.

1 Like

But I think that it would be nice to turn try ErrorUnion to ErrorUnion.try so that instead of writing (try ErrorUnion).func() you would write ErrorUnion.try.func().

Or something along those lines.

1 Like

I came up with the very same solution to this problem. In debug it even grabs the offending call stack into a pre-allocated buffer, that’s probably overkill for matrixes, though if you allow dynamically sizes arrays you might want that too.

1 Like