Advanced project ideas

If I see something like this, I instantly tell to myself - “oh, no, it’s just doing it wrong”. Give me a tool to form any MODBUS-RTU request and to parse any possible replies, but do not do no I/O for me. It’s general design flaw of almost every library I’v ever seen, no matter what protocol is beneath.

A protocol (at least an “application level” one) imply two absolutely different things:

  • message exchange pattern (simple ‘request/reply’, as in MODBUS-{RTU,TCP}, auth phase, async messages from server/slave, whatever)
  • data formats (are floats in BE/LE? are strings terminated by zero byte? how can you know if a complete message is received (is any data length in a header or is there a terminator byte)? )

My experience tells me that these two should/must never be mixed in a library, otherwise you’ll got “lots and lots of plugins/libraries”, which would be hard to maintain and develop.

Don’t mix flies with cutlets :slight_smile:

1 Like

Something I’d like to see is a Zig keyboard firmware framework, like qmk or zmk. It would be an excellent proving ground for using Zig in the embedded space, and Zig’s comptime features would be an exceptionally good fit for that kind of project.

There’s an article from a few years back where someone switched from Rust to Zig for their custom keyboard firmware, which is a solid exploration of what Zig could bring to the table.

There’s some prior art, albeit a bit quirky, one would take a different approach for a generalized framework. I run a heavily customized QMK firmware on my keeb, enough to have run into some of the limitations of the design. Specifically, it can send left or right with modifiers, but can’t apply them individually, so you can’t represent left-command-right-alt, for instance.

It would be a great fit for the language.

10 Likes

Parametric CAD (computer aided design) library like OpenSCAD.

If you really like pain:

  • finite element solvers for structural analysis
  • computational fluid dynamics
  • meshing tools
  • thermal simulation
  • circuit simulation like LTspice
2 Likes

The firmware in today’s digital device:

  1. UEFI in every PC
  2. TEE OS in every mobile phone
  3. SBI in every RISC-V SOC
  4. coreboot in every chromebook and many cloud servers
  5. u-boot in every linux SBC and embedd devices
2 Likes

I’d like to pitch an idea I’m chewing on recently: Something like Ansible in Zig.

Before I make my pitch, I want to describe what I like and dislike about Ansible, because that makes it easier to explain why I believe Zig would be a great choice to make something better and how that might help Zig.

Ansible is fun to use. Writing up Yaml files with tasks and seeing your servers being set up is great. The inventory is magic, they really did a wonderful job making it intuitive to write up configurations. Ansible has a huge number of built in and community modules for almost everything. Ansible, other than most alternatives works agent-less, meaning that you don’t have an ansible-server running on targets.

That’s the good stuff. Here’s the bad: Ansible requires the target to have a python installation (you may get around it, but practically you need it). Task execution is done by copying over python-tars which is slow and makes debugging impossible. Custom modules may want to use python dependencies, but you really don’t want to install them on targets. The whole module infrastructure is a mess, inconsistent and redundant. As great as the inventory is, it has huge problems with scoping. There is no good way to do logic on task level. Ansible pretends to be declarative, but it’s not.

Why Zig?

Zig has great support for cross compilation. If you need to deploy temporary code on a target to run complex operations (what Ansible does with Ansibalz), Zig can do that so much better. This would especially allow for sane dependency management - dependencies are compiled in, nothing to manage.

Zig can easily create self-contained executables without any runtime dependencies. Great for embedded devices which might not have a python runtime or never saw an update in a decade but still run the show.

Zig is easy to read and write. Maybe not as easy as python, but much easier than Perl or Rust (old and new school choices for such purposes).

Rust’s safe memory argument doesn’t matter for short running operations, scripts can just leak memory (that’s probably not good marketing).

Zig has easy access to C libraries, making a lot of functionality that’s too big to be ported in time easily accessible.

What does Zig get from that?

I see Ansible tasks as a vocabulary. It provides a whole lot of primitives like “create a user”, “copy a file”, “install package”, “add to ssh-host”, “update block in config file”. Having them available in the form of Zig libraries is a huge win, because they have proven to be useful and while they are easy to implement, doing it right is time consuming. That’s exactly what makes python and ansible so sexy.

If there is some coordination between such an Ansible-in-Zig project and the zig standard library, the Zig library could evolve into the most convenient system level development tool. And other than crappy python scripts or yaml tasks that can’t be reused, the result would be highly optimized code that can of course be used outside of automation contexts.

The Ansible task pattern with a check mode (dry-run) could also be a great pattern being used in such a library, especially if it’s combined with transaction support. Something like this is only feasible if it’s part of an early design, and remaking Ansible in Zig would be a great opportunity to do that from the start.

Community catching: There are a whole lot of people working in automation. I did that as a side gig. I love Ansible, but I feel like it’s caught in its history and it can’t really evolve much further. It’s also caught in its corporate context where monetization dictates how it can evolve and that’s making it even slower and less attractive. And yet, this python scripting project gets enormous amounts of community working hours. I would love to see that time being invested in a much more sustainable environment. If such a project would succeed, a lot of that success would stick to Zig. That’s a big “if”, Ansible is strong and it’s hard to establish such a project, but then again, that’s the same for any grass root project.

What’s done

I reserved the domains “zensible.dev” and “zensible.org” to have a place to host stuff and a name. It’s just the domain, no content yet.

Zensible is obviously a reference to Ansible, the sensible Ansible done in Zig (plus some Zen).

Then I have a whole lot of ideas that need writing down and more importantly need to be challenged and discussed. I didn’t have much time lately and nobody to discuss it with, so this is really just an idea in search of some people to brainstorm over it. You’re more than welcome to join if this speaks to you.

3 Likes

Oh I forgot to mention one thing. I love Andrew’s approach to open source vs. corporate driven stuff. But I really don’t want to get into legal or administrative work, I always hated that and there is a non-profit Zig foundation that’s already doing the work.

It would be great if that would open up to projects like this, forming a kind of family where people can play with technology without projects being canibalized by corps or development resources kept busy with legal nonsense.

Zig is really awesome, both the tech and the social environment. Thanks guys!

3 Likes

The nice thing about ‘top to bottom’ programs (as in, program starts, does things, finishes, exits) is that they’re a natural fit for arenas. It’s pretty hard to screw up memory management when it consists of three lines to set up an ArenaAllocator and defer deinit.

Sounds like an interesting project all around.

2 Likes

Zig reset my way of thinking about life times and memory management. I did a lot of C ages ago, but then spend decades working mostly with GC’ed langauges. But I also keep remembering how Knuth was implementing things like TeX and Metafont with static memory (at least in part). It’s nice to get back to that mindset where you talk to a computer and make it do stuff, instead of talking to paradigms.

I love the concept of arena allocators, because its existence motivates me to think of the life time of processes (in terms of workflows, not OS). Then you quickly end up having a whole lot of one-shot tasks where memory management is incredibly easy.

The “just leak memory” was a jab at Rust’s nose. Cool that you caught it :slight_smile:

I find it mind boggling how often Zig does/gets things exactly right.

Thanks! Would love to see it become real.

1 Like

POC may be done without rewriting Ansible

Ansible modules are units of code that can control system resources or execute system commands. Ansible provides a module library that you can execute directly on remote hosts or through playbooks. You can also write custom modules.

Custom modules may be Binary

From Ansible 2.2 onwards, modules may also be small binary programs

top to bottom’ programs

This way we will

  • check whether zig is suitable for this kind of tasks
  • provide possiblity to use zig in Ansible (ZDK for Ansible)
  • study problem domain

WARNING - all my Ansible knowledge is based on video :joy:

@mutech , may be you can open a separate topic?

2 Likes

That’s correct, but this is only one of many problems. If it was the only or just the biggest problem, it would be easy to solve.

The reason why I like this idea enough to push it is because I believe that Zig and Ansible’s problem domain synergize so well.

I set up a web site for Zensible, there is nothing much there yet, but there will be a link to some forum in time, where this discussion can move to. I think it’s too specific to live in a Zig forum, at least outside of this “project ideas” topic.

may be ZDK for Ansible will be first project on Zensible ?

I’m about to dump some ideas randomly on the web site until there is some suitable forum or wiki. I guess once I’m tired of that, it’s easier to see where I want to go.

I’m not set on competing with Ansible. If what I want is doable inside the scope of Ansible, I would be more than happy. But from discussions with some Ansible guys (great people working there, btw.) I guess this is difficult, considering that they are focusing on their ecosystem that is bound by all the blessings and curses of “backward compatibility”.

1 Like

I’ve created a new post from scratch to move the discussion there and keep this post focused on general brainstorming. So other people with other ideas can keep posting here and not worry that it turned into a Zensible post.

@mutech @g41797 @mnemnion

4 Likes

A post was split to a new topic: Context Application/Library design