"If this file is cached/installed, then..." in the build system

Help
1

My embedded Linux application has a daemon for doing remote updates. It’s a simple server execving the application in debug builds and listening for file transfers on a socket. Not complicated, not mission critical, just some dev QoL stuff.

If I want to update the updater, I now have a shaving versioning issue.

The target is running version n, but the host just built version n+1.

Can I make the build system do something like “if the updater is cached, run it then build it, otherwise, build it then run it?”

2

You can make the build system do anything. Whether the build system is the right place to be orchestrating your dev environment is a different question.

3

Do all four of these pronouns actually refer to the same noun?

4

Yes.

It’s a Zig application with a runtime flag for which side is the target and which is the build host. Build it for the target, build it for the host, use the host build to upload to the target. Host can send simple commands to the target while the main application is running.

5

If you know any CAN based test/update harnesses that work cross-platform on 32 bit arm Linux and x86_64 Windows without vendor lock-in on the Windows side, I’m all ears. :slight_smile:

6

I wouldn’t call “host build” and “target build” the same noun, it remains unclear how you intend this to work.

That said, Zig’s caching should just take care of this for you, because it takes configuration options into account.

7

It remains unclear how you intend this to work.

I’m sorry for not expressing it clearly.

Here’s a longer description, hopefully being more explicit helps:

host is a developer laptop, device is the embedded device.

The build.zig file has a main application that only gets built for the device. We build various libraries that need to be loaded on the device for it to run as well. The build.zig file for the project also builds some tooling. Some for the host and some for the device.

One of those tools, call it harness for the sake of the example, facilitates communication between the two machines during development. harness --server <program> runs a server that forks off program into a child process, then accepts various remote commands. harness --client <command> [args] sends a command to the server.

When setting up a new dev machine or doing a clean build, we build harness for both the host and the device and load the software on the device offline. The device runs harness --server main after startup.

During active development, the build system has a zig build update step, where the host runs harness --client --upload-files [generated build artifacts], then harness --client --restart-server. Zig’s caching does work in this case - only artifacts with cache misses get rebuilt and transferred, the device restarts harness --server main, running the new version of the program.

It takes about three minutes and faffing about in several GUI tools to do the offline software update, whereas the slowest runs of zig build update take on the order of 30 seconds, so the dev cycle is much nicer with the update harness.

When there’s a cache miss on harness itself, I’d like to be able to do something like

[previous host version] --client --upload-files [new device version]
[previous host version] --client --restart-server
[new host version] --client --upload-files [artifacts]
[new host version] --client --restart-server

That way, the code for harness can have breaking changes applied without breaking the dev experience. As long as each version can transfer files to itself, it should work. That’s much easier to test than always ensuring no breaking changes between versions.

1 Like
8

Thanks, I understand you now.

This should be possible to express by using dependsOn correctly. A step b depending on another step a doesn’t force a to happen: it means that if b has to happen, and a also has to happen, then a will happen before b.

So it should be possible for [new host] to dependsOn [previous host], where [previous host] depends-on harness, and [new host] has the “active development” dependency as well.

So if [new host] has to happen, but the reason isn’t harness, then [previous host] should not run. If the reason is harness, then you get the order of operations you need.