Compiler cooperation - Metadata sections from code

[strajabot]

I don’t quite know how to convey this idea without an example, so you’re getting one

DEBUG_LOG("Assert message {d}, {d}", a, b);

On embedded platforms that are memory constrained, you wouldn’t want to:

1. Store the template strings in device memory
2. Spend device time processing the string (might remove race conditions for example)

So you come up with the following Idea:

• Store the actual template string in some metadata section that is not going to be loaded on the device, saving on memory
• In the actual implementation , instead of spending time formatting the string, just push the pointer to the string in the metadata section to a buffer, and then push the arguments to the buffer too, saving on performance
• After the execution is done dump the buffer over JTAG or some similar protocol and reconstruct the whole log on the host using the metadata from the ELF file

When it comes to implementing something like this in C like languages, it becomes a lot of hacking using the preprocessor, static variables, gnu attributes, linker scripts and maybe something else I missed.

Over the last week I was wondering what an actually good design for something like this would look like in Zig: My mental conclusion is a compile builtin providing a API like the following:

const format_pointer = @push_meta(".log_meta", .{ format: "Assert message {d} {d}" });

I am looking to hear other peoples opinions on how this type of thing could be implemented

[Sze]

No direct discussion of the topic here, but still related:

Advice - deferred formatting for logging on embedded systems

Hello, I am learning Zig for embedded projects. I would like to experiment with deferred formatting for logging on embedded systems (similar to “Trice” or Rust’s “defmt”). I started working on something inspired by std.fmt, but the generated code size is too large (probably due to a poor use of inline or comptime). Could you give me some advice?

[ww520]

This is a good topic. Basically you want to separate the logging and the text rendering.

In system logging (e.g. Windows Event) or in language localization, typically you associate a message ID with a message template (in localization it could be one ID to a number of message templates, one for each language). When you log, you save a record of the message ID along with any parameters. The rendering of the text can be done later with looking up of the message template for the ID and applying the parameters to the template.

In the localization system I’ve done before, the message templates are in locale.json file.

[
  {
    name: name_for_use_in_code,
    id: 123,
    title: "English text {1}, {2}, and {3}",
    text: "Local language {1}, {2}, and {3}",
  },
]

The message ID can be collected with a build process and code-gen into a source code file with the name as the const name and id as the value.

e.g.

pub const MSG_LOG1 = 100;
pub const MSG_ERR1 = 101;
pub const MSG_ERR2 = 102;

For the logging, you can save the log as a fixed size record to save space.

const LogRec = packed struct {
    id: u16,                     // message ID
    flags: u16 = 0,              // bit flags, for log level, etc.
    params: [3]u32 = undefined,  // add more as needed.
};

Log.add(MSG_LOG1, LEVEL_LOG, 1, 2);
Log.add(MSG_ERR2, LEVEL_ERR, 99);

The records can be save in an ArrayList, ring buffer, or in Sqlite DB. You can download it later and have the log renderer to render them into text.