Tofu - messaging for future Zib based systems?

Brainstorming
1

Recently, I’ve been thinking: how are we supposed to connect systems written in Zig?

Not everything is a web client, a broker-based system, or a part of some structured RPC flow.

Sometimes, you just want to exchange good old-fashioned messages—without worrying about transport-level details or heavyweight abstractions.

So I present to you tofu - a protocol and an asynchronous messaging library :

- Message-Based: Uses discrete messages for communication.
- Asynchronous: Enables non-blocking message exchanges.
- Duplex: Supports two-way communication.
- Peer-to-Peer: Allows equal roles after connection establishment.
- Streaming - "client", "server", bi-directional.
- Multithread-friendly - All APIs are safe for concurrent access.
- Memory management for messages - Internal message pool
- Backpressure management - Allows to control receive of messages
- Customizable application flows - Allows to build various application flows not restricted to request/response or pub/sub
- Simplest API - You do not have to bother with socket interfaces
- Callback enabled

All above implemented on plain Zig and std.posix. Nothing else.

Simplest way to got tofu approach - read tofu’s mantra

Or visit docs site

I’d like to hear your thoughts:

  • Is tofu a meaningful direction?

  • Are there use cases you’ve faced where something like this would help ?

  • Does Zig need a protocol like this

  • Or if existing ones already serve us well enough

This post is slightly doctored brainstorming post

Main difference - now you have code and documentation for discussion

1 Like
2

Do you have a (code) example of doing something useful with this? I’ve looked at your project multiple times and I don’t understand what problem is it solving.

1 Like
3

I have built systems out of ZeroMQ with MessagPack as the data format, both have Zig libraries for it (zimq and mzg). What advantages does tofu have over using such an approach?

1 Like
4

for many enterprise systems messaging is only small part of functionality

most developers don’t wont to know details of network api’s, callbacks, events

and other interesting for us staff

just send message and return back to main important for boss functionality e.g.

raster image processing

there are several zig libraries with aysnc io, coroutines, callbacks - it’s very important

for building of messaging infrastructure, e.g. tofu may be created on the base of one of them

but in production when messaging is not the product , developers prefere more

convinient software - tofu style

e.g. all commuinication (include proprietary distrubuted file system) in Digital Front End for this product was developed using the same approach.

Still there are not Zig based systems . Now we are working on low-level infrastructure

tofu tries to sove future problem - how to connect zig based systems

it’s the reason that this post has brainstorming category and not showcase

5

i did not say that tofu is solely way to build systems

developers should have the choice

tofu approach was “battle tested” in huge distributed system - i am pretty sure

that it also will be useful for zig’s system

e.g - possiblity to build different flows, “dialog” based development, simple apis