Compute graph and variant arraylist

Hello there,
I’ve just started learning zig and am trying to wrap my head around the best way to handle arrays of unknown ( at compile time ) object pointers and polymorphism in general.

I want to build a ‘stream compute’ directed graph system where nodes are general functions that process some inputs and produce outputs.
Edges connect outputs from one function to inputs of another. The inputs and outputs would typically be values such as ints and floats but could be slices of value types or arraylists / tensor type datastructures.
The graph would be built at runtime.

I’m thinking that each node would be a struct with some some input struct, some output struct, and a process() function.

Then I would have a tagged union type of pointers to these, that would use compile time duck typing on the process() function, and store all the nodes of the graph in some arraylist ?

One concern I have with this approach is that the number of processor nodes might grow very large in time and explicitly adding them one by one to the union would be quite ugly. I wonder if there is a better way to ‘register’ them with the union using comptime magic.
Similar question for the edges, where I would want to eventually support various types of broadcasting, casting as well as potentially multiplexing and demultiplexing, which would result in many possible types.

I’d very much appreciate some pointers towards the best direction for this !
I should mention that performance would be important and that down the line I would want to parallelise the processing of nodes based on their topological sort.

3 Likes

Hello, and welcome to the forum!

Sounds like you’re building a computation graph - neat stuff! I’m working on something similar myself with a tensor library so I’m always happy to chat with people about this type of stuff.

Anyhow, more to your point - have you looked into interfaces? It sounds like you’re looking for more “genuinely” polymorphic behaviour. Here’s an article: Zig Interfaces for the Uninitiated, an update - Zig NEWS

The main takeaway there is the @fieldParentPtr builtin method. I’d recommend starting there and see where it takes you. There is a change in idioms and the article I sent you outlines that. It’s still worth knowing about the built-ins and what they do.

3 Likes