Carrying dependencies

Usually such questions don’t matter.
You write til hit the wall and then refactor.
but my wall is almost here.

I have to carry many dependencies til the very end and it feels the amount keeps growing.
everything starts from an allocator, but then:

  • long living allocator (for data living beyond the current request lifecycle)
  • caches
  • io
  • application config
  • runtime content (config derived from the hardware)
  • trace id
  • logger
  • zstd context and its pool

and perhaps many more coming.

Currently I’m too lazy to carry them and I live a few as global (config, logger) but some I have to carry having more than one instance.

What is your solution usually? Any useful patterns I miss to apply?

2 Likes

You can make a context type to bundle commonly passed things.

You would want to avoid giving access to things that a function/type shouldn’t need, so it shouldn’t contain everything, you could even have multiple context types.

2 Likes

it’s was the first idea, but exactly as you pointed my concern to create a god object that hard to track what and why has access to it.
it’s technically makes me loosing control on the dependencies

1 Like

I use Mach and I like how they do it. Basically you write code as “Mach Modules” which is a little extra comptime metaprogramming over a normal file-level struct.

Functions in a “Mach Module” get parameters injected automatically. This means you can’t (or shouldn’t) call them directly, but there’s an indirect (my_mod.run(.my_fun)) way so you get the auto-parameter injection when you need to do a manual call.

So this can be nice since not all functions need all objects, and instead of a “context” object, you just pass in what you want to use, and provide it in your metaprogramming. Of course downsides is as said above.

And thinking about how I would describe it, it’s kinda like you have the context object in one metaprogrammy spot and pick out what you need and pass it in as functions need.

1 Like

In my opinion there is no inherently bad thing about globals. Most of these things don’t change throughout the lifetime of most of the program, so you are just making your life needlessly harder if you carry them around everywhere. e.g. Io, global config, hardware info, some allocators are probably something you setup once and use everywhere.

Of course there is some gain from seeing at a glance “this function does IO and allocation”, but unless you actively want to limit those sites, I don’t think it matters a lot for general purpose applications.

I do however recommend to still pass the allocator in some cases, since there are good benefits from it: It allows using different allocators for e.g. data structures, and it can also be used as a signal that you might need to free the result you are getting (at least in my codebase allocators are mainly passed when the caller is responsible for the result, and all temporary internal allocations are handled with a cheap global threadlocal allocator)

5 Likes

I respect that you came to this opinion by maintaining a large codebase over a long period of time. However, I have done the same and arrived at the opposite conclusion: globals make code significantly harder to read, and particularly difficult to refactor. Speaking from experience, when a collaborator uses global variables, it’s extremely frustrating to edit their code with any sort of confidence.

Addressing OP: just put what you need in a struct and pass it everywhere.

19 Likes

Ad I mentioned above it sounds even worse than globals, creating god objects remind me how people in Go find ability of the context to hold a key value.

I do like passing an allocator as an argument, it’s important, e g cached objects needs to survive the request lifetime or being just reset, not freed.

But not all 12 friends

Maybe @IntegratedQuantum didn’t write it as clearly, but I think what is meant here is:

There’s nothing bad about a global context with a very limited set of things that are needed in most places.

In other words: a few useful global variables defined in the same module, after careful consideration.

I consider this good/best practice, even though global variables generally are clearly a code smell.

When I first started programming, I often used god objects, but now I prefer to painstakingly carry these parameters, as each one has a worthwhile reason to be passed along.

If you’re really sure that all the procedures in a module necessarily need all these parameters, or you’re just really tired of dealing with them, you can abstract them into a ‘local god’.
I think the design of a ‘God object’ also requires some thought. If you’re passing it as ctx: *God, you might want to find a way to redesign it to allow passing ctx: *const God or ctx: God.

1 Like

IMHO (if your code base is ‘closed’ e.g. you have full control over it, e.g. not a library):

Define a context object which is like a global registry/database, and pass a handle to that context object once at startup into each subsystem’s initialize function (but not into each individual function which needs data from the context).

PS: or alternatively: define a per-system context object with just the subset of context data this system needs, and then populate per-system-context in the init call from the global context object… but tbh this sounds like overkill - I would just let the system pick what it needs from the global context in the system’s init function instead. It may improve ‘readability’ though since when looking at the system’s init call, you already know what dependencies the system requires.

That way you still have the freedom to inject different context objects (e.g. for mocking in tests), but you still don’t need to carry the dependencies into every little corner of the code base, since everything is resolved during the initialization phase.

For reusable libraries I would (probably) use the opposite approach: pass just the required state into each function and let the library user worry about writing a little shim which adheres to the user’s coding style.

FWIW, this problem is exactly why I would like filesystem-like permissions in programming languages. I want to put all state into a single, globally accessible nested struct, but then want to define access rights for certain parts of the code to certain parts of the global state struct.

E,g, the gfx subsystem should have full read/write access to state.gfx.*, but other systems would only have read-access, or no access at all. This would also apply to module interfaces. E.g. currently I can only say “this function is public to everyone”, but I’d like to define more granular rules like “this function is callable by subsystem X but not subsystem Y” (this would be the equivalent of the “executable” access right in file systems, while data would have “read” and “write” access rights).

6 Likes

Can you elaborate on why god objects seem worse than globals? Yes, you may pass in unnecessary context. But with globals you have access to that same unnecessary context, it is just not as obvious, which seems to be worse.

1 Like

Not the OP, but when you work in a team, such god objects usually spiral out of control. People tend to add things to the god object that they need in some random part in their code, and at some point you get to a situation where a subsystem which really shouldn’t have access to some specific data in the god object suddenly uses such an ‘illegal part’ of the data.

It’s the equivalent of spaghetti code, just for data (eg. when all code has access to all data, all code will eventually access all data).

IMHO there’s no strict rule. Things that are completely fine when working alone or in a small tight-knit team will not work in a bigger team with differing skills and opinions.

3 Likes

Hm, I understand, but doesn’t the same happen with scattered globals?

1 Like

Hm, I understand, but doesn’t the same happen with scattered globals?

Yeah same problem, but there’s some nuance:

Read-only public globals are generally fine. Read/write globals are fine as private data within a module.

Also simply adding public setter/getter function to access those globals is not a solution :wink:

In the end it’s all about defining the boundary of a module or subsystem. First think about what the public interface should look like, only then worry about the internal implementation, and don’t let implementation details leak into the public interface.

I think one possible advantage of code that includes global variables is that it’s hard to refactor and port. So even if it’s the same kind of makeshift code, once you consider porting code with global variables, you quickly realize you have to design it to be more correct. On the other hand, porting code that uses a god object isn’t that difficult, so it’s easy for its code smell to affect other projects…

  1. most of the fields in “ctx” have different lifetime, like allocators (one arena buffer comes from a query request, another is a “global” one) or caches that are ARC managed objects (not to give cache eviction task kill it while the reference is alive).

  2. some of the are not thread safe as zstd ctx, so I need a “copy of ctx per task”.

  3. it feels not obvious, but to me data in the ctx is easier to access, autocomplete helps to tell you “you have it right there” and one doesn’t need to think log to access it, while for global you need to search codebase, read it, think if you really should access it this way

  4. hard to setup dependencies for tests, it will make me create tons of data I don’t need while for global I can do it once per parent test block package

If I understand you correctly, you mean, that with globals you can have a bit control over who has access to what. To me that sounds a bit like the private field discussion, where I am more on the “everything-should-be-accessible” side. I have to think about it. Thanks for the explanations!

For points 1. and 2. I would say you have the same issues with globals, or I do not understand why it is an issue for god objects specifically.

Point 3 makes sense to some degree, but to say “having to search the whole code base” is an advantage just feels wrong :smiley: I tried to pin down why, but was not able clearly express it yet
The disadvantages of that exact “feature” of globals are just outweighing this point by orders of magnitudes for me. On the other hand you can easily grab a global everywhere, whereas passing in context is more effort and by that logic would make you think twice.

Point 4 I do not fully understand, I will think about it. Thanks!

Well, maybe I’ll change my mind once my project gets more contributors. I can see that your approach is the right call for the compiler where you have one state object for each stage of the compiler with its own allocators.
It would be quite a mess if everyone could access any state of prior stages, and it is also quite useful for the community as it makes it possible to export parts of the code to the standard library (e.g. having access to the Ast is really useful, just recently I’ve used it for making a linter for my project).

However I think that for most applications it is not as easy to find such cuts, and especially in smaller few-person projects (which I assume is what most people on this forum end up making) it is more of a pointless chore to pass around stuff everywhere.

And yes (mutable) global variables are hard to argue about (and hard to test for), but so are giant context objects.
In my opinion people (maybe less in the Zig community, but in java I used to see this attitude quite frequently) too often repeat rules like “globals are bad” without actually understanding why they are bad, and that their alternative actually has the same drawbacks. Passing a single god context object absolutely everywhere falls into that category in my opinion.

5 Likes

The god object is the meta, honestly I’ve tried the globals early on in C because it was easy and convenient, but now i just dump everything in a big context, or app struct and that’s it, if you have issues with ressources your can make lock/unlock functions with unreachable inside works well for me

1 Like