Idiomatic way to hide 'internal' implementation details

Building a small graphics engine, I’ve caught myself doing kind of pub fn _internal(...) from time to time to incapsulate stuff. But this cannot be the way to go in Zig, right?

How do you separate your internal implementation details from the public facing API (besides vtables) ? And do you? Since Zig doesn’t have per-field privacy, it seems very intended..

3 Likes

I usually structure Zig packages similarly to Rust’s lib.rs .

  • Put the public API in root.zig .
  • Re-export only the types/functions intended for users from root.zig .
  • Import internal modules directly from within the package instead of re-exporting them.

That gives me a clear public API surface without needing language-level private , and it’s worked well for me so far.

3 Likes

Do you have an example?

As an example, here’s one of my own package project:

The package exposes its public API from src/root.zig , while implementation details such as src/message/message_impl.zig are imported internally and are not part of the public API.

1 Like

I use handle-based API approach

I have no simple example, but I’l try to explain

let’s say you have mailbox.zig
with

const _Mailbox = struct {
    poly: polynode.PolyNode,
    mutex: Io.Mutex,
    cond: Io.Condition,
    list: std.DoublyLinkedList,
    len: usize,
}

pub const MailboxHandle = *anyopaque;

pub fn close(mbh: MailboxHandle) void {
 here you cast mbh to *_Mailbox and so on
}

in another source

pub const mailbox = @import("mailbox.zig");
.....
mailbox.close(mbh);

Pay attention

  • it’s just explanation
  • snippet extracted from the source
  • not compiled
  • not tested

You implementation of _Mailbox

  • is completely hidden
  • without vtable

This is nice, but it doesn’t actually do the ‘cherry-picking’ of fns. It just re-exports a bunch or types and errors. Maybe I’m wrong.

Is there any reason not to use a distinct opaque type instead there?

pub const MailboxHandle = opaque {
    pub fn close(mbh: *MailboxHandle) void {
        const mb : *Mailbox = @ptrCast(@alignCast(Mailbox));
        ...
    }
}

Also does this mean the _Mailbox has to be be owned/allocated by the library internals, in order to pass a pointer to the user?

3 Likes

A library I enjoyed using, Sokol has it such that the only objects it passes to you are indeces (handles). I think it’s a great strategy.
Related:

3 Likes

Are you looking for something like Rust’s pub(crate) or Java’s package-private?

Yes. pub(crate) is actually very nice, but we obviously are not getting such feature (not complaining) and need to emulate something similar with the features we have.

I’m using sokol-gfx too :slight_smile:

But I believe this concept of handles actually solves a bit different problem. Giving out raw pointers is generally a bad idea, handles solve this. I’m talking more about actually hiding APIs which are internal-only.

not simpler but better to see real source

WIP btw

Prefixing an underscore to “private” names is indeed not very idiomatic in Zig. If for some reason simply documenting that “this is an internal member and should not be used” is insufficient and you really want to go the extra mile, placing all the private implementation details with a child type is an easy soltion:

pub const Foo = struct {

public_field: u32,

pub fn publicFunction(self: Food) void { ... }

const Internal = const struct
    private_field: u32,
    fn privateFunction(foo: Foo) void { ... }
};

Foo has access to the Internal declarations so long as it is within the same file.

3 Likes

If I understand it correctly, you want to hide functions then that are pub to be shared inside the module but don’t want them to be public outside it?

Maybe you will like this idea:
let’s say it’s called “myLib” with it’s root being myLib.zig

pub const api = struct {
    pub const shareThis = shareThis;
};

pub fn shareThis() void {}
pub fn dontShareThis() void {}

Then when you import it:

const myLib = @import("myLib.zig").api;

Or you could have this API file be a separate file, same thing. downside Is you write every function you want to share manualy upside is that once you actually do that it reads very very clean, because you have every declaration in one place.

2 Likes

If the internal functions are only needed in a single file:

  1. (single file) you can declare them in that file without making the functions public.

If you need the function from multiple files you can:
2. declare them publicly in a file you import directly *
3. or in a whole other module B you import *

*Then don’t publicly re-exported the result of that import (or connected types) and only use the type internally which has those pub fns instead of using the type in parameters / return values / fields of your public api.

If you use 3. you can use createModule instead of addModule in the build system, that way you don’t expose your internal module to users of your package.

Or alternatively you could:

  1. document that the function doesn’t belong to the public api (but I would prefer 1.,2. or 3. most of the time)
  2. let people use internals if they want to, this is basically ‘don’t hide anything’ but then people need to understand what they are doing, if they choose to work with internals (could be good for very lowlevel code where it is difficult to find a good api that always works efficiently)

If you have fields you don’t want people to access you could export them only as opaque types or handles. (However opaques only hide the details, the memory is still accessible, if you can guess or find out how to interpret it (but this only matters if somebody is expected to write adversarial/malicious code))


I wonder whether it would be worth it, to add a feature to the build system, to have two lists of imports for every module one with imports that are allowed to appear in its own api and another which are only allowed internally, then the compiler would be able to give a compile error if an implementation-module-import-ed type appears in the public api (which is exported by the root file).

Might make it easier to separate api and implementation, without making a mistake and leaking implementation details.


That said, so far it seems to me that implementation details are often public in Zig, which can be a good thing, because it sometimes makes it way easier to create your own implementations which sometimes can extend existing implementations. So I would say 5. or maybe 6.‘only rarely hide something’ is also a sensible approach, because it can be annoying having to copy a whole bunch of code to be able to re-use it (or needing to fork it just for that). For example also ask yourself whether somebody could have a valid reason for wanting to wrap one of your implementations, re-using most of it directly to create a slight variant of your implementation.

6 Likes

zigs pub is already quite fine-grained, you can hide things by not re-exporting them:

// foo.zig
pub const x = 4;
// bar.zig
const foo = @import("foo.zig");
_ = foo.x; // can access public x
pub const x = foo.x;
// baz.zig
const bar = @import("bar.zig");
_ = bar.foo; // compile error, foo is not public.
_ = bar.x; // x is publicly re exported

const S = struct {
    const z = 3;
};
_ = S.z; // works cause `S` is defined in same file
7 Likes

I’ve marked the option I liked the most as the solution, but it is not necessarily the most idiomatic. In fact, there isn’t an idiomatic one.

Just pointing out that that solution exposes the internal api still

If you put the things you don’t want to share outside the module into a separate file, and import that directly where needed, then it wont be exposed.

You can also avoid having to .api to get the actual api.

1 Like

I see. Sounds reasonable to use a separate file