Customization mechanism

Brainstorming
1

I am trying to figure out how to do customization in Zig. Customization as in having format function in a struct will have std.fmt.format using it instead of the normal code flow. This however does not work for third party types.

So on a more general level, in Zig, what is the idiomatic way to provide a way for users to add customization that allows both in and out of struct functions. For example, let’s say I have a library that reads/writes from/to SQLite databases. I want to let users have a way to specify the table and column names, maybe column types also, I can specify that if a struct has functions like table_name, column_name, I will use it instead. However, for third party types, users obviously cannot modify the structs to have those functions, how should the library then handle this case?

1 Like
2

You can wrap the type and put your own format function in it, like so:

const ThirdPartyType = struct{
  data: u32,
};

const CustomFormated = struct{
  payload: ThirdPartyType,
  pub fn format(value: @This(), otherArgs: IForgotTheSignature) !void{
    //Do your custom formatting here.
  }

  pub fn wrap(value: ThirdPartyType) @This(){
    return .{.payload = value};
  }
};

//Usage:
pub fn main() void{
  const value: ThirdPartyType = .{ .data = 2};
  std.debug.print("{}", .{ CustomFormatted.wrap(value)});
}
6 Likes
3

If your library exposes a function that creates a generic type, you could add a customization option that accepts a user provided function that maps types to structs with implementations, something like this:

const std = @import("std");

const database = struct {
    pub const TypesOverride = @TypeOf(NoTypesOverride);
    pub fn NoTypesOverride(comptime T: type) type {
        return T;
    }

    const Config = struct {
        types_override: TypesOverride = NoTypesOverride,
    };

    pub fn Database(comptime config: Config) type {
        return struct {
            const Self = @This();
            const DB = struct {};
            db: DB,

            pub fn init() Self {
                return .{ .db = .{} };
            }
            pub fn deinit(_: *Self) void {}

            pub fn readCell(_: *const Self, data: anytype) void {
                const T = @TypeOf(data);
                const Override = config.types_override(T);

                const column_name = Override.columnName(data);
                std.debug.print("name: {s}\n", .{column_name});
            }

            pub fn readCellOverride(_: *const Self, data: anytype, comptime Override: type) void {
                const column_name = Override.columnName(data);
                std.debug.print("name: {s}\n", .{column_name});
            }
        };
    }
};

pub const thirdPartyTypes = struct {
    pub const Image = opaque {};
    pub const Number = opaque {};
};

const customDefinitions = struct {
    pub const Image = struct {
        pub fn columnName(_: *thirdPartyTypes.Image) []const u8 {
            return "Image";
        }
    };
    pub const Number = struct {
        pub fn columnName(_: *thirdPartyTypes.Number) []const u8 {
            return "Number";
        }
    };
};

pub fn CustomTypes(comptime T: type) type {
    return switch (T) {
        *thirdPartyTypes.Image => customDefinitions.Image,
        *thirdPartyTypes.Number => customDefinitions.Number,
        else => T,
    };
}

pub const User = struct {
    pub fn columnName(_: User) []const u8 {
        return "User";
    }
};

pub const SmallImage = struct {
    pub fn columnName(_: *thirdPartyTypes.Image) []const u8 {
        return "SmallImage";
    }
};

pub fn main() !void {
    const DB = database.Database(.{ .types_override = CustomTypes });
    var db = DB.init();
    defer db.deinit();

    const image: *thirdPartyTypes.Image = undefined;
    const number: *thirdPartyTypes.Number = undefined;

    db.readCell(image);
    db.readCellOverride(image, SmallImage);
    db.readCell(number);

    const user: User = .{};
    db.readCell(user);
}

I found this pattern in readCellOverride especially helpful, with that you can give the user full control and let the user specify a type that contains all the information to do specific things, without having to create a generic type that accepts a configuration.

This also could be generalized to have a type that describes how a table is mapped with potential column name overrides, etc.

You then also could create functions that help in the construction of specific types that can be passed to these override functions.

Hopefully this is helpful, if this doesn’t quite match what you had in mind, then it would help to get some example code that describes your pattern, I sort of can imagine it, but I don’t quite know specific things like what things are comptime available and what is run time and it doesn’t seem helpful for me to try and guess that and then suggest something that doesn’t fit your api.

4 Likes
4

yeah, I think this is what needs to be done in Rust also.

5

I don’t have any code to demonstrate the idea. However, in C++, you can have something like template specialization, or ADL to add customization for third party type. Python has a pattern for adding entries to a global variable that will be used by the library to customize its behavior. So I have in my mind something like

// somelib.zig
var config_table = std::ArrayList(Config).init(some_allocator);

// usercode.zig
const ThirdPartyType = @import("somelib").CustomType;
var config_table = @import("somelib").config_table;

config_table.append(.{ ... }); // some config

And since the user code and the library code communicate through config_table, a user can control the behavior of the library by modifying the config_table accordingly.

1 Like
6

In zig you can do something similar with @import("root") which will import the root file of the user code (usually main.zig). The user can then put a struct with all the config there. The advantage of this over your solution is that it allows options to be comptime known. A great example for this pattern is std.log.

3 Likes
7

I think runtime customization like this should be possible, but I think this would require that the implementation converts types to ids or vtables.

However when having these sorts of customizations I want the library to be able to use comptime known things to create specialized code at comptime and because mutation of comptime variables is constrained to local scope, I don’t think a mutation based interface is actually helpful for aggregating the configuration in the comptime-config-case.
I think for that case, Zig prefers a scheme where you compose different configs into one config which then can be passed as a single config, so you could imagine a merge funtion Merge(comptime configs:anytype) type and then all the different configs need to be included in one place merged into one (getting a compile error if they aren’t logically consistent (if that is possible)) and then you end up with a single comptime value that is used as config.

I think this mutation based interface doesn’t play well with the goals of incremental compilation, but I think aggregations and having a place that merges all the different things allows the same thing, I think it requires you to be more explicit about the accumulation of the config (and accumulate more like a functional merge, instead of global mutation appending), but I don’t think there is currently a way to avoid that for comptime configurations.

8

I think this is what I’m looking for. I just want to write this in my own words to check if I understand this correctly.

The library will provide a function to construct the “behavior object” that the user can pass in their options to create the behavior according to what they want, then the library can either accept the “behavior object” in each of its functions (the tedious approach) or have another function that accepts the “behavior object” which then generates a struct that contains all the functions which will behave according to the behavior specified by the user.

Is this correct?

9

yeah, it’s just something I wrote out quickly just to demonstrate the effect I want, not necessarily the actual mechanism. Of course everything being resolved at comptime is more desirable. I asked this mainly for the comptime customization, not vtable based approach.

1 Like
10

Just to mention it: another possibility would be to use either build options (build options are used to generate a module internally) or modules, so basically using the build system and imports to customize behavior.

Both of these allow you to change what values will be accessible through a specific @import that is used by the library, so the library could be given a specific build option, or also require you to create a module and plug it into the library to be used by it. You also could have two instantiations of a specific library-module that use different modules for this ‘config-module’, which then get imported via different import names.

I imagine something like this could be useful if you write for example a ui library that defines a generalized drawing api, and then requires that the user plugs in a specific implementation that uses an api like opengl, vulkan, etc. to implement that drawing.

1 Like
11

While both of these are valid approaches, that’s not what I mean.

Maybe it helps to show you an example, similar to the example you had above:

// user main.zig
const some_library_options = struct {
    const objectConfigs: []const some_library.Config = &.{
        .{...}, // config 1
        .{...}, // config 2
    } ++ some_other_file.extra_config; // Could be spread over multiple files
};

// somelib.zig
const config_table = blk: {
    const root = @import("root.zig");
    // Check if the config exists, using an empty slice as the default.
    if(!@hasDecl(root, "some_library_options") break :blk &.{};
    if(!@hasDecl(root.some_library_options, "config") break :blk &.{};
    
    break :blk root.some_library_options.config;
}; // Now it's comptime-known!

const Config = struct{...}; // or union, whatever you want

// Then in all your functions you can just reference the global config to change behavior.
fn sampleFunction(...) ... {
    comptime var behavior: fn(...)... = default_behavior;
    inline for(config_table) |config| { // Inline to unroll the loop at compile time, you can also use a comptime block to be extra sure.
        if(config.special_condition) {
             behavior = config.custom_behavior;
        }
    }
    behavior(...);
}
1 Like
12

what confuses me here is @import("root.zig"), how does the library have access to this?

13

Ah sorry, I messed up, it’s just @import("root")
root is a module which gets automatically created when you make an executable or library build step. It’s an alias to the root file of the project.

It’s one of the three default modules std, builtin and root: Documentation - The Zig Programming Language

3 Likes
14

ah ok, makes sense, thank you very much.