What is the logic behind translate c function shortcuts? Is it bugged?

Explain
1

So I have recently realized translate-c generates methods on translated opaque structs (realized way too late because ZLS currently does not show them and I rely way too much on LSP). And it is very nice, because it also provides short names where the struct name prefix is stripped.

But I noticed that it seems really inconsistent. Some functions are arbitrarily missing, or have the name different than expected.

Example 1

Look at this translated libnl3 nl_cache_mngr

pub const struct_nl_cache_mngr = opaque {
    pub const nl_cache_mngr_add = __root.nl_cache_mngr_add;
    pub const nl_cache_mngr_add_cache = __root.nl_cache_mngr_add_cache;
    pub const nl_cache_mngr_add_cache_v2 = __root.nl_cache_mngr_add_cache_v2;
    pub const nl_cache_mngr_get_fd = __root.nl_cache_mngr_get_fd;
    pub const nl_cache_mngr_poll = __root.nl_cache_mngr_poll;
    pub const nl_cache_mngr_data_ready = __root.nl_cache_mngr_data_ready;
    pub const nl_cache_mngr_info = __root.nl_cache_mngr_info;
    pub const nl_cache_mngr_free = __root.nl_cache_mngr_free;
    pub const add = __root.nl_cache_mngr_add;
    pub const add_cache = __root.nl_cache_mngr_add_cache;
    pub const add_cache_v2 = __root.nl_cache_mngr_add_cache_v2;
    pub const get_fd = __root.nl_cache_mngr_get_fd;
    pub const data_ready = __root.nl_cache_mngr_data_ready;
    pub const info = __root.nl_cache_mngr_info;
};
pub extern fn nl_cache_mngr_alloc(?*struct_nl_sock, c_int, c_int, [*c]?*struct_nl_cache_mngr) c_int;
pub extern fn nl_cache_mngr_add(?*struct_nl_cache_mngr, [*c]const u8, change_func_t, ?*anyopaque, [*c]?*struct_nl_cache) c_int;
pub extern fn nl_cache_mngr_add_cache(mngr: ?*struct_nl_cache_mngr, cache: ?*struct_nl_cache, cb: change_func_t, data: ?*anyopaque) c_int;
pub extern fn nl_cache_mngr_add_cache_v2(mngr: ?*struct_nl_cache_mngr, cache: ?*struct_nl_cache, cb: change_func_v2_t, data: ?*anyopaque) c_int;
pub extern fn nl_cache_mngr_get_fd(?*struct_nl_cache_mngr) c_int;
pub extern fn nl_cache_mngr_poll(?*struct_nl_cache_mngr, c_int) c_int;
pub extern fn nl_cache_mngr_data_ready(?*struct_nl_cache_mngr) c_int;
pub extern fn nl_cache_mngr_info(?*struct_nl_cache_mngr, [*c]struct_nl_dump_params) void;
pub extern fn nl_cache_mngr_free(?*struct_nl_cache_mngr) void;

You can see the short names are really nice for consumer side to just call mgr.free() instead of c.nl_cache_mngr_free(mgr). But mgr.poll(x) is missing. Why?

Example 2

This is much abbreviated translation of BoringSSL BIO_METHOD

pub const struct_bio_method_st = opaque {
    // Abbreviated full function names
    pub const BIO_meth_get_create = __root.BIO_meth_get_create;
    pub const BIO_meth_set_create = __root.BIO_meth_set_create;
    // All shortcuts, not abbreviated
    pub const new = __root.BIO_new;
    pub const write = __root.BIO_meth_get_write;
    pub const ex = __root.BIO_meth_get_write_ex;
    pub const read = __root.BIO_meth_get_read;
    pub const gets = __root.BIO_meth_get_gets;
    pub const ctrl = __root.BIO_meth_get_ctrl;
    pub const create = __root.BIO_meth_get_create;
    pub const destroy = __root.BIO_meth_get_destroy;

};
// Abbreviated function declarations
pub extern fn BIO_meth_get_create(bion: ?*const BIO_METHOD) ?*const fn (?*BIO) callconv(.c) c_int;
pub extern fn BIO_meth_set_create(biom: ?*BIO_METHOD, create: ?*const fn (?*BIO) callconv(.c) c_int) c_int;

Here you can see there is a create shortcut. But there are BIO_meth_get_create and BIO_meth_set_create methods. I expected get_create and set_create shortcuts. Is this a bug, or a feature?

The ex shortcut seems even more arbitrary.

2

https://codeberg.org/ziglang/translate-c/pulls/270

You can see the rules in this PR.
In short, it first attempts to trim by matching the container name prefix; if there is no match, it takes the last segment after _ as the alias.
Additionally, if the resulting name conflicts with an existing declaration, the alias name will not be used (this is likely why poll doesn’t have one).

1 Like
3

I see, thank you, that explains everything perfectly. Sadly that makes it not very usable - i’d rather avoid the shortcuts completely rather than deal with these inconsistencies. I could deal with it if it was rare that something was missing or had unexpected name, but not if it is everywhere I use translate c…

I wonder if that could be improved upon.

For example the prefix could be attempted to be extracted from the function returning the opaque pointer of the type. In other words - search longest common prefix for the function instantiating and the method.

For the poll situation. I believe that is a libc function. Maybe libc should have its own namespace in the translated files…? Is it possible to put extern fn into a namespace? If that would be possible that would also clean up c.<cursor> completions alot.

Please let me know what you think of these ideas.

4

https://codeberg.org/ziglang/translate-c/issues/294

For example, this proposal suggests a --trim-prefix=foo option.

However, given the variety of situations in C libraries, as well as different naming styles such as underscores or camelCase, there may be no perfect translate rule.

Perhaps a way could be provided for users to use custom processing functions during translation (which could be compiled into an external tool program for invocation). This function would accept an agreed-upon input and output format, so that users could customize how function aliases are generated according to their own habits.

5

Let me elaborate on the idea for the new prefix detection rules.

  1. Check if some function returning the type has common prefix with the function.
pub extern fn BIO_meth_get_read_ex(biom: ?*const BIO_METHOD) ?*const fn (?*BIO, [*c]u8, usize, [*c]usize) callconv(.c) c_int;

// BIO_meth_new would be checked for common prefix because it returns ?*BIO_METHOD
// When detecting prefix for BIO_meth_get_read_ex, it would find BIO_meth_ resulting in get_read_ex name
pub extern fn BIO_meth_new(@"type": c_int, name: [*c]const u8) ?*BIO_METHOD;
  1. Check if some function accepting the type as parameter as double pointer [*c]?* has common prefix with the function.
pub extern fn nl_cache_mngr_add_cache(mngr: ?*struct_nl_cache_mngr, cache: ?*struct_nl_cache, cb: change_func_t, data: ?*anyopaque) c_int;

// nl_cache_mngr_alloc would be checked for common prefix because it accepts [*c]?*struct_nl_cache_mngr
// When detecting prefix for nl_cache_mngr_add_cache, it would find nl_cache_mngr_ resulting in add_cache name
pub extern fn nl_cache_mngr_alloc(?*struct_nl_sock, c_int, c_int, [*c]?*struct_nl_cache_mngr) c_int;
6

This is indeed a use case, but there are some problems.

Take PHP’s zend_string as an example: it has functions prefixed with the struct name, like zend_string_equals, zend_string_starts_with, which can be directly translated to equals, starts_with.

There are also member functions defined in different .h files, such as zend_fetch_class, zend_get_constant, zend_ini_get_value, php_safe_bcmp, etc.

And functions that return [*c]zend_string include zend_string_init(...) [*c]zend_string, php_get_uname, zend_get_callable_name, zval_get_string_func, and so on, which are also usually defined in different .h files.

Therefore, when your project’s c.h includes different PHP .h files, the longest prefix might be zend_string_ when #include "a.h" is done, and zend_ when b.h is included.

This results in: when you use a.h, you use starts_with, but once you include b.h, it becomes string_starts_with, which is a terrible development experience.

Double pointer [*c]?* may also have this situation.

1 Like