Casting a structure .ob_base = @as(c.PyObject, @bitCast({ .ob_refcnt = 1, .ob_type = null, })),

Back in 2020 I created a Python package setuptools-zig that can be used as an extention to the standard way of packaging binary Python extensions (as source or pre-compiled). I once updated this to change the supported Zig version from 0.7 to 0.10, and as well as for newer Python releases, and wanted to do so now again.

It is not so much that the tool itself changed, as a setuptools/pip plugin it is pure Python, but the (Zig) examples need to be adapted and the invocation methods have changed.

The adaptation of the Zig examples is no problem for Python versions 3.7-3.11, but 3.12 throws me a problem, that I haven’t found a solution for yet. In the C example that corresponds to the Zig example, I do this:

static struct PyModuleDef zigmodule = {
    PyModuleDef_HEAD_INIT,
    "sum",
    NULL,
    -1,
    methods
};

the nasty bit being the macro PyModuleDef_HEAD_INIT that is at the head of this structure, its definition depends on other #define’s other macros, etc, spread out over multiple .h files from the Python installation. So back in 2020, trying to trace this, I flipped the table and gave up, and used zig-translate to give me the expanded Zig code :slight_smile:
That code looks like this (the commented code is the original, replaced by simpler working Zig code, and I added c., it from the const c = @cImport({....} of Python.h at the top of the Zig file. ):

pub var zigmodule = c.struct_PyModuleDef{
    .m_base = c.PyModuleDef_Base{
        .ob_base = c.PyObject{
            .ob_refcnt = 1, // @bitCast(c.Py_ssize_t, @as(c_long, @as(c_int, 1))),
            .ob_type = null,
        },
        .m_init = null,
        .m_index = 0,  //  @bitCast(c.Py_ssize_t, @as(c_long, @as(c_int, 0))),
        .m_copy = null,
    },
    .m_name = "sum",
    .m_doc = null,
    .m_size = -1, // @bitCast(c.Py_ssize_t, @as(c_long, -@as(c_int, 1))),
   ....

( the line containing .m_name = "sum", corresponds to the "sum", line in the C example. Above that is the expansion of the PyModuleDef_HEAD_INIT macro).
As indicated that all still works well for 3.7-3.11

Using Python 3.12’s Python.h file this part of the translation gives:

pub var zigmodule: struct_PyModuleDef = struct_PyModuleDef{
    .m_base = PyModuleDef_Base{
        .ob_base = PyObject{
            .unnamed_0 = union_unnamed_6{
                .ob_refcnt = @as(Py_ssize_t, @bitCast(@as(c_long, @as(c_int, 1)))),
            },
            .ob_type = null,
        },
        .m_init = null,
        .m_index = @as(Py_ssize_t, @bitCast(@as(c_long, @as(c_int, 0)))),
        .m_copy = null,
    },
    .m_name = "sum",
    ...

so somewhere in the bowels of the Python include files the first field of PyObject changed from a simple size type to a union of said type and a vector of two smaller units.

Inserting c. prefixes, I have not been able to get this compile. For one is that union_unnamed_6 is not public (as can be seen from the translated C code), so the compiler complains. And if I just use the old code, the compiler compilains that the c.PyObject structure has no field .ob_refcnt which makes sense.

Ideally I would not have seperated example sources for Python 12, because AFAICT the actual bits have not changed. I tried to do the following

        .ob_base = @as(c.PyObject, @bitCast({
            .ob_refcnt = 1,
            .ob_type = null,
        })),

but get an error on the comma after .ob_refcnt =1, telling me: “expected ';' after statement” but inserting a semicolon there before or instead of the comma gives different errors.

How can I cast those two fields as a c.PyObject as an acceptable value for .ob_refcnt?

The syntax for struct construction is StructName{…} or to infer the type .{…}. You are missing the dot, after @bitCast(

Using the struct name:

.ob_base = c.PyObject{
    .ob_refcnt = 1,
    .ob_type = null,
},

by inferring the struct name:

.ob_base = .{
    .ob_refcnt = 1,
    .ob_type = null,
},
2 Likes

Thanks, adding the dot makes sense, and I think I even tried that at some point.
With the dot the compiler erros on the struct not having a guaranteed in memory layout (i.e. should be extern struct {}).
Since I didn’t know how to solve that, I discontinued that line. I’ll try to make an extern struct definition with those to fields and cast that.

Since I could not get inserting the force to extern struct in the @bitcast part, I ended up doing

const ModuleBase = extern struct {
     ob_refcnt: u64 = 1,
     ob_type: ?*u8 = null,
};

pub var zigmodule: c.struct_PyModuleDef = c.struct_PyModuleDef{
    .m_base = c.PyModuleDef_Base{
        .ob_base = @as(c.PyObject, @bitCast(ModuleBase{})),
        .m_init = null,
        .m_index = @as(c.Py_ssize_t, @bitCast(@as(c_long, @as(c_int, 0)))),
        .m_copy = null,
    },
    .m_name = "sum",
   ...

That makes everything compiler nicely for Python 3.7 through 3.12