Using Zig with MPI

Help
, ,
1

Hi,

I tried using zig with MPI (specifically the openmpi implementation) before, but their mpi header seems to be too complex for the zig compiler. It fails due to layers upon layers of preprocessor magic. For example, when I try to compile a simple program:

const std = @import("std");
const mpi = @cImport(@cInclude("mpi.h"));

pub fn main() !void {
    _ = mpi.MPI_Init(null, null);
    var nranks: c_int = 1;
    _ = mpi.MPI_Comm_size(mpi.MPI_COMM_WORLD, &nranks);
    var myrank: c_int = 0;
    _ = mpi.MPI_Comm_rank(mpi.MPI_COMM_WORLD, &myrank);

    std.debug.print("Process {d} of {d}\n", .{ myrank, nranks }); 

    _ = mpi.MPI_Finalize();
}

Also needed to add

exe.linkSystemLibrary2("mpi", .{ .preferred_link_mode = .dynamic });
exe.linkLibC();

to the build.zig
Zig cannot resolve the type of the MPI_COMM_WORLD constant

cimport.zig:2110:73: error: cannot load opaque type 'cimport.struct_ompi_predefined_communicator_t'
pub inline fn MPI_COMM_WORLD() @TypeOf(OMPI_PREDEFINED_GLOBAL(MPI_Comm, ompi_mpi_comm_world)) {

In this post @glycerine suggested to preprocess the header manually. So i ran

gcc -E /usr/include/mpi.h -o ./src/mpi.h

I also had to add exe.addIncludePath(b.path("./src/")); to build.zig. Otherwise the systemheader would have been used.

zig seems to be able to resolve the type of the constant now, but I get the following error, which do not know how to resolve.

install
└─ install zigmpi
   └─ zig build-exe zigmpi Debug native 2 errors
src/main.zig:2:13: error: C import failed
const mpi = @cImport(@cInclude("mpi.h"));
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    main: src/main.zig:5:9
    main: /usr/lib/zig/std/start.zig:660:37
    3 reference(s) hidden; use '-freference-trace=5' to see all references
/usr/lib/gcc/x86_64-pc-linux-gnu/15.1.1/include/stddef.h:465:22: error: use of undeclared identifier 'nullptr'
  typedef __typeof__(nullptr) nullptr_t;
                     ^
error: the following command failed with 2 compilation errors:
/usr/bin/zig build-exe -lmpi -ODebug -I /home/fuhl/test/zig/zigmpi/src -Mroot=/home/fuhl/test/zig/zigmpi/src/main.zig -lc --cache-dir /home/fuhl/test/zig/zigmpi/.zig-cache --global-cache-dir /home/fuhl/.cache/zig --name zigmpi --zig-lib-dir /usr/lib/zig/ --listen=- 
Build Summary: 0/3 steps succeeded; 1 failed
install transitive failure
└─ install zigmpi transitive failure
   └─ zig build-exe zigmpi Debug native 2 errors
error: the following build command failed with exit code 1:
/home/fuhl/test/zig/zigmpi/.zig-cache/o/50943a64fece2134d34221ec7b54e8ad/build /usr/bin/zig /usr/lib/zig /home/fuhl/test/zig/zigmpi /home/fuhl/test/zig/zigmpi/.zig-cache /home/fuhl/.cache/zig --seed 0x149e961f -Zd893115abc5843c2

I cannot preprocess all headers in the chain.

Does anyone know how to resolve this issue?
Thanks

2

You shouldn’t need to run a preprocessor before using translate-c. It has a full C front-end already, and you’re probably losing information by using gcc.

I’d start with looking at the the translated output to understand why you’re getting those errors. The error should point to a specific file in the cache which contains the translated code.

3

I encountered the same problem when working with MPI in Zig. What did was to move the definitions that translate-c couldn’t handle into a separate c compilation like this:

// src/zmpi.h
#ifndef ZMPI_MPI_H
#define ZMPI_MPI_H

#include <mpi.h>
// add whatever is needed (predefined datatypes, reserved communicators, null handles, etc.)
extern const MPI_Datatype ZMPI_CHAR;
extern const MPI_Comm ZMPI_COMM_WORLD;
extern const MPI_Comm ZMPI_COMM_SELF;

// src/zmpi.c
#include "zmpi.h"
const MPI_Datatype ZMPI_CHAR = MPI_CHAR;
const MPI_Comm ZMPI_COMM_WORLD = MPI_COMM_WORLD;
const MPI_Comm ZMPI_COMM_SELF = MPI_COMM_SELF;

Then update the build.zig to include the zmpi.c file:

const mpi_module = b.addModule("zig-mpi", .{
    .root_source_file = b.path("src/mpi.zig"),
    .target = target,
    .optimize = optimize,
    .link_libc = true,
});
mpi_module.addIncludePath(b.path("src"));
mpi_module.addCSourceFile(.{ .file = b.path("src/zmpi.c") });

// Run `mpicc -show` to figure out how to link mpi.
const mpicc_show = b.run(&.{ "mpicc", "-show" });
var mpicc_show_it = std.mem.tokenizeAny(u8, mpicc_show, &std.ascii.whitespace);
_ = mpicc_show_it.next(); // skip cc
while (mpicc_show_it.next()) |arg| {
    if (std.mem.startsWith(u8, arg, "-I")) {
        mpi_module.addSystemIncludePath(.{ .cwd_relative = arg[2..] });
    } else if (std.mem.startsWith(u8, arg, "-L")) {
        mpi_module.addLibraryPath(.{ .cwd_relative = arg[2..] });
    } else if (std.mem.startsWith(u8, arg, "-l")) {
        mpi_module.linkSystemLibrary(arg[2..], .{});
    }
}

Then include the zmpi.h file when running translate-c:

const mpi = @cImport({
    @cInclude("mpi.h");
    @cInclude("zmpi.h");
});

You can then use mpi.ZMPI_COMM_WORLD instead of mpi.MPI_COMM_WORLD().

For reference, this is the same approach that rsmpi took.

3 Likes
4

Unfortunately it does not work out of the box. The openmpi header is too complicated for zig to work with. This was an attempt to work around this.

5

Thanks, I will try that.
But I wish it would work out of the box.