Building wasm library with Zig 0.16.0

Help
, ,
1

Hello ziggit community,

I’m upgrading a small library to Zig 0.16.0 and it mostly works. However I cannot get it to compile for wasm32-freestanding target. So I tried to make a minimal reproducible example like so:

$ zig init
$ zig build -Dtarget=wasm32-freestanding

The above build fails already with:

~/zig/zig-x86_64-linux-0.16.0/lib/std/Io/Threaded.zig:2064:45: error: struct ‘posix.system__struct_7663’ has no member named ‘getrandom’
and
~/zig/zig-x86_64-linux-0.16.0/lib/std/posix.zig:90:27: error: struct ‘posix.system__struct_7663’ has no member named 'IOV_MAX’

(Note that the same commands work with Zig 0.15.2).
Even after removing juicy main and setting exe.entry = disabled in build.zig somehow Io.Threaded gets pulled into the exe/wasm library. And it looks like wasm32-freestanding does not provide all the posix stuff that std.Io.Threaded needs. I honestly don’t fully understand what’s going on or even how to fix it.

I guess my question is: How to compile a zig library for wasm32-freestanding in 0.16.0?

Thanks!

2

I tested this and ran into the same issue. It seems like you have to be very careful with what you pull into your binary for WASM. Even having a std.debug.print in your code will cause Io to be pulled in, which is what you are probably noticing in your case.

If main looks like the following, it should compile correctly.

const std = @import("std");
const log = std.log.scoped(.main);

const mod_name = @import("mod_name");

pub fn main() !void {
    log.debug("Hello!", .{}); // log seems to be fine, but not `std.debug`
}

test "simple test" {
    const gpa = std.testing.allocator;
    var list: std.ArrayList(i32) = .empty;
    defer list.deinit(gpa); // Try commenting this out and see if zig detects the memory leak!
    try list.append(gpa, 42);
    try std.testing.expectEqual(@as(i32, 42), list.pop());
}

test "fuzz example" {
    try std.testing.fuzz({}, testOne, .{});
}

fn testOne(context: void, smith: *std.testing.Smith) !void {
    _ = context;
    // Try passing `--fuzz` to `zig build test` and see if it manages to fail this test case!

    const gpa = std.testing.allocator;
    var list: std.ArrayList(u8) = .empty;
    defer list.deinit(gpa);
    while (!smith.eos()) switch (smith.value(enum { add_data, dup_data })) {
        .add_data => {
            const slice = try list.addManyAsSlice(gpa, smith.value(u4));
            smith.bytes(slice);
        },
        .dup_data => {
            if (list.items.len == 0) continue;
            if (list.items.len > std.math.maxInt(u32)) return error.SkipZigTest;
            const len = smith.valueRangeAtMost(u32, 1, @min(32, list.items.len));
            const off = smith.valueRangeAtMost(u32, 0, @intCast(list.items.len - len));
            try list.appendSlice(gpa, list.items[off..][0..len]);
            try std.testing.expectEqualSlices(
                u8,
                list.items[off..][0..len],
                list.items[list.items.len - len ..],
            );
        },
    };
}
3

Thanks for the quick reply! Not being able to use anything io related will cause me some head ache but I’m glad to know the issue now. (I could swear I tried with empty main before but I must have fumbled that..)

Well, I’ll be back.. probably :slight_smile:

4

I don’t think any of the Io implementations would work with wasm at all. It doesn’t fit the model. Wasm is sandboxed by default, doesn’t have an OS layer. Io expects an OS to work with.

You would have to create your own Io implementation for it. Best case would be to try to use wasi, that might work out of the box.

5

Have a look at the start.zig file from the std library, a few things where added there that lets you control that. for thinks like bro, how am I gonna debug my debugs?! :smiley: .

Just adding a:
pub const std_options_debug_io: std.Io = undefined;
should do the trick. But you could replace that with a minimal implementation that would use some external console.log imports from the browser or so.
So the new Io stuff is using some features that are not available in freestanding

3 Likes
6

Ok, that makes sense. I don’t want to do I/O anyway (unless i compile for native), just stringing text/bytes together in memory. Need to find alternatives for stuff like formatting/printing text into a buffer with Io.Writer.fixed() though. That pulls in Io.Threaded as well.

7

That works for wasm32-freestanding but not for native. Maybe I need to better separate the stuff that’s only native and/or only wasm. Food for thought :slight_smile:
Still I’m a bit cautious about using a debug feature for a “finished product”.

8

You should be able to use Writer/Reader directly without pulling in all of Io. Lazy field analysis should allow that.

As long as you don’t use any of the things that use Io Threaded, you should be good.

2 Likes
9

I checked it and it works file with wasm32-freestanding. One thing to note, you have to remove the error set from the return type. It seems that having the error set will cause Zig to pull in all of the machinery from debug.

const std = @import("std");
const Writer = std.Io.Writer;

const repro = @import("repro");

pub fn main() void {
    var buf: [256]u8 = undefined;
    var writer = Writer.fixed(&buf);

    writer.print("Hello, the number is {d}", .{42}) catch unreachable;
}
10

also, these here will be picked in the default case, and as the comments say, they are more like for outputting stuff before you have inited your own Io instance. But if something happens before, some Io has to exist….

/// Statically initialize such that calls to `Io.VTable.concurrent` will fail***

/// with `error.ConcurrencyUnavailable`.

///

/// When initialized this way:

/// * cancel requests have no effect.

/// * `deinit` is safe, but unnecessary to call.

pub const init_single_threaded: Threaded = init: {

**const env_block: process.Environ.Block = if (is_windows) .global else .empty;**

break :init .{

.allocator = .failing,

.stack_size = std.Thread.SpawnConfig.default_stack_size,

.async_limit = .nothing,

.cpu_count_error = null,

.concurrent_limit = .nothing,

.old_sig_io = undefined,

.old_sig_pipe = undefined,

.have_signal_handler = false,

.argv0 = .empty,

.environ_initialized = env_block.isEmpty(),

.environ = .{ .process_environ = .{ .block = env_block } },

.worker_threads = .init(null),

.disable_memory_mapping = false,

**};**

};

var global_single_threaded_instance: Threaded = .init_single_threaded;

/// In general, the application is responsible for choosing the `Io`

/// implementation and library code should accept an `Io` parameter rather than

/// accessing this declaration. Most code should avoid referencing this

/// declaration entirely.

///

/// However, in some cases such as debugging, it is desirable to hardcode a

/// reference to this `Io` implementation.

///

/// This instance does not support concurrency or cancelation.

pub const global_single_threaded: *Threaded = &global_single_threaded_instance;

11

Oh, wow, that’s good to know! Ty!

12

Way over my head :joy:

Will need time to absorb this!

13

They are just pointing out the global threaded instance and the single threaded configuration it uses. That is the Io the debug machinery uses.

IDK why they are bringing it up, since it is already established it doesn’t work for your use case.

But I need to point out the earlier suggestion

is a bad idea, if you accidentally call into it, you will get undefined behaviour.

Instead, use std.Io.failing which will just always return an error.

4 Likes
14

(Not sure to whom to reply or if this should be a new topic)

Could the io be overwritten depending on the compilation target?

What i want is:

#ifdef IS_WASM
pub const std_options_debug_io: std.Io = std.Io.failing;
#endif

What I tried is:

if (arch.isWasm()) {
pub const std_options_debug_io: std.Io = std.Io.failing;
}

and

pub const std_options_debug_io: std.Io = if (arch.isWasm()) std.Io.failing else <don't know what goes here>;

Top level if statements are not allowed and I don’t know what io instance to use for native. Typically one gets the default io from juicy main which is not an option here.

1 Like
15

yes, it would be
pub const std_options_debug_io: std.Io = if (arch.isWasm()) std.Io.failing else std.Options.debug_threaded_io.?.io();

You cant make the existence of a declaration conditional, only its type and value.

The value I put in the else is just the default value std uses if your decl didnt exist.

And you can override that with std_options_debug_threaded_io as well. You can disable it completely by overriding it to be null, though that does require you provide your own std_options_debug_io since its default value relies on it not being null.

The default value for std.Options.debug_threaded_io is just a pointer to std.Io.Threaded.global_single_threaded_instance

The reason that exists is because std needs to give the implementation some things at startup in order for its debug/panic machinery to have feature parity with 0.15, and also so that std.Io.Threaded.global_single_threaded has feature parity with 0.15 APIs, as it can be used as a drop in replacement before rewiring your code to pass an io everywhere.

16

Great! It compiles fine and after (conditionally) switching from page_allocator to wasm_allocator it also works! :slight_smile: