I just merged the new writer changes to master. I estimate I spent maybe 40 hours on these changes. Here’s the before/after of the simple “hello” example:
BEFORE: zigx/examples/hello.zig at 22ec7ed2f8e2de87811daed40bace040da4c0ac1 · marler8997/zigx · GitHub
AFTER: zigx/examples/hello.zig at 4b4035caba8811c1df20d8d8722135c2896564da · marler8997/zigx · GitHub
note the BEFORE/AFTER also has some unrelated cleanup
The gamble I made with this new API was seeing if I could remove needing an allocator to read data from the server. Most X11 messages are small and fixed in size, but, some messages can include dynamic data of arbitrary size. The old API would require the caller to provide an allocator for those. With the new API, instead of reading the full message into a buffer with an allocator, the API allows the client to read each section of the reply as needed. The downside of this approach is it takes the code that reads/separates messages that was once in a single place and disperses it throughout the application…alot more surface area for mistakes. To address this, I wrote an “X11 Aware Reader” that would track the state of the incoming data as it’s read. It’s more complex than a simple function that takes an allocator and reads the entire message into a buffer but, it also unlocks new abilities. Take this example: zigx/examples/getserverfontnames.zig at 4b4035caba8811c1df20d8d8722135c2896564da · marler8997/zigx · GitHub
This example requests the full list of fonts on the server and prints them to stdout. Here’s the old approach:
const msg_bytes = try x11.readOneMsgAlloc(allocator, reader);
defer allocator.free(msg_bytes);
const msg = try x11.asReply(x11.ServerMsg.ListFonts, msg_bytes);
var it = msg.iterator();
while (try it.next()) |path| {
try stdout.print("{f}\n", .{path});
}
We allocate a buffer for the message (likely a few dozen kilobytes), read the entire reply into it, then iterate over it as we write each string back to stdout. Here’s the new code:
const fonts, _ = try source.readSynchronousReplyHeader(sink.sequence, .ListFonts);
std.log.info("font count {}", .{fonts.count});
for (0..fonts.count) |_| {
const len = try source.takeReplyInt(u8);
try source.streamReply(stdout, len);
try stdout.writeByte('\n');
}
Here’s a comparison of each approach:
| Step |
Old API |
New API |
| 1 |
Allocate large buffer |
No allocator/buffer needed |
| 2 |
Read entire message into buffer (dozens of kilobytes) |
Read message header (32 bytes) |
| 3 |
Iterate/write each font to stdout |
Stream each font to stdout |
Note that we actually stream each font directly from the x11 reader to stdout. This means that it could potentially use sendfile and never even copy the data into the current process. This wouldn’t be possible with the old API.
Caching/Flushing
The other big problem this change solved was caching/flushing messages. The old API simply provided a way to calculate message sizes and serialize them. This left buffering up to the application. Up to now my applications have opted to take the simple approach, no buffer, 1 message per syscall.
// render the "hello window" with the OLD API
// 1 message per syscall
{
var msg: [x11.poly_fill_rectangle.getLen(1)]u8 = undefined;
x11.poly_fill_rectangle.serialize(&msg, .{
.drawable_id = window_id.drawable(),
.gc_id = bg_gc_id,
}, &[_]x11.Rectangle{
.{ .x = 100, .y = 100, .width = 200, .height = 200 },
});
try x11.ext.sendOne(sock, sequence, &msg);
}
{
var msg: [x11.clear_area.len]u8 = undefined;
x11.clear_area.serialize(&msg, false, window_id, .{
.x = 150,
.y = 150,
.width = 100,
.height = 100,
});
try x11.ext.sendOne(sock, sequence, &msg);
}
{
const text_literal: []const u8 = "Hello X!";
const text = x11.Slice(u8, [*]const u8){ .ptr = text_literal.ptr, .len = text_literal.len };
var msg: [x11.image_text8.getLen(text.len)]u8 = undefined;
const text_width = font_dims.width * text_literal.len;
x11.image_text8.serialize(&msg, text, .{
.drawable_id = window_id.drawable(),
.gc_id = fg_gc_id,
.x = @divTrunc((window_width - @as(i16, @intCast(text_width))), 2) + font_dims.font_left,
.y = @divTrunc((window_height - @as(i16, @intCast(font_dims.height))), 2) + font_dims.font_ascent,
});
try x11.ext.sendOne(sock, sequence, &msg);
}
The new API decouples these two concerns. The number of messages per syscall is now a result of the size of the write buffer. No code changes are needed to adjust this relationship, the code is now agnostic of buffering.
// render the "hello window" with the NEW API
// number of syscalls depends on writer, but, it'll probably
// just be 1 syscall at the end to send all of them
try sink.PolyFillRectangle(
window_id.drawable(),
bg_gc_id,
.initComptime(&[_]x11.Rectangle{
.{ .x = 100, .y = 100, .width = 200, .height = 200 },
}),
);
try sink.ClearArea(
window_id,
.{
.x = 150,
.y = 150,
.width = 100,
.height = 100,
},
.{ .exposures = false },
);
const text = "Hello X!";
const text_width = font_dims.width * text.len;
try sink.ImageText8(
window_id.drawable(),
fg_gc_id,
.{
.x = @divTrunc((window_width - @as(i16, @intCast(text_width))), 2) + font_dims.font_left,
.y = @divTrunc((window_height - @as(i16, @intCast(font_dims.height))), 2) + font_dims.font_ascent,
},
.initComptime(text),
);
Conclusion
I think my gamble was a success! The approach of creating a “structured reader” that takes care of shepherding protocol-specific data over a generic reader seems to be the way forward. I’ve already started doing the same thing for DBUS here GitHub - marler8997/dbus-zig and I’m excited to start using these libraries in some real software!
