Cricket.zig: Cryptography formats reader and writer (PEM, DER, etc.)

Hi everyone,

I wanted to showcase cricket.zig. A library for writing and reading cryptography file formats (such as PEM files).

Currently only reading EC (Elliptic Curve) private and public keys is supported (either in PEM or DER format).

Reading a key from a PEM file should be as easy as:

const f = try std.fs.cwd().openFile(path, .{});
defer f.close();

const file_contents = try f.readToEndAlloc(allocator, some_limit);
defer allocator.free(file_contents);

const decoded_data = try cricket.decode.fromPem(allocator, file_contents);
defer decoded_data.deinit();

// Then decoded_data.bytes can be used with e.g. std.crypto.sign.ecdsa
// for signing and verifying.

The library is still under heavy development. I have yet to make a tagged release and finish writting a proper README. But any feedback will be greatly appreciated.

9 Likes

Very cool - I can personally speak to the fact that utilities like this are very appreciated. Not everyone needs them, but when you do it’s great when you can find them. So kudos.

Couple small things:

comptime {
    std.testing.refAllDecls(decode);
    std.testing.refAllDeclsRecursive(formats);
    std.testing.refAllDeclsRecursive(utils);
}

You may want to consider std.testing.refAllDecls(@This()) or even the recursive version since most of your imports are public. There may be a reason that you’re restricting decode to not be recursive though. If you intend on having contributors, you may want to add a comment as to why.

2 Likes

In terms of this bit here:

pub const Decoded = struct {
    arena: *ArenaAllocator,
    value: Value,

If you want an arena, you can make temporary ones where you need them and let the user pass whatever they want to it (you see this in std.json.parseSlice). In general, I would avoid passing pointers to allocator implementations unless you are trying to keep memory contiguous over time on the same arena via reset. Even then, the user could decide that by using the same arena in multiple places. You’ll notice that there are “leaky” versions in the json module that afford this capabiliity.

The same thing goes for this part here:

arena: *ArenaAllocator,

// later...

pub fn deinit(self: Self) void {
    self.arena.deinit();
    self.arena.child_allocator.destroy(self.arena);
}

In this case, this seems to imply that this actually owns the arena implementation but the pointer suggests otherwise imo. Arena’s are cool, but I wouldn’t force a user to use an arena. There’s plenty of times where it’s not the best tool for the job (sometimes a free list style allocator is way better for certain tasks… etc…).

I guess one thing that makes me curious is this part here:

/// Signed arbitrary precision ASN.1 `INTEGER` type.
pub const Integer = struct {
    /// Bytes representing the signed integer in two's complement representation.
    bytes: []const u8,

The size of a slice is almost twice the size of most integers you’ll come across. You could read this as a packed int in place or even use a fixed buffer and just own the bytes directly. You may have a reason to keep it as a slice though (I haven’t read the whole library). Just food for thought.

5 Likes

decode not being recursive was a typo. I’ll change this to what you suggested, it’s exactly what I wanted to do. Thanks a lot for that.

I’m new to implementing libraries having to think about the kind of allocators the users could want to use with them. I was thinking about the arenas as a way to simplify the memory management but it didn’t end up being so complex.

I was using the std.json implementations as a API reference. I’m wondering why does it hold a *ArenaAllocator?

Thanks again for the pointers. I just made the changes and it looks nice: chore: use Allocator instead of *ArenaAllocator by furpu · Pull Request #12 · furpu/cricket.zig · GitHub.

That’s a really good suggestion. I kept it as a slice because ASN.1 integers can be arbitrarily large but I haven’t yet decided if it’s really necessary to handle integers that large for implementing reading and writing the cryptography formats. At least I have not seen integers in the specs that would not fit in the primitive types. I’ll keep that in mind for sure.

2 Likes

Great question! Can you point to where that’s happening so I can look at it more closely? To be clear, I’m not saying it can never happen but I personally would advocate against it where possible and I think context is key here. I’m glad some of those suggestions helped :slight_smile:

I believe that’s because when parsing JSON the std lib gives you an option to allocate only if necessary? So there’s no easy way to know which data was allocated and which data comes from the original JSON string.

Here’s a tip: proper handling of memory gets tricky in the face of errors, which is why the standard library has std.testing.checkAllAllocationFailures. You can use this in the test suite to verify that functions don’t leak in the event that any allocation in that function fails.

This doesn’t test errors which aren’t error{OutOfMemory} though, and we don’t (yet) have a tool to deterministically force all other errors to happen, so you’ll want to provide test cases which trigger the other possible errors and check for leaks on those code paths.

The reward is code which robustly and correctly handles failures to allocate, which is rare in any language. Zig’s fine-grained approach to memory policy makes this more likely to occur, because someone could be using a stack-backed allocator, or another allocator with limited memory, but/and it also makes code useful in memory-constrained environments.

4 Likes

I wasn’t aware of std.testing.checkAllAllocationFailures. Thanks a lot for that.

Currently almost every function in the library doesn’t allocate but I plan to have allocating versions of some functions and that is super useful.

1 Like