Plain-changes - An unique element permutation iterator

While doing Advent of code 2015 day 13 I realized I needed to make some combinations of people around a table. Turns out this concept is called a permutation and well, one thing led to the other until I made my first library ever.

You can find it here: https://codeberg.org/T3kla/plain-changes

And it kinda does this:

n = 1 n = 2 n = 3
1 1 2 1 2 3
2 1 1 3 2
3 1 2
3 2 1
2 3 1
2 1 3

I had a lot of fun optimizing it. I found out many little things, like some times u32 is faster than u8 if it involves indexing, or that unwrapping an optional takes a few more nanoseconds than comparing two ints.

The biggest optimization of them all came from the inverse index array, which I have no clue if it’s mainstream to use in this algorithm but it’s basically a *3 speedup.

Help!

This is truly my first library ever and I have no clue if my setup is actually correct, since I just imitated other Zig libraries I saw in Codeberg.

If you see anything that catches your eye, I’m completely open to criticism!

Supported Zig versions

zig-0-16

7 Likes

I remember loving the book Another Fine Math You’ve Got Me Into by Ian Stewart as a kid (although his stories are far from kid stuff!), which is, as i understand as an adult, mostly a compilation of some of his best stories as a regular columnist on mathematics for Scientific American. In that book, one of the stories features a character based on the Hunchback of Notre Dame named Quasimodulo, and “ringing the changes” is discussed at some length!

Anyway, fun trip down memory lane, cheers on your first library!

4 Likes

That is what Tom Scott is trying to do in his recent video.

Is a unit test the best place for the performance benchmark? Why not a main() in a separate file?

Why not std.testing.allocator in tests?

Why deinit() but no init()?

Why allocator field? You only access it from one function. Make it a function parameter.

Why public function is returning an inferred error set. Define the error set. We need to know what errors it returns without reading the whole file.

deinit() should free memory in the reverse order to better co-operate with arena style allocators.

2 Likes

I have no clue! At this point I don’t think I care enough about the performance of the implementation to put up with it.

For a real thing I’d definitely create a way more extense testing suite.

I read the Choosing an Allocator guide at the start of the project. After, when writing the tests, I remembered the std.heap.DebugAllocator name and since it has the word “Debug” in it, I assume it was the one for testing/debugging.

Thanks for mentioning it, I will change it for the testing one.

Most of my inspiration comes from the std.mem.tokenize API but it is true they don’t use allocators. I might wrap all that in init() and deinit() for clarity and also to resemble the Allocator API.

After a quick read about inferred error sets, I coincide I should be using a declared error set. Maybe even merge it with the Allocator ones since permutate() can throw one of those through try

Maybe pub const Error = Allocator.Error || error{WrongSize};

This is sooo niche and I both love it and hate it. Will change!

2 Likes

undefined default values do not feel right.

Another public function with an inferred error set returns some custom error instead of familiar error.Overflow. Would it be simpler to call std.math.mul()?

It feels like you are abusing inline functions. Docs say:

It is generally better to let the compiler decide when to inline a function

Did performance measurments demand inlining? Why do you allocate on heap then? Would not it be better for performance to make buffer length a comptime parameter?

pub fn PermutationIterator(comptime T: type, len: comptime_int) type {
    return struct {
        buffer: [len]T,
        trackers: [len]Tracker,
        ascending: [len]u32,
1 Like

Nice read! I’m kinda tempted to do a named default… but I’m also kinda tempted to just not make the iterator public so you can only get it through permutate().

It would force the same kind of sanitation without additional code.

Anyway, it’s good to know that I can expect a default function for me in general in Zig APIs.

True! I gave little thought to that function to be honest. Nice to know about std.math.mul() since it does optimize for comptime_int. Will change.

Yes, I did the testing and I gain some margin using inline. When using -OReleaseFast I can confirm they are being inlined every time, which makes sense since I only use them once and they process the same value.

About allocations, I thought about just allocating in the stack but I don’t feel confortable with that since I accept buffers of arbitrary sizes. This means that I’ll be consuming an arbitrary amount of memory, thus I can not grantee I won’t be consuming too much of the stack.

In any case, if someone wants to allocate on the stack they can just use FixedBufferAllocator I guess. What I’m thinking is that it would be sweet to add a helper function that tells you how much the iterator is gonna allocate given the size of the buffer, so you can decide where the memory goes.


A while later:

const Int = u32;
const IntMax = std.math.maxInt(Int);

/// Given a buffer length, returns the exact number of permutations you can
/// expect the buffer to experience. It's equal to the factorial of `n`.
pub fn calculatePermutations(len: usize) error{Overflow}!usize {
    var res: usize = 1;
    for (1..len + 1) |i|
        res = try std.math.mul(usize, res, @truncate(i));
    return res;
}

/// Given a buffer length, returns the exact number of bytes the iterator will
/// be allocating during initialization.
pub fn calculateAllocations(len: usize) error{Overflow}!usize {
    var res: usize = 0;
    res += try std.math.mul(usize, @sizeOf(Tracker), len);
    res += try std.math.mul(usize, @sizeOf(Int), len);
    return res;
}

With this I can change the backing int if I want to, and I will be able to calculate allocations.

Sadly calculating the expected allocation is a bit more complex than this since I’m not taking into account alignment but yeah, I’ll get there if I get some more time on my hands.

Since you mentioned -OReleaseFast already inlines them, does that mean you chose to use inline based on performance gains under -ODebug?

I have never used -ODebug, but if it is the standard mode of zig test, yes.

While I understand that the compiler behavior for inlining depends on many factors, stating that I want the inlining ensures that anyone using amd64 will have faster permutations while testing their software. And that is a good thing.

In other words, the “the compiler is smarter than you” argument is, at this time, not convincing me of slowing user’s tests, specially knowing the most common performance-critical-case (amd64 release fast) will not suffer.

There was also a point where I made a version without functions at all, and I doubt that if I happened to upload that instead anyone would have said “hey but maybe just do many functions and let the compiler decide”. I divided it in inline functions just to display more clearly the steps of the algorithm.

In any case, if you or anyone has a better argument, or more info, I’m all ears. I’m just not convinced at the moment.

My opinion is: don’t measure performance / benchmarks in debug mode (or at least not primarily).

If somebody wants to use your library while testing their code in debug mode, they already have the option to pass one of the release modes to the optimize argument of your module, let the user choose with what optimize setting they want to compile your library. (If your library is used to write the test and isn’t what is actually being tested)

For example when working on raylib projects I define a separate raylib-optimize build option I can set, which defaults to the value of optimize.
That way I can use -Doptimize=Debug -Draylib-optimize=ReleaseFast to get a debug build of my application code which uses a release build of raylib and just -Doptimize=Debug to get a full debug build.

So I don’t think you need to inline functions for better debug mode performance, when the user can compile the library in release while using other code in debug mode. Debug mode should be optimized for being easy to debug, so using inline rarely is better.

5 Likes

You truncate() an usize into usize. I think it is redundant.

What do you mean by “undefined or empty elements”? I do not think we have such things in Zig at least not in integer types.

Is this doc comment attached to const std?

You never test the correctness of every step of the iterator. You only test the last value. What if there was a bug in the middle?

You never test edge cases. What if the buffer was the smallest possible or the largest possible? Are you sure that nothing is going to panic?

Solved.

It means that you can permutate an undefined buffer or a zeroed buffer. It’s just a statement about how little I care for the actual uniqueness of the given buffer, since this algorithm is usually used with elements of that nature.

Well of course. Someone had to document it at some point :upside_down_face:

Well, I wanted to add a new optimization that will change the movement of the elements but I can add a test in the mean time.

True! I added a bunch of those.


Btw, I removed functions altogether because I wanted to test some other optimizations, and performance went up by 20% in debug and 3% in release fast (versus inline) without doing anything else, so I’m leaving it like that.

You could create a small test that exhaustively accumulates/collects all permutations in an unordered set and then compare that value, that way the order is irrelevant for that test.

2 Likes

“Assumes the elements are unique”.

You leak memory. Test it with checkAllAllocationFailures().

You seem to struggle to understand alignment and padding. You may want to read the documentation about alignment and look into an implementation of alloc().

Your test do not use the value returned by next() function. You could use it:

try std.testing.expect(it.next());

I don’t think that phrase is accurate since I don’t assume anything. Elements can literally not even have a coherent bit representation for all I care. Technically I don’t even need the type, just the length in memory of each element.

I think the current definition is sufficiently accurate and necessary given the context around this algorithm.

Still, I reworded it as:

/// Returns an iterator capable of generating all permutations of the elements
/// present in the given buffer, regardless of their value, uniqueness or
/// initial ordering.

Wtf. First of all, that function is magic. Second of all, I was falsely under the impression that std.testing.allocator was doing that kind of job for me and that was the point of using it. It is weird for me having a debug allocator and a testing allocator but neither seems to do a great job a catching these.

To be honest the checkAllAllocationFailures function is god-like already and knowing of it’s existence will prove useful, but I kinda feel like this testing should be somehow included in test blocks or allocators by default.

On the other hand, reading the docs seems like errdefer is basically mandatory after an allocation so this could just be a warning at LSP level, an error at compile time, or just embedded in the language itself honestly.

I do struggle! Some time ago I saw Andrew’s DOP talk and I really liked to learn about alignment, but I didn’t used it so I lost it. At this point I feel like to write Zig I need a degree in computer science.

After 2 hours of trying to understand this I get to various conclusions:

  • I can do the stupid easy and allocWithOptions to align my big backing ints to 1 so they fit right after the Trackers. I thought this might hurt performance or even give me random performance per iteration depending of the amount of Trackers I have, but apparently it doesn’t.

  • I can adapt calculateAllocations so it takes into account alignment but that requires me to pass the pointer to the backing buffer to the function since that pointer itself might be aligned in various ways such as that even when Trackers len is % 4 == 0 the allocator’s end is not aligned to 4 but to 2…

  • I can burn my PC

Given that the alignment of the backing int doesn’t affect performance and keeps everything smaller, I’ll go that route for now I think.

You are quite exhaustive. Changed it to:

try std.testing.expectEqualStrings(&toPermutate, "123");
try std.testing.expect(it.next());
try std.testing.expectEqualStrings(&toPermutate, "132");
try std.testing.expect(it.next());
try std.testing.expectEqualStrings(&toPermutate, "312");
try std.testing.expect(it.next());
try std.testing.expectEqualStrings(&toPermutate, "321");
try std.testing.expect(it.next());
try std.testing.expectEqualStrings(&toPermutate, "231");
try std.testing.expect(it.next());
try std.testing.expectEqualStrings(&toPermutate, "213");
try std.testing.expect(!it.next());

Btw, I discovered that certain tests can affect the performance of other tests. Commenting out test "test fixed buffer" will add 5ns per .nest() to test "test iteration performance".

Kinda funny, still, I just don’t care enough to make a proper performance testing suit.

:magic_wand:

And it does! checkAllAllocationFailures() only triggers OutOfMemory errors a few times. The allocator actually detects the leak.

std.testing.allocator is an instance of DebugAllocator with some options suited for testing enabled.

It is not. I have seen it fail. It only detects leaks that are triggered by OutOfMemory and ignores any other condition. It has false positive in case you somehow recover from the error without bubbling it up. For example, std.AutoHashMap.getOrPut() can recover from OutOfMemory.

Maybe you are onto something. Build a custom test runner to see if it is actually possible.

… unless the allocator is known to be some kind of Arena and you intend to free everything it with one arena.deinit() call.

That would restrict Arenas and all kinds of smart stuff the programmers like to invent.

Did you read the assembly? Did you try to compile for another target CPU?

You could skip the allocator if you know the length at compile time:

fn PermutationIteratorComptimeKnownLength(comptime T: type, len: comptime_int) type {
    return struct {
        buffer: []T = undefined,
        trackers: [len]Tracker = undefined,
        ascending: [len]Int = undefined,
1 Like