StackFallbackAllocator vs FixedBufferAllocator patterns

StackFallbackAllocator doesn’t follow the typical init() creation pattern, but uses a kind of factory function, stackFallback(), instead. I’m guessing this has something to do with it being a kind of composite, rather than a “normal”, fundamental allocator, but comment on that would be a nice bonus point.

What interests me most is the fact that StackFallbackAllocator takes a comptime size: usize, and holds its own buffer, given that size, whereas FixedBufferAllocator asks for a reference to a buffer that the caller creates as a preliminary step to init()ing the FixedBufferAllocator object. Why the difference?

1 Like

I think `stackFallback()` is just a convenience function that got introduced to make it easier to use.
But the fundamental reason is likely just that you are looking at a 7+ years old API decision. I’d expect this API (alongside the rest of the standard library) to get reworked and streamlined before the 1.0 release.

I also wouldn’t be surprised if it gets removed entirely. I used the StackFallbackAllocator a while ago in my project, but ultimately it was rather clunky to use due to it’s requirement to free (in case the allocation spilled into the backing allocator) and it’s inability to reuse memory (so you kind of have to create a new allocator in each function level).

3 Likes

This is very interesting. It feels like there’s a motivation for this concept (StackFallbackAllocator) to exist, and you highlight that this implementation might be legacy convenience code; perhaps there’s motive to create a StackFallbackAllocator that properly meets needs (since it’s apparently not uncommon to use this pattern), and, perhaps, that lives in its own file (not just in std.heap), etc… even if it’s still a fairly simple implementation? If so, I suppose the devil’s in the details - can it even be done? Can a useful generalization be imagined, or are there too many application-to-application differences that it doesn’t make sense to provide a general solution? That doesn’t seem likely to me… so, probably it’s worth the effort?

I’ve always used arenas rather than stack to avoid having to worry about running out. This allocator seems very useful as stack rarely have cache misses. So I would say this allocator was worth some effort.

Indeed, I vote for this, conceptually, for sure. A use case: a website/app that serves dynamic pages in which, it’s well known that 90% of the pages for this particular app, though possibly wildly dynamic in size and content, fall under 10k in size (empirically discovered/designed, for this hypothetical). If you know you’re building the content to send serially (at least, if you can bind the “page” datastruct (that will eventually serialize) to the given core/thread/coroutine), you could re-use 10k (times n-cores or whatever) of stack a zillion times. Only 10% of the time would you bother the heap with allocations. But in this use case, you would, of course, have to account for those 10%; it would be more “simple” to just use a gpa in an arena all the time, but then, of course, you’ve got many more allocs, even if you’re efficiently freeing with each arena dump upon serialization (…rinse, lather, repeat).

I realize I didn’t need to “explain” this to anybody, but… anyway…. :slight_smile:

1 Like

In the current implementation, using a .init method would require manually providing a type rather than allowing it to be inferred from the RHS of the assignment due to the array size being part of the type.

I think it would make more sense to rewrite StackFallbackAllocator to receive a slice on initialization, much like Io.Reader/Writer. It’d also make StackFallbackAllocator non-generic, which I can only see as a plus.

If that were to happen, it should probably be renamed to something like BufferFallbackAllocator instead.

2 Likes

All good. So… indeed, I feel like an evolution of StackFallbackAllocator (to BufferFallbackAllocator!) is alluring; indeed, as it is, even though it has advantages over my late-night off-the-cuff ….

But, is it fundamental/pure enough to even be in std? I suppose if it was used a lot, then even if it was an amalgam, it could be handy. Is it a right way forward to take these insights and propose a change/contribution? Or does it make more sense to just make this a piece of my own application code?

I think StackFallbackAllocator is quite handy actually, like many things one must know when to use it and why.

Weird name though. It doesn’t fall back to the stack, that’s backward! It’s a HeapFallbackStackAllocator!

2 Likes

Ha! Maybe it’s the term fall”back” that confuses where the predicate belongs. I kinda like BufferFallbackAllocator (read “buffer-base that falls back (forward?) to a provided (heap) allocator) and @invlpg’s supporting comments about modeling the buffer-supplied Io Readers and such…

And I’m curious about more support for de-genericizing it. Would many consider that a good move, or are counter-arguments worth dialog?

1 Like

It’s barely generic to begin with. The pattern of using a numeric comptime parameter to size-specialize a data structure used to be pretty common, but those have been disappearing (like BufferedWriter, which we just call Writer now, and is no longer specialized to buffer size).

In principle, comptime-knowability of the buffer bound could lead to more optimization opportunities. My hunch is that it doesn’t show up in practice, or we’d see more use of the pattern rather than less over time.

I have no objection to making it fully concrete. In principle (again!) there might be chances to reuse some buffer which code has handy, in practice… maybe? Finish line-reading a file, close it, build a StackFallbackAllocator in the same frame?

Personally, I’d declare a second buffer, make it the same size, and cross my fingers that liveness analysis in the compiler figures out it can reuse it. I’d only force the issue if it was clear that it isn’t doing that, and leave a little note to my future self that we’re reusing memory on purpose.

Excellent.

I hate to get pigeonholed, but I’m unsettled and inspired to flesh some more things out.

  1. You highlight a pattern: frequently, an off-the-shelf solution simply doesn’t cut it for somebody, but it’s “close”. Then the developer just says, “oh well, I’ll write my own”. An std wants enough general usefulness to attract developers to use the well-considered implementations. Therefore, with reservations, I’d advocate for two implementations.
  2. First, StackFallbackAllocator uses a FixedBufferAllocator, so it should follow the FixedBufferAllocator motif of taking a buffer, rather than a size arg, regardless of the “generic” argument. This is symmetry/consistency.
  3. Second, maybe(?) a (generic type) FixedSizeAllocator is in order; it would be a thin wrapper of FixedBufferAllocator, merely taking responsibility for the buffer, and taking a size arg for that purpose. It would be a mere convenience for all the developers who don’t want to two-line-create their stack Allocator and don’t see any value in re-using their own buffer. Yes, it would be very little code, but yet another thing to maintain… it sounds a lot like the whole “managed/unmanaged” argument all over again… would it be worth it?
  4. Then, a fallback allocator that references this FixedBufferAllocator would be natural. Which makes me think about names…
  5. “fallback” is a tough word. I find myself attracted to my earlier late-night stab at creating a StackFirstAllocator, before I realized that StackFallbackAllocator existed. “StackFirst” sings to me more clearly than “fallback”, the direction of which requires an extra synapse to resolve. Words like failover, alternate, and standby all seem inadequate, too. Four-word combos seem like atrocities, but, for the sake of argument: FixedBufferFirstAllocator and FixedSizeFirstAllocator (complements to FixedBufferAllocator and FixedSizeAllocator). Am I being ridiculous? For the sake of completeness, I should be ornery and put on the table: allocator.FixedBuffer, allocator.FixedSize, allocator.FixedBufferFirst, allocator.FixedSizeFirst, (and fundamentals: allocator.GeneralPurpose, allocator.Arena, allocator.Debug???)
  6. I’d really like to flesh out code to an end like this… I see pros and cons to my first-stab StackFirstAllocator. I see in the StackFallbackAllocator’s get() is a bit of a “side-effect” implementation; might be less-so if it was simply renamed reset(), especially since FixedBufferAllocator.reset() is so simple and effective anyway. In my case, my only fallback interest was in Arena, which also has a reset(), so it was simple to think of this StackFirstAllocator as “simply resettable”; I realize that FallbackAllocator could have anything for a fallback, and other allocators don’t always have reset()s (though one could contrive an implementation, it would just suggest some extra underhood magic, and would probably require init() calls). get() is ALSO weird in that it asks the caller to know they’re not supposed to call it more than once! Sort-of get it, but…. Other contrivances, like forcing no-use of allocator() method, well… feel… motivated but a little ugly. First: this is not a “fundamental” allocator; rather, it’s a compound, and so it’s natural to expect that it’ll be different, even in the interface. But I think it COULD still implement the interface, informally; I don’t necessarily see why it has to be danced around. But I might be missing something behind the get()-instead-of-allocator() (and the “don’t reuse”) motivations.

I’d like to try my hand at a “real” version of my StackFirstAllocator; if nothing else, I’ll have something that suits me. But feedback could be helpful, and maybe could result in std modification, at least in comments, for clarification on why things are the way they are. If I “succeeded”, I’d be happy to try a pull request; if it was rejected, oh well. Not sure if that’s how it’s done, not having been around for long.

long, long ago (? to 0.14) there was this alternative to ArrayList.initBuffer (was Unmanaged at the time), called BoundedArray which was generic over an array size and stored the array, not a slice nor a pointer to an array, this was convenient to some who did need a comptime known size, but most did not, but they used it anyway for whatever reason.

But alas, BoundedArray had some serious character flaws, namely bad slow and stinky code gen. It was also redundant to the faster ArrayList.initBuffer, both was more work to maintain.

One day, Andrew decided to take BoundedArray out back for a surprise. Some saw this as horrific, most didn’t care. Most got faster code.


IDK why I turned this into a story, I just started writing it like that, and it was too fun to stop.


seriously, though, a few less lines is not a convincing argument for zig to do something.

I do think StackFirstAllocator is a better name, I agree that it should take a slice, instead of being generic of the size of an internal array. I will acknowledge that the performance pitfalls of BoundedArray were from copying the array around, particularly bad for large sizes, this doesn’t apply to the allocator since It’s used through a pointer (via the interface).

If nothing else It’s still a good learning oportunity and fun

4 Likes

Very valuable backstory, thanks. Indeed, if one had to choose between buffer-initialized and size-initialized, I’d choose buffer in a heartbeat, even if there weren’t underlying pitfalls as with BoundedArray. Similar to other “managed” arguments - it’s quite easy for one to make their own tiny wrapper to aggregate/manage things like buffers, Io objects, etc.; the trend for zig to lean away from providing a million thin wrappers and convenience “managers” seems a good one, imo.

Oh dumb. I was short-sighted. Of course the existing StackFallbackAllocator is generalized not just on the size arg – the fallback allocator itself is not provided instantiated; rather, the class is provided, to build the type. And this seems right, really. But for me, I only value an ArenaAllocator because I want to reset() and reuse the compound allocator over and over, and reusability isn’t easy to achieve with the generic approach of the StackFallbackAllocator. At the end of the day, I could see motivation for 1) buffer-instantiation rather than size-arg for the type, 2) maybe a rename… but otherwise, meh. I’m back to wanting a non-generic StackFirstArena allocator that is buffer-instantiated, can be reset(), and is rather simple, actually. Seems that such a StackFirstArena might not be std-worthy, I don’t know, but force-fitting StackFallback doesn’t seem my cup of tea. Sorry if this journey was a bit of a waste, other than the educational benefits.

There is nothing stopping it from being a non-generic type, the generic size parameter is used in ONE spot, and that is the size of the array field.

So it’s still the same question of if it should be an array or a slice, which we have practically all came to agreement on.

lets not use the same word generic with different meanings in the same paragraph or even the whole post. ideally we would only use one deffinition ever.

the different meanings are, first ‘generic types’, then just above you used it as a synonym for ‘abstraction’.

i think you are tripping yourself up with your double use of the word. the reason you cant easily have a reset is the abstraction over the buffer and the other allocator that makes that hard, as you can reset an arbitrary allocator, you’d need to specifically take an arena instead, which you do mention.

though there is nothing stoping you from only reseting the buffer, then you can use the buffer till its full again then back to the other allocator.

the thing with that is the user of such a StackFirstAllocator, cannot rely on the the reset ‘freeing’ their memory. they still have to free each thing individually manually, or wrap this in an arena is also a solution.

Ah, the thing is, I DID mean generic, as in generic types (not abstracted)… but that’s because I was missing something. Just before that last recent post of mine, I looked back at heap.zig and misread fn stackFallback(…) to return a type that was generic on the fallback allocator type as well as size, but I just misread it. My apologies. I see that it does not build a type with the fallback allocator TYPE, but with an instantiated fallback allocator object. That was my flub. This led me to think there was more to it in terms of the ownership/lifetime of the secondary (fallback) allocator. I think it’s worth my gaining an understanding, accordingly. So, I thought: if the allocator TYPE was provided, then that (secondary) allocator would be created within, and would live the life of the StackFallbackAllocator object. But if the calling code simply provided an instantiated (e.g., ArenaAllocator) secondary, then …. does the caller have to be careful about the provided (instantiated) allocator going out of scope while the StackFallbackAllocator that was created with it is still around? In practice, it seems simple enough that they’d both live, e.g., in the same {} block, so perhaps it’s not a problem. And maybe it’s not a problem because a copy is made(?) … I just don’t trust myself quite enough yet wrt/ the whole copy-semantics/aliasing function-argument stuff… but perhaps I’m overthinking it. My quick (and wrong) conclusion was: “oh, they made this generic on the secondary allocator type, too… I wonder….” I need to not post late at night when my brain is half-off. :slight_smile:

So, thanks for enduring my baby steps.

That’s true; I hadn’t thought about that. Then the secondary allocator could grow ad infinitum if that’s what the calling code wanted. I could, then, within reset(), in addition to resetting the static buffer, check to see if the secondary allocator can be reset(). My reset() could take a bool reset_secondary_allocator_also, to keep that from being hidden behavior, and, in the case that that flag is set to true but the provided secondary allocator can’t be reset(), an error could be returned, to make it explicit (that is, if you create a StackFirstAllocator and provide an Arena as the secondary, then reset() will reset everything, which is probably what you want in some use cases; but if you provide a secondary that does NOT have a reset(), then you’d set this flag to false, to indicate that you want to reset the primary static buffer, but that’s really all you can throw away in a fell-swoop). This seems like it could maximize usability options. Just thinking out loud. But if I’m approximately on course, then I’m re-inspired. Thanks for walking me through it all!

Well… of course, memory could be freed along the way, as you say, free() by free()… I didn’t mean to imply that some kind of reset would be equivalent, except in the case of something like an Arena.

Yes it needs to stay valid, but you are overthinking it. all you have to do is not return pointers to stack variables from functions, that includes data that contains pointers, such as the allocator interface (its just 2 pointers).

in the future zig will catch this at compile time, on master it already catches one specific case that is returning a pointer to a var, it doesnt catch if its within another type though, nor if its pointing to a const not to be confused with a *const poniter. Both of those are more complex.

This isn’t a feature of the allocator interface, you would have to have the implementation which would make it generic, or you can hardcode a specific implementation such as ArenaAllocator.

Great, thanks; all makes sense. I’m going to try my hand at this tomorrow evening. Thanks all.

Ok, I gave it a try this morning, instead. Please critique. Tests at bottom, though not as comprehensive as I’d like yet. Thank you!

const mem = std.mem;
const Allocator = mem.Allocator;
const FixedBufferAllocator = std.heap.FixedBufferAllocator;

pub const StackFirstAllocator = struct {
   primary: FixedBufferAllocator,
   secondary: Allocator,

   pub fn init(buffer: []u8, secondary_allocator: Allocator) StackFirstAllocator {
      return .{
         .primary = .init(buffer),
         .secondary = secondary_allocator,
      };
   }

   pub fn allocator(self: *StackFirstAllocator) Allocator {
      return .{
         .ptr = self,
         .vtable = &.{
            .alloc = alloc,
            .resize = resize,
            .remap = remap,
            .free = free,
         },
      };
   }

   pub fn alloc(ctx: *anyopaque, len: usize, alignment: mem.Alignment, ret_addr: usize) ?[*]u8 {
      const self: *StackFirstAllocator = @ptrCast(@alignCast(ctx));
      return FixedBufferAllocator.alloc(&self.primary, len, alignment, ret_addr) orelse
                               self.secondary.rawAlloc(len, alignment, ret_addr);
   }

   pub fn resize(ctx: *anyopaque, memory: []u8, alignment: mem.Alignment, new_len: usize, ret_addr: usize) bool {
      const self: *StackFirstAllocator = @ptrCast(@alignCast(ctx));
      return if (self.primary.ownsPtr(memory.ptr))
         FixedBufferAllocator.resize(&self.primary, memory, alignment, new_len, ret_addr) else
                           self.secondary.rawResize(memory, alignment, new_len, ret_addr);
   }

   pub fn remap(ctx: *anyopaque, memory: []u8, alignment: mem.Alignment, new_len: usize, ret_addr: usize) ?[*]u8 {
      const self: *StackFirstAllocator = @ptrCast(@alignCast(ctx));
      return if (self.primary.ownsPtr(memory.ptr))
         FixedBufferAllocator.remap(&self.primary, memory, alignment, new_len, ret_addr) else
                           self.secondary.rawRemap(memory, alignment, new_len, ret_addr);
   }

   pub fn free(ctx: *anyopaque, memory: []u8, alignment: mem.Alignment, ret_addr: usize) void {
      const self: *StackFirstAllocator = @ptrCast(@alignCast(ctx));
      return if (self.primary.ownsPtr(memory.ptr))
         FixedBufferAllocator.free(&self.primary, memory, alignment, ret_addr) else
                           self.secondary.rawFree(memory, alignment, ret_addr);
   }

   pub fn reset(self: *StackFirstAllocator) void {
      self.primary.reset();
   }
};


test "sfa" {
   var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
   defer arena.deinit();
   var buffer: [10]u8 = undefined;
   var sfa = StackFirstAllocator.init(&buffer, arena.allocator());

   const expect = std.testing.expect;
   const expectEqualStrings = std.testing.expectEqualStrings;

   const allocator = sfa.allocator();
   const txt = "0123456789";
   const dest = try allocator.alloc(u8, txt.len);
   @memcpy(dest, txt);
   std.debug.print("In stack: {s}\n", .{dest});
   try expectEqualStrings(txt, dest);
   try expect(sfa.primary.ownsPtr(dest.ptr));

   const txt2 = "abcde";
   const dest2 = try allocator.alloc(u8, txt2.len);
   @memcpy(dest2, txt2);
   std.debug.print("In heap: {s}\n", .{dest2});
   try expectEqualStrings(txt2, dest2);
   try expect(!sfa.primary.ownsPtr(dest2.ptr));

   sfa.reset();
   //NOTE: this will NOT reset the secondary allocator - arena.reset() would have to be called here explicitly if wanted

   const txt3 = "0123";
   const dest3 = try allocator.alloc(u8, txt3.len);
   @memcpy(dest3, txt3);
   std.debug.print("In stack: {s}\n", .{dest3});
   try expectEqualStrings(txt3, dest3);
   try expect(sfa.primary.ownsPtr(dest3.ptr));
}