Do I misuse or don't understand Io.Writer?

Help
1

I have a function that collects, compress, writes data to a buffer.

    const bound = try encoding.compressBound(self.uncompressedMetaindexBuf.items.len);
    try self.metaindexBuf.ensureUnusedCapacity(bound);
    try self.metaindexBuf.writer.ensureUnusedCapacity(bound);
    const n = try encoding.compressAuto(
        self.metaindexBuf.unusedCapacitySlice(),
        self.uncompressedMetaindexBuf.items,
    );
    self.metaindexBuf.items.len += n;

Here I compress, preallocate at once and write to the destination the content.

Now I need a file as a destination, so it feels correct to use Writer, but

  1. Writer doesn’t support “close” api, so it means I have to hold a link to a file to close it? so it means I hold them nullable, and if I have 3 destination implementations (e.g. ArrayList, File and S3) then I might have 3 fields per buffer to hold different deinit, and I have 4 of them, so it’s 12 fields in total only for deinit?
  2. no direct access to a file buffer, therefore I need to allocate an intermediate buffer to write a compressed content and then write the buffer to a file/destination array, right? can I trick somehow the implementation to use a file’s buffer, in streaming mode write compression and avoid intermediate buffer allocation?
2

Just take an *Io.Writer, the caller can deal with what writer they are using and closing the file if they are using one.

You can require the writer have a minimum buffer size and use its buffer.
But that might not be the best solution, depending on what you’re doing with it. Ideally, you wouldn’t need an intermediary buffer and could just write to the writer.

1 Like
3

creating the files in the caller is overwhelming, it duplicates quite a lot of code across the codebase, but yeah, it’s the only option at the moment.