Io.Reader: How to read an unknown-size line in a file?

Help
,
1

Hi!

I’ve been studying the Io.Reader implementation.
I really like its zero-copy [1] API: take, toss, and peek.

That said, I’m struggling to understand how to use Io.Reader in this scenario:
I need to read an unknown-size line in a file.

Because the line size is unknown, I can’t guarantee that my .buffer will be big enough.
I was expecting Io.Reader to have a way of filling a buffer until a delimiter is detected.
Maybe something like Io.Reader.readSliceDelimiterExclusive()?

But I can’t seem to find anything like this.
What is the idiomatic approach for solving this problem?

Thanks !

  1. except when a rebase occurs, of course ↩︎

2

I believe Reader.takeDelimiterExclusive is what you’re looking for. Note that it will throw an error if the line is longer than the buffer.

1 Like
3

Hi @nonzeroq. Thanks for your answer.

Note that it will throw an error if the line is longer than the buffer.

Sorry if that was not clear: this is exactly what my problem is.
I cannot know the maximum size of a line, and therefore I can’t size my buffer.

Note that I could give my Io.Reader an unreasonably big buffer, but I wanted to know if there was a more elegant solution to this.

4

I suspected that was the case but wasn’t sure. In that case I think that using Reader.streamDelimiter (or one of its variants) with a Writer.Allocating writer is pretty elegant :^)

3 Likes
5

I cannot know the maximum size of a line, and therefore I can’t size my buffer.

Would it work for you to catch error.StreamTooLong and then process the partial read in some way?

I was looking at this exact problem the other day and at least for my use case, it works well enough to consider the end of the buffer as if there was a '\n’ at the end.

6

using Reader.streamDelimiter (or one of its variants) with a Writer.Allocating writer is pretty elegant :^)

Ohhh. This is interesting!
Give me a few minutes so I can tinker with this idea!

1 Like