Fix AbstractInputStream#gets to not assume a_sep_string will appear…#662
Conversation
… within the bytes that are being limited by `number_of_bytes`
|
Hi @ListlessPanda, Thanks for looking at this. I've been doing a load of updates to |
|
Looking at the tests I added in #661, I think I test the case you mention above, here: You can see that some of the returned lines are truncated by the limit before a separator is found. Is this what you were fixing? |
|
I've now merged #661 and released as v3.3.0, so please can you test that and see if it fixes the issue you're seeing? |
|
I think this has been fixed in v3.3, so closing this PR, but please reopen if not the case and I will have another look. |
The
AbstractInputStream#getshas logic to check if the number of bytes within the@output_bufferare over the limit then setsover_limit. It seems that if a separator was passed in along with the number of bytes the code assumes the separator will appear within the bytes read.This was picked up when streaming a CSV out of a ZIP where the CSV passes the
a_sep_stringas\nand the number of bytes as8192and the file contains a single column value that is longer than8192bytesI made a change that if the separator wasn't found and we are above the limit it will return up to the number of bytes and increase the
@posby that amount.(This is my first pull request please let me know if I need to do anything else)