You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by "metesynnada (via GitHub)" <gi...@apache.org> on 2023/02/08 10:04:48 UTC

[GitHub] [arrow-rs] metesynnada commented on issue #3674: Deadlock in arrow-csv reader for FIFO file reading

metesynnada commented on issue #3674:
URL: https://github.com/apache/arrow-rs/issues/3674#issuecomment-1422339772

   I intentionally make a test like this, to make it more clear. 
   
   I think waiting for the next byte to produce a RecordBatch even if we have the necessary bytes is against the streaming semantics. 
   
   We use this test pattern for testing stream pipelines. We test "If I gave a batch, can I get a batch as output?", but testing with actual files becomes "If I gave a batch_size + 1 row, can I get a batch as output?".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org