[GitHub] [arrow-datafusion] JasonLi-cn opened a new issue #1879: CoalesceBatchesStream poll_next_inner function bug

[GitBox <gi...@apache.org>]

JasonLi-cn opened a new issue #1879:
URL: https://github.com/apache/arrow-datafusion/issues/1879


   **Describe the bug**
   SQL: select * from table where col1 = 'datafusion' limit 10;
   Some files are still being read after the result is returned。
   
   Because：
   [https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/physical_plan/coalesce_batches.rs](url)
   CoalesceBatchesStream poll_next_inner function has a loop logic, when sync table scan stream don't return Poll::Pending(eg. Parquet), when CoalesceBatchesStream's parent abort him, still can't quit loop, util table scan stream return Poll::Ready(None).
   Also, spawn_execution function in common.rs has this problem.
   
   **To Reproduce**
   Steps to reproduce the behavior:
   
   **Expected behavior**
   End stream when has sync table scan in a friendly way.
   
   **Additional context**
   Add any other context about the problem here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org