You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/07 13:46:43 UTC

[GitHub] [arrow-datafusion] tustvold opened a new issue #1939: SortExec No Longer Streams Correctly

tustvold opened a new issue #1939:
URL: https://github.com/apache/arrow-datafusion/issues/1939


   **Describe the bug**
   
   https://github.com/apache/arrow-datafusion/pull/1596/files#diff-68811b72d27f9f5173223e0da1af2a467c2e4fff2f5f2237665fa29e1a6575c0L165 appears to have accidentally changed the behaviour of `SortExec` so that it no longer returns a stream that performs the sort operation, but instead performs the sort within `ExecutionPlan::execute`.
   
   This effectively stalls out constructing the rest of the physical plan until the sort has completed, and prevents result streaming from working correctly.
   
   **To Reproduce**
   
   Run a query with a large `SortExec`, observe surprising amount of time spent in `ExecutionPlan::execute`
   
   **Expected behavior**
   
   `ExecutionPlan::execute` should return a stream of results, but should not block on those results being available
   
   **Additional context**
   
   This resulting in what looked like missing traces in IOx (https://github.com/influxdata/influxdb_iox/issues/3822) as it never actually finished constructing the physical plan from which to collect metrics :sweat_smile: 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] yjshen commented on issue #1939: SortExec No Longer Streams Correctly

Posted by GitBox <gi...@apache.org>.
yjshen commented on issue #1939:
URL: https://github.com/apache/arrow-datafusion/issues/1939#issuecomment-1062607102


   Thanks, @tustvold . I'll have time to work on this later in the week. 😅 Please let me know if it has a higher priority.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] tustvold commented on issue #1939: SortExec No Longer Streams Correctly

Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #1939:
URL: https://github.com/apache/arrow-datafusion/issues/1939#issuecomment-1062731740


   No rush, whenever you have time 😀


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org