You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/21 05:56:32 UTC
[GitHub] [arrow] jorgecarleitao opened a new pull request #8729: ARROW-10672: [Rust] [DataFusion] Made Limit be computed on the stream.
jorgecarleitao opened a new pull request #8729:
URL: https://github.com/apache/arrow/pull/8729
Currently, the `Limit` operator in DataFusion collects all batches `ExecutionPlan::execute` to memory and returns a `MemoryExec` that mimics a stream. This can cause an expensive execution to happen in `ExecutionPlan::execute`.
This makes the limit operator in DataFusion to postpone the collection of RecordBatches to the stream, thereby avoiding the potentially expensive execution to happen inside of the `Plan::execute` and instead happen on the stream, as other operators already do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on pull request #8729: ARROW-10672: [Rust] [DataFusion] Made Limit be computed on the stream.
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8729:
URL: https://github.com/apache/arrow/pull/8729#issuecomment-731650912
I reviewed this code -- and it looks good to me too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8729: ARROW-10672: [Rust] [DataFusion] Made Limit be computed on the stream.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8729:
URL: https://github.com/apache/arrow/pull/8729#issuecomment-731515022
https://issues.apache.org/jira/browse/ARROW-10672
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb closed pull request #8729: ARROW-10672: [Rust] [DataFusion] Made Limit be computed on the stream.
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8729:
URL: https://github.com/apache/arrow/pull/8729
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org