You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/24 12:45:22 UTC

[GitHub] [arrow-datafusion] tustvold commented on pull request #378: Add support for multiple partitions with SortExec (#362)

tustvold commented on pull request #378:
URL: https://github.com/apache/arrow-datafusion/pull/378#issuecomment-847017349


   I've added a new constructor that allows opting into the new behaviour. I wasn't aware of the way that MergeExec is plumbed into the plans and that this would break it.
   
   I do wonder if instead of relying on an `AddMergeExec` optimisation pass, the plan conversion from `LogicalPlan::Sort` should just inspect the input partitioning and add the Merge if necessary. After all, it already has to inspect the partitioning for operators such as `LogicalPlan::Limit`, and so not just generating a valid plan from the outset seems a touch surprising to me...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org