You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "gene-bordegaray (via GitHub)" <gi...@apache.org> on 2026/06/06 13:09:26 UTC

[I] Support planning logical range repartitioning [datafusion]

gene-bordegaray opened a new issue, #22786:
URL: https://github.com/apache/datafusion/issues/22786

   Part of #22395.
   
   DataFusion now has a logical `Partitioning::Range` representation, but `LogicalPlan::Repartition` with range partitioning is rejected during physical planning.
   
   Add planner support to lower logical range repartitioning to an appropriate physical plan once range repartition execution support is available.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray closed issue #22786: Support planning logical range repartitioning
URL: https://github.com/apache/datafusion/issues/22786


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4780635312

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4783409638

   > yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍 
   
   Sounds good, yeah for sure! I'll make an issue for that if there isn't one already 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4774287542

   Hey @gene-bordegaray ! If this is still open I'm happy to take it and start digging around!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4782191815

   yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4782194071

   completed in #22777 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Support planning logical range repartitioning [datafusion]

Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4781314064

   @gene-bordegaray Hmm, I definitely could be mistaken here but it looks like in the below that logical range repartitioning is now supported during physical planning:
   ```
       async fn map_logical_node_to_physical(...) {
   ....
                       LogicalPartitioning::Range(range) => {
                           let sort_exprs = create_physical_sort_exprs(
                               range.ordering(),
                               input_dfschema,
                               execution_props,
                           )?;
                           let ordering = LexOrdering::new(sort_exprs).ok_or_else(|| {
                               internal_datafusion_err!(
                                   "Range repartitioning requires non-empty ordering"
                               )
                           })?;
                           Partitioning::Range(RangePartitioning::try_new(
                               ordering,
                               range.split_points().to_vec(),
                           )?)
                       }
   ...
   }
   ```
   
   With the test `physical_planner::tests::logical_range_repartition_plans_output_partitioning` passing as well! Is the above what this issue was encompassing, or is there something else that I might be missing here? Thanks again for the help! 😁 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org