You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "gene-bordegaray (via GitHub)" <gi...@apache.org> on 2026/06/06 13:09:26 UTC
[I] Support planning logical range repartitioning [datafusion]
gene-bordegaray opened a new issue, #22786:
URL: https://github.com/apache/datafusion/issues/22786
Part of #22395.
DataFusion now has a logical `Partitioning::Range` representation, but `LogicalPlan::Repartition` with range partitioning is rejected during physical planning.
Add planner support to lower logical range repartitioning to an appropriate physical plan once range repartition execution support is available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray closed issue #22786: Support planning logical range repartitioning
URL: https://github.com/apache/datafusion/issues/22786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4780635312
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4783409638
> yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍
Sounds good, yeah for sure! I'll make an issue for that if there isn't one already
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4774287542
Hey @gene-bordegaray ! If this is still open I'm happy to take it and start digging around!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4782191815
yes forgot to close this, but the actual physical implementation for repartitioning for Range is not implemented if you would like to look at that @saadtajwar 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "gene-bordegaray (via GitHub)" <gi...@apache.org>.
gene-bordegaray commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4782194071
completed in #22777
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support planning logical range repartitioning [datafusion]
Posted by "saadtajwar (via GitHub)" <gi...@apache.org>.
saadtajwar commented on issue #22786:
URL: https://github.com/apache/datafusion/issues/22786#issuecomment-4781314064
@gene-bordegaray Hmm, I definitely could be mistaken here but it looks like in the below that logical range repartitioning is now supported during physical planning:
```
async fn map_logical_node_to_physical(...) {
....
LogicalPartitioning::Range(range) => {
let sort_exprs = create_physical_sort_exprs(
range.ordering(),
input_dfschema,
execution_props,
)?;
let ordering = LexOrdering::new(sort_exprs).ok_or_else(|| {
internal_datafusion_err!(
"Range repartitioning requires non-empty ordering"
)
})?;
Partitioning::Range(RangePartitioning::try_new(
ordering,
range.split_points().to_vec(),
)?)
}
...
}
```
With the test `physical_planner::tests::logical_range_repartition_plans_output_partitioning` passing as well! Is the above what this issue was encompassing, or is there something else that I might be missing here? Thanks again for the help! 😁
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org