You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zml1206 (via GitHub)" <gi...@apache.org> on 2024/01/22 09:18:41 UTC
[PR] [SPARK-46790][SQL] Support coalesce partition through cartesian product [spark]
zml1206 opened a new pull request, #44830:
URL: https://github.com/apache/spark/pull/44830
### What changes were proposed in this pull request?
Like `Union`, support coalesce partition for partial shuffle child with `CartesianProduct`.
```
CartesianProduct
HashAggregate
ShuffleQueryStage
FileScan
```
### Why are the changes needed?
Like `Union`, `CartesianProduct` does not need all children have same number of output partitions. Coalesce partition for partial shuffle child shuffle can improve performance.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46790][SQL] Support coalesce partition through cartesian product [spark]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 closed pull request #44830: [SPARK-46790][SQL] Support coalesce partition through cartesian product
URL: https://github.com/apache/spark/pull/44830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46790][SQL] Support coalesce partition through cartesian product [spark]
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on PR #44830:
URL: https://github.com/apache/spark/pull/44830#issuecomment-1905442700
thank you @zml1206 , there is an on going pr for this https://github.com/apache/spark/pull/44661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46790][SQL] Support coalesce partition through cartesian product [spark]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on PR #44830:
URL: https://github.com/apache/spark/pull/44830#issuecomment-1905346118
cc @ulysses-you @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org