You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/02/17 11:01:08 UTC

[GitHub] [spark] HeartSaVioR opened a new pull request #35551: [SPARK-38237][SQL] Rename back StatefulOpClusteredDistribution to HashClusteredDistribution

HeartSaVioR opened a new pull request #35551:
URL: https://github.com/apache/spark/pull/35551


   ### What changes were proposed in this pull request?
   
   This PR proposes to rename back `StatefulOpClusteredDistribution` to `HashClusteredDistribution`. This PR retains the content of the classdoc for stateful operators in `HashClusteredDistribution`, along with new general content of the classdoc.
   
   ### Why are the changes needed?
   
   We figured out that in some case `HashClusteredDistribution` is still desirable other than stateful operators; `HashPartitioning` with subset of grouping keys can satisfy `ClusteredDistribution`, which means the cardinality of the subset of grouping keys technically defines the max parallelism. Increasing the number of partitions does not always help to solve the skew of the partitions.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Existing tests since this PR just renames a class.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #35551: [SPARK-38237][SQL] Rename back StatefulOpClusteredDistribution to HashClusteredDistribution

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #35551:
URL: https://github.com/apache/spark/pull/35551#issuecomment-1043944293


   @HeartSaVioR Could you remove the branch https://github.com/apache/spark/tree/SPARK-38237 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR closed pull request #35551: [SPARK-38237][SQL] Rename back StatefulOpClusteredDistribution to HashClusteredDistribution

Posted by GitBox <gi...@apache.org>.
HeartSaVioR closed pull request #35551:
URL: https://github.com/apache/spark/pull/35551


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on pull request #35551: [SPARK-38237][SQL] Rename back StatefulOpClusteredDistribution to HashClusteredDistribution

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35551:
URL: https://github.com/apache/spark/pull/35551#issuecomment-1042829337


   Sigh, I pushed to the wrong repo... I'll push to my repo and submit a PR again. Sorry for the noise.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HeartSaVioR commented on pull request #35551: [SPARK-38237][SQL] Rename back StatefulOpClusteredDistribution to HashClusteredDistribution

Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35551:
URL: https://github.com/apache/spark/pull/35551#issuecomment-1044041678


   Thanks for reminder! I just remove the branch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org