You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2022/10/09 08:38:00 UTC

[jira] [Commented] (SPARK-40715) Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size

    [ https://issues.apache.org/jira/browse/SPARK-40715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614652#comment-17614652 ] 

Apache Spark commented on SPARK-40715:
--------------------------------------

User 'wankunde' has created a pull request for this issue:
https://github.com/apache/spark/pull/38176

> Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size
> ----------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-40715
>                 URL: https://issues.apache.org/jira/browse/SPARK-40715
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: Wan Kun
>            Priority: Major
>
> Now DynamicJoinSelection will prefer shuffled hash join only if *advisoryPartitionSize <= maxShuffledHashJoinLocalMapThreshold*
> We can also prefer shuffled hash join if the partitions in one side are smaller are smaller than advisoryPartitionSize, and the partitions in the other side are all larger enough(larger than advisoryPartitionSize)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org