You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wan Kun (Jira)" <ji...@apache.org> on 2022/10/09 07:57:00 UTC

[jira] [Created] (SPARK-40715) Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size

Wan Kun created SPARK-40715:
-------------------------------

             Summary: Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size
                 Key: SPARK-40715
                 URL: https://issues.apache.org/jira/browse/SPARK-40715
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.4.0
            Reporter: Wan Kun


Now DynamicJoinSelection will prefer shuffled hash join only if *advisoryPartitionSize <= maxShuffledHashJoinLocalMapThreshold*
We can also prefer shuffled hash join if the partitions in one side are smaller are smaller than advisoryPartitionSize, and the partitions in the other side are all larger enough(larger than advisoryPartitionSize)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org