You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "exmy (Jira)" <ji...@apache.org> on 2021/04/08 06:42:00 UTC

[jira] [Created] (SPARK-34987) AQE improve: change shuffle hash join to sort merge join when skewed shuffle hash join exists

exmy created SPARK-34987:
----------------------------

             Summary: AQE improve: change shuffle hash join to sort merge join when skewed shuffle hash join exists
                 Key: SPARK-34987
                 URL: https://issues.apache.org/jira/browse/SPARK-34987
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.1.1, 3.0.1, 3.2.0
            Reporter: exmy


In our production, `spark.sql.join.preferSortMergeJoin` is false by default. AQE currently can only optimize skewed join for sort merge join, it will be better if we can change shuffle hash join to sort merge join when skewed shuffle hash join exists.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org