You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Yuming Wang (Jira)" <ji...@apache.org> on 2022/12/06 06:51:00 UTC

[jira] [Resolved] (SPARK-34987) AQE improve: change shuffle hash join to sort merge join when skewed shuffle hash join exists

     [ https://issues.apache.org/jira/browse/SPARK-34987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yuming Wang resolved SPARK-34987.
---------------------------------
    Resolution: Not A Problem

> AQE improve: change shuffle hash join to sort merge join when skewed shuffle hash join exists
> ---------------------------------------------------------------------------------------------
>
>                 Key: SPARK-34987
>                 URL: https://issues.apache.org/jira/browse/SPARK-34987
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.1, 3.1.1, 3.2.0
>            Reporter: exmy
>            Priority: Minor
>
> In our production, `spark.sql.join.preferSortMergeJoin` is false by default. AQE currently can only optimize skewed join for sort merge join, it will be better if we can change shuffle hash join to sort merge join when skewed shuffle hash join exists.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org