You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ulysses you (Jira)" <ji...@apache.org> on 2021/05/06 01:38:00 UTC
[jira] [Updated] (SPARK-35282) Support AQE side shuffled hash join
formula
[ https://issues.apache.org/jira/browse/SPARK-35282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ulysses you updated SPARK-35282:
--------------------------------
Parent: SPARK-33828
Issue Type: Sub-task (was: New Feature)
> Support AQE side shuffled hash join formula
> -------------------------------------------
>
> Key: SPARK-35282
> URL: https://issues.apache.org/jira/browse/SPARK-35282
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: ulysses you
> Priority: Major
>
> Use AQE runtime statistics to decide if we can use shuffled hash join instead of sort merge join. Currently, the formula of shuffled hash join selection is not worked due to the dymanic shuffle partition number.
>
> Add a new config `spark.sql.adaptive.shuffledHashJoinLocalMapThreshold` to decide if join can be converted to shuffled hash join safely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org