You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ulysses you (Jira)" <ji...@apache.org> on 2021/04/29 02:42:00 UTC

[jira] [Created] (SPARK-35264) Support AQE side broadcastJoin threshold

ulysses you created SPARK-35264:
-----------------------------------

             Summary: Support AQE side broadcastJoin threshold
                 Key: SPARK-35264
                 URL: https://issues.apache.org/jira/browse/SPARK-35264
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: ulysses you


The main idea here is that make join config isolation between normal planner and aqe planner which shared the same code path.
In order to achieve this we use a specific join hint in advance during AQE framework and then at JoinSelection side it will take and follow the inserted hint.

For now we only support select strategy for equi join, and follow this order
 1. mark join as broadcast hash join if possible
 2. mark join as shuffled hash join if possible
Note that, we don't override join strategy if user specifies a join hint.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org