You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/09/02 15:04:00 UTC
[jira] [Commented] (SPARK-36652) AQE dynamic join selection should
not apply to non-equi join
[ https://issues.apache.org/jira/browse/SPARK-36652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17408914#comment-17408914 ]
Apache Spark commented on SPARK-36652:
--------------------------------------
User 'c21' has created a pull request for this issue:
https://github.com/apache/spark/pull/33899
> AQE dynamic join selection should not apply to non-equi join
> ------------------------------------------------------------
>
> Key: SPARK-36652
> URL: https://issues.apache.org/jira/browse/SPARK-36652
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.0, 3.3.0
> Reporter: Cheng Su
> Priority: Minor
>
> Currently `DynamicJoinSelection` has two features: 1.demote broadcast hash join, and 2.promote shuffled hash join. Both are achieved by adding join hint in query plan, and only works for equi join. However the rule is matching with `Join` operator now - [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/DynamicJoinSelection.scala#L71,] so it would add hint for non-equi join by mistake.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org