You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Peter Toth (Jira)" <ji...@apache.org> on 2020/01/21 15:53:00 UTC
[jira] [Updated] (SPARK-30598) Detect equijoins better
[ https://issues.apache.org/jira/browse/SPARK-30598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Toth updated SPARK-30598:
-------------------------------
Issue Type: Improvement (was: Bug)
> Detect equijoins better
> -----------------------
>
> Key: SPARK-30598
> URL: https://issues.apache.org/jira/browse/SPARK-30598
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Peter Toth
> Priority: Minor
>
> The following 2 query produce different plans, as the second one is not recognised as equijoin.
> {noformat}
> SELECT * FROM t1 FULL OUTER JOIN t2 ON t1.c2 = 2 AND t2.c2 = 2 AND t1.c = t2.c
> SortMergeJoin [c#225], [c#236], FullOuter, ((c2#226 = 2) AND (c2#237 = 2))
> :- *(2) Sort [c#225 ASC NULLS FIRST], false, 0
> : +- Exchange hashpartitioning(c#225, 5), true, [id=#101]
> : +- *(1) Project [_1#220 AS c#225, _2#221 AS c2#226]
> : +- *(1) LocalTableScan [_1#220, _2#221]
> +- *(4) Sort [c#236 ASC NULLS FIRST], false, 0
> +- Exchange hashpartitioning(c#236, 5), true, [id=#106]
> +- *(3) Project [_1#231 AS c#236, _2#232 AS c2#237]
> +- *(3) LocalTableScan [_1#231, _2#232]
> {noformat}
> {noformat}
> SELECT * FROM t1 FULL OUTER JOIN t2 ON t1.c2 = 2 AND t2.c2 = 2
> BroadcastNestedLoopJoin BuildRight, FullOuter, ((c2#226 = 2) AND (c2#237 = 2))
> :- *(1) Project [_1#220 AS c#225, _2#221 AS c2#226]
> : +- *(1) LocalTableScan [_1#220, _2#221]
> +- BroadcastExchange IdentityBroadcastMode, [id=#146]
> +- *(2) Project [_1#231 AS c#236, _2#232 AS c2#237]
> +- *(2) LocalTableScan [_1#231, _2#232]
> {noformat}
> We could detect the implicit equalities from the join condition.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org