You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yin Huai (JIRA)" <ji...@apache.org> on 2017/03/10 19:15:05 UTC
[jira] [Resolved] (SPARK-19620) Incorrect exchange coordinator Id
in physical plan
[ https://issues.apache.org/jira/browse/SPARK-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai resolved SPARK-19620.
------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0
Issue resolved by pull request 16952
[https://github.com/apache/spark/pull/16952]
> Incorrect exchange coordinator Id in physical plan
> --------------------------------------------------
>
> Key: SPARK-19620
> URL: https://issues.apache.org/jira/browse/SPARK-19620
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: Carson Wang
> Priority: Minor
> Fix For: 2.2.0
>
>
> When adaptive execution is enabled, an exchange coordinator is used to in the Exchange operators. For Join, the same exchange coordinator is used for its two Exchanges. But the physical plan shows two different coordinator Ids which is confusing.
> Here is an example:
> {code}
> == Physical Plan ==
> *Project [key1#3L, value2#12L]
> +- *SortMergeJoin [key1#3L], [key2#11L], Inner
> :- *Sort [key1#3L ASC NULLS FIRST], false, 0
> : +- Exchange(coordinator id: 1804587700) hashpartitioning(key1#3L, 10), coordinator[target post-shuffle partition size: 67108864]
> : +- *Project [(id#0L % 500) AS key1#3L]
> : +- *Filter isnotnull((id#0L % 500))
> : +- *Range (0, 1000, step=1, splits=Some(10))
> +- *Sort [key2#11L ASC NULLS FIRST], false, 0
> +- Exchange(coordinator id: 793927319) hashpartitioning(key2#11L, 10), coordinator[target post-shuffle partition size: 67108864]
> +- *Project [(id#8L % 500) AS key2#11L, id#8L AS value2#12L]
> +- *Filter isnotnull((id#8L % 500))
> +- *Range (0, 1000, step=1, splits=Some(10))
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org