You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2016/09/24 00:17:20 UTC
[jira] [Created] (SPARK-17653) Optimizer should remove unnecessary
distincts (in multiple unions)
Reynold Xin created SPARK-17653:
-----------------------------------
Summary: Optimizer should remove unnecessary distincts (in multiple unions)
Key: SPARK-17653
URL: https://issues.apache.org/jira/browse/SPARK-17653
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.0.0
Reporter: Reynold Xin
Query:
{code}
select 1 a union select 2 b union select 3 c
{code}
Explain plan:
{code}
== Physical Plan ==
*HashAggregate(keys=[a#13], functions=[])
+- Exchange hashpartitioning(a#13, 200)
+- *HashAggregate(keys=[a#13], functions=[])
+- Union
:- *HashAggregate(keys=[a#13], functions=[])
: +- Exchange hashpartitioning(a#13, 200)
: +- *HashAggregate(keys=[a#13], functions=[])
: +- Union
: :- *Project [1 AS a#13]
: : +- Scan OneRowRelation[]
: +- *Project [2 AS b#14]
: +- Scan OneRowRelation[]
+- *Project [3 AS c#15]
+- Scan OneRowRelation[]
{code}
Only one distinct should be necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org