You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tanel Kiis (Jira)" <ji...@apache.org> on 2021/06/03 12:18:00 UTC

[jira] [Created] (SPARK-35630) ExpandExec should not introduce unnecessary exchanges

Tanel Kiis created SPARK-35630:
----------------------------------

             Summary: ExpandExec should not introduce unnecessary exchanges
                 Key: SPARK-35630
                 URL: https://issues.apache.org/jira/browse/SPARK-35630
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: Tanel Kiis


The Expand is commonly introduced by the RewriteDistinctAggregates optimizer rule.
In that case there can be several attributes that are kept as they are by the Expand.
If the child's output is partitioned by those attributes, then so will be the output of the Expand.
In general case the Expand can output data with arbitrary partitioning, so set it as UNKNOWN partitioning.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org