You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tanel Kiis (Jira)" <ji...@apache.org> on 2020/12/19 18:51:00 UTC

[jira] [Created] (SPARK-33851) Push partial aggregates bellow exchanges

Tanel Kiis created SPARK-33851:
----------------------------------

             Summary: Push partial aggregates bellow exchanges
                 Key: SPARK-33851
                 URL: https://issues.apache.org/jira/browse/SPARK-33851
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: Tanel Kiis


The `EnsureRequirements` rule inserts exchanges between partial and final aggregates to reduce the exchanged data amount. However, if the user manually partitions the data correctly, then partial aggregates will be after the exchange.

If possible we should push partial aggregates bellow the manually inserted exchanges to reduce the exchanged data amount.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org