You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/03/07 20:06:00 UTC
[jira] [Commented] (SPARK-33851) Push partial aggregates bellow
exchanges
[ https://issues.apache.org/jira/browse/SPARK-33851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296962#comment-17296962 ]
Apache Spark commented on SPARK-33851:
--------------------------------------
User 'tanelk' has created a pull request for this issue:
https://github.com/apache/spark/pull/31772
> Push partial aggregates bellow exchanges
> ----------------------------------------
>
> Key: SPARK-33851
> URL: https://issues.apache.org/jira/browse/SPARK-33851
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Tanel Kiis
> Priority: Major
>
> The `EnsureRequirements` rule inserts exchanges between partial and final aggregates to reduce the exchanged data amount. However, if the user manually partitions the data correctly, then partial aggregates will be after the exchange.
> If possible we should push partial aggregates bellow the manually inserted exchanges to reduce the exchanged data amount.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org