You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2014/06/24 09:28:25 UTC
[jira] [Updated] (SPARK-2253) Disable partial aggregation
automatically when reduction factor is low
[ https://issues.apache.org/jira/browse/SPARK-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin updated SPARK-2253:
-------------------------------
Description:
Once we see enough number of rows in partial aggregation and don't observe any reduction, Aggregator should just turn off partial aggregation. This reduces memory usage for high cardinality aggregations.
This one is for Spark core. There is another ticket tracking this for SQL.
was:
Once we see enough number of rows in partial aggregation and don't observe any reduction, Aggregator should just turn off partial aggregation.
This one is for Spark core. There is another ticket tracking this for SQL.
> Disable partial aggregation automatically when reduction factor is low
> ----------------------------------------------------------------------
>
> Key: SPARK-2253
> URL: https://issues.apache.org/jira/browse/SPARK-2253
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Reynold Xin
> Assignee: Reynold Xin
>
> Once we see enough number of rows in partial aggregation and don't observe any reduction, Aggregator should just turn off partial aggregation. This reduces memory usage for high cardinality aggregations.
> This one is for Spark core. There is another ticket tracking this for SQL.
--
This message was sent by Atlassian JIRA
(v6.2#6252)