You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2016/11/02 06:14:58 UTC
[jira] [Commented] (SPARK-14960) Don't perform treeAggregation in
local mode
[ https://issues.apache.org/jira/browse/SPARK-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627902#comment-15627902 ]
Reynold Xin commented on SPARK-14960:
-------------------------------------
[~joshrosen] any reason why you closed the pull request?
> Don't perform treeAggregation in local mode
> -------------------------------------------
>
> Key: SPARK-14960
> URL: https://issues.apache.org/jira/browse/SPARK-14960
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Josh Rosen
>
> I don't think that {{treeAggregation}} will help performance in {{local}} mode and based on measurement of some unit tests it looks like it actually severely harms performance in certain cases. Therefore, treeAggregate should fall back to plain aggregate when running in local mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org