[jira] [Commented] (SPARK-1956) Enable shuffle consolidation by default

["Mridul Muralidharan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011741#comment-14011741 ] 

Mridul Muralidharan commented on SPARK-1956:
--------------------------------------------

shuffle consolidation MUST NOT be enabled - whether by default, or intentionally.
In 1.0, it is very badly broken - we have a whole litany of fixes for it, before it was reasonably stable.

Current plan is to contribute most of these back in 1.1 timeframe.

> Enable shuffle consolidation by default
> ---------------------------------------
>
>                 Key: SPARK-1956
>                 URL: https://issues.apache.org/jira/browse/SPARK-1956
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle, Spark Core
>    Affects Versions: 1.0.0
>            Reporter: Sandy Ryza
>
> The only drawbacks are on ext3, and most everyone has ext4 at this point.  I think it's better to aim the default at the common case.



--
This message was sent by Atlassian JIRA
(v6.2#6252)