[jira] [Commented] (SPARK-9808) Remove hash shuffle file consolidation

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805144#comment-14805144 ] 

Apache Spark commented on SPARK-9808:
-------------------------------------

User 'rxin' has created a pull request for this issue:
https://github.com/apache/spark/pull/8812

> Remove hash shuffle file consolidation
> --------------------------------------
>
>                 Key: SPARK-9808
>                 URL: https://issues.apache.org/jira/browse/SPARK-9808
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle, Spark Core
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> I think that we should remove {{spark.shuffle.consolidateFiles}} and its associated implementation for Spark 1.5.0. This feature isn't properly tested and does not work with the external shuffle service. Since it's likely to be buggy and since its motivation has been subsumed by sort-based shuffle, I think it's a prime candidate for removal now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org