You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "pankaj arora (JIRA)" <ji...@apache.org> on 2015/02/25 19:30:06 UTC

[jira] [Created] (SPARK-6011) Out of disk space due to Spark not deleting shuffle files of lost executors

pankaj arora created SPARK-6011:
-----------------------------------

             Summary: Out of disk space due to Spark not deleting shuffle files of lost executors
                 Key: SPARK-6011
                 URL: https://issues.apache.org/jira/browse/SPARK-6011
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 1.2.1
         Environment: Running Spark in Yarn-Client mode
            Reporter: pankaj arora
             Fix For: 1.3.1


If Executors gets lost abruptly spark does not delete its shuffle files till application ends.
Ours is long running application which is serving requests received through REST APIs and if any of the executor gets lost shuffle files are not deleted and that leads to local disk going out of space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org