You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Aaron Davidson (JIRA)" <ji...@apache.org> on 2014/05/17 05:10:14 UTC

[jira] [Created] (SPARK-1865) Improve behavior of cleanup of disk state

Aaron Davidson created SPARK-1865:
-------------------------------------

             Summary: Improve behavior of cleanup of disk state
                 Key: SPARK-1865
                 URL: https://issues.apache.org/jira/browse/SPARK-1865
             Project: Spark
          Issue Type: Improvement
          Components: Deploy, Spark Core
            Reporter: Aaron Davidson


Right now the behavior of disk cleanup is centered around the exit hook of the executor, which attempts to cleanup shuffle files and disk manager blocks, but may fail. We should make this behavior more predictable, perhaps by letting the Standalone Worker cleanup the disk state, and adding a flag to disable having the executor cleanup its own state.



--
This message was sent by Atlassian JIRA
(v6.2#6252)