You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Martin Loncaric (JIRA)" <ji...@apache.org> on 2019/02/28 20:22:00 UTC

[jira] [Created] (SPARK-27014) Support removal of jars and Spark binaries from Mesos driver and executor sandboxes

Martin Loncaric created SPARK-27014:
---------------------------------------

             Summary: Support removal of jars and Spark binaries from Mesos driver and executor sandboxes
                 Key: SPARK-27014
                 URL: https://issues.apache.org/jira/browse/SPARK-27014
             Project: Spark
          Issue Type: New Feature
          Components: Mesos
    Affects Versions: 2.4.0
            Reporter: Martin Loncaric


Currently, each Spark application run on Mesos leaves behind at least 500MB of data in sandbox directories, coming from Spark binaries and copied URIs. These can build up as a disk leak, causing major issues on Mesos clusters unless their grace period for sandbox directories is very short.

Spark should have a feature to delete these (from both driver and executor sandboxes) on teardown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org