You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/08/21 16:35:45 UTC

[jira] [Assigned] (SPARK-9708) Spark should create local temporary directories in Mesos sandbox when launched with Mesos

     [ https://issues.apache.org/jira/browse/SPARK-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-9708:
-----------------------------------

    Assignee:     (was: Apache Spark)

> Spark should create local temporary directories in Mesos sandbox when launched with Mesos
> -----------------------------------------------------------------------------------------
>
>                 Key: SPARK-9708
>                 URL: https://issues.apache.org/jira/browse/SPARK-9708
>             Project: Spark
>          Issue Type: Bug
>          Components: Mesos
>            Reporter: Timothy Chen
>
> Currently Spark creates temporary directories with Utils.getConfiguredLocalDirs, and it writes to YARN directories if YARN is detected, otherwise just writes in a temporary directory in the host.
> However, Mesos does create a directory per task and ideally Spark should use that directory to create its local temporary directories since it then can be cleaned up when the task is gone and not left on the host or cleaned until reboot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org