You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-dev@hadoop.apache.org by "Marco Gaido (JIRA)" <ji...@apache.org> on 2018/05/31 17:56:00 UTC

[jira] [Created] (YARN-8385) Clean local directories when a container is killed

Marco Gaido created YARN-8385:
---------------------------------

             Summary: Clean local directories when a container is killed
                 Key: YARN-8385
                 URL: https://issues.apache.org/jira/browse/YARN-8385
             Project: Hadoop YARN
          Issue Type: Bug
    Affects Versions: 2.7.0
            Reporter: Marco Gaido


In long running applications, it may happen that many containers are created and killed. A use case is Spark Thrift Server when dynamic allocation is enabled. A lot of containers are killed and the application keeps running indefinitely.

Currently, YARN seems to remove the local directories only when the whole application terminates. In the scenario described above, this can cause serious resource leakages. Please, check https://issues.apache.org/jira/browse/SPARK-22575.

I think YARN should clean up all the local directories of a container when it is killed and not when the whole application terminates.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org