You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "Haibo Chen (JIRA)" <ji...@apache.org> on 2016/03/04 18:45:40 UTC

[jira] [Created] (YARN-4766) NM should not aggregate logs older than the retention policy

Haibo Chen created YARN-4766:
--------------------------------

             Summary: NM should not aggregate logs older than the retention policy
                 Key: YARN-4766
                 URL: https://issues.apache.org/jira/browse/YARN-4766
             Project: Hadoop YARN
          Issue Type: Bug
          Components: log-aggregation, nodemanager
            Reporter: Haibo Chen
            Assignee: Haibo Chen


When a log aggregation fails on the NM the information is for the attempt is kept in the recovery DB. Log aggregation can fail for multiple reasons which are often related to HDFS space or permissions.

On restart the recovery DB is read and if an application attempt needs its logs aggregated, the files are scheduled for aggregation without any checks. The log files could be older than the retention limit in which case we should not aggregate them but immediately mark them for deletion from the local file system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)