[jira] [Created] (YARN-6356) Allow different values of yarn.log-aggregation.retain-seconds for succeeded and failed jobs

["Robert Kanter (JIRA)" <ji...@apache.org>]

Robert Kanter created YARN-6356:
-----------------------------------

             Summary: Allow different values of yarn.log-aggregation.retain-seconds for succeeded and failed jobs
                 Key: YARN-6356
                 URL: https://issues.apache.org/jira/browse/YARN-6356
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: log-aggregation
            Reporter: Robert Kanter


It would be useful to have a value of {{yarn.log-aggregation.retain-seconds}} for succeeded jobs and a different value for failed/killed jobs.  For jobs that succeeded, you typically don't care about the logs, so a shorter retention time is fine (and saves space/blocks in HDFS).  For jobs that failed or were killed, the logs are much more important, and it's likely to want to keep them around for longer so you have time to look at them.

For instance, you could set it to keep logs for succeeded jobs for 1 day and logs for failed/killed jobs for 1 week.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org