[jira] [Created] (MAPREDUCE-5773) Provide dedicated MRAppMaster syslog length limit

["Gera Shegalov (JIRA)" <ji...@apache.org>]

Gera Shegalov created MAPREDUCE-5773:
----------------------------------------

             Summary: Provide dedicated MRAppMaster syslog length limit
                 Key: MAPREDUCE-5773
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5773
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mr-am
    Affects Versions: 2.3.0
            Reporter: Gera Shegalov
            Assignee: Gera Shegalov
            Priority: Blocker
             Fix For: 2.4.0


Currently mapreduce.task.userlog.limit.kb controls both the length of task attempt logs and MA-AM attempt logs. Obviously, MR-AM log is not userlog.
We are interested in keeping MR-AM log either in its entirety or in much larger size than task logs for debugging. 

MAPREDUCE-5672 introduced CRLA. We already use a dedicated setting for how many backups of rolled files we keep for MR-AM. However, for large jobs with tens of megabyte AM log, it means that you have a long series of files. A natural improvement is to have a configuration yarn.app.mapreduce.am.container.log.limit.kb . It allows to either disable rolling the log for altogether while keeping it for the task attempts, or to use much a larger limit to make rolling less frequent. 




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)