[jira] [Updated] (MAPREDUCE-5875) Make Counter limits consistent conf across JobClient, MRAppMaster, and YarnChild

["Gera Shegalov (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gera Shegalov updated MAPREDUCE-5875:
-------------------------------------

    Attachment: MAPREDUCE-5875.v04.patch

Hi [~kasha], thanks for the review. I addressed 2. As for 1, I prefer to consistently check whether log level is enabled regardless of the actual level.

> Make Counter limits consistent conf across JobClient, MRAppMaster, and YarnChild
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5875
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster, client, task
>    Affects Versions: 2.4.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>         Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch, MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized asymmetrically: on the client side, and on the AM, job.xml is ignored whereas it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max counters/groups is only increased when needed. With the current Limits implementation relying on static constants, it's going to be challenging for tools that submit jobs concurrently  without resorting to class loading isolation.
> The patch that I am uploading is not perfect but demonstrates the issue. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)