You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Gera Shegalov (JIRA)" <ji...@apache.org> on 2014/05/04 03:34:14 UTC

[jira] [Created] (MAPREDUCE-5875) Make Counter limits consistent conf across JobClient, MRAppMaster, and YarnChild

Gera Shegalov created MAPREDUCE-5875:
----------------------------------------

             Summary: Make Counter limits consistent conf across JobClient, MRAppMaster, and YarnChild
                 Key: MAPREDUCE-5875
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: applicationmaster, client, task
    Affects Versions: 2.4.0
            Reporter: Gera Shegalov
            Assignee: Gera Shegalov


Currently, counter limits "mapreduce.job.counters.*" handled by {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized asymmetrically: on the client side, and on the AM, job.xml is ignored whereas it's taken into account in YarnChild.

It would be good to make the Limits job-configurable, such that max counters/groups is only increased when needed. With the current Limits implementation relying on static constants, it's going to be challenging for tools that submit jobs concurrently (e.g. via class loading isolation).

The patch that I am uploading is not perfect but demonstrates the issue. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)