You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Praveen Sripati <pr...@gmail.com> on 2011/12/24 05:11:27 UTC

Limiting the usage of counters

Hi,

I find the below code in 203, 205 and 1.0 and not in trunk and other
releases. Is it not in trunk or done some other way? Also, noticed that
some of the parameters are not configurable and hard-coded.

/** limit on the size of the name of the group **
  private static final int GROUP_NAME_LIMIT = 128;
  /** limit on the size of the counter name **/
  private static final int COUNTER_NAME_LIMIT = 64;

  private static final JobConf conf = new JobConf();
  /** limit on counters **/
  public static int MAX_COUNTER_LIMIT =
    conf.getInt("mapreduce.job.counters.limit", 120);

  /** the max groups allowed **/
  static final int MAX_GROUP_LIMIT = 50;

Regards,
Praveen

Re: Limiting the usage of counters

Posted by Praveen Sripati <pr...@gmail.com>.

Hi,

I found out the following in MRJobConfig.java in 0.23 and trunk and missing
in 0.22. The configuration parameters have been changed between 0.20.* to
0.23/trunk and so initially could not find it in the code.

  public static final String COUNTERS_MAX_KEY =
"mapreduce.job.counters.max";
  public static final int COUNTERS_MAX_DEFAULT = 120;

  public static final String COUNTER_GROUP_NAME_MAX_KEY =
"mapreduce.job.counters.group.name.max";
  public static final int COUNTER_GROUP_NAME_MAX_DEFAULT = 128;

  public static final String COUNTER_NAME_MAX_KEY =
"mapreduce.job.counters.counter.name.max";
  public static final int COUNTER_NAME_MAX_DEFAULT = 64;

  public static final String COUNTER_GROUPS_MAX_KEY =
"mapreduce.job.counters.groups.max";
  public static final int COUNTER_GROUPS_MAX_DEFAULT = 50;

Regards,
Praveen

On Sat, Dec 24, 2011 at 9:41 AM, Praveen Sripati
<pr...@gmail.com>wrote:

> Hi,
>
> I find the below code in 203, 205 and 1.0 and not in trunk and other
> releases. Is it not in trunk or done some other way? Also, noticed that
> some of the parameters are not configurable and hard-coded.
>
> /** limit on the size of the name of the group **
>   private static final int GROUP_NAME_LIMIT = 128;
>   /** limit on the size of the counter name **/
>   private static final int COUNTER_NAME_LIMIT = 64;
>
>   private static final JobConf conf = new JobConf();
>   /** limit on counters **/
>   public static int MAX_COUNTER_LIMIT =
>     conf.getInt("mapreduce.job.counters.limit", 120);
>
>   /** the max groups allowed **/
>   static final int MAX_GROUP_LIMIT = 50;
>
> Regards,
> Praveen
>