You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Mapred Learn <ma...@gmail.com> on 2011/06/24 23:15:51 UTC

sample usage of custom counters with new map Reduce API

Hi,
Could anyone point me to an example of custom counters with new map reduce
API ?

Thanks,
-JJ

Re: sample usage of custom counters with new map Reduce API

Posted by Mapred Learn <ma...@gmail.com>.

Thanks Shrijeet !
I will try it out

On Fri, Jun 24, 2011 at 2:27 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:

> public class MyReducer
>        extends Reducer<Text, Text, Text, Text> {
>
>  private enum MyCounters {
>    INPUT_UNIQUE_USERS
>  }
>
>  @Override
>  protected void setup(Context context) throws java.io.IOException,
>          InterruptedException {
>    Configuration conf = context.getConfiguration();
>    doSetup(conf);
>  }
>
>  @Override
>  public void reduce(Text key, Iterable<Text> values, Context context)
>          throws IOException, InterruptedException {
>    context.getCounter(MyCounters.INPUT_UNIQUE_USERS).increment(1);
>    doReduce();
>  }
>
>  @Override
>  protected void cleanup(Context context) throws java.io.IOException,
>          InterruptedException {
>      doClean();
>   }
> }
>
> On Fri, Jun 24, 2011 at 2:15 PM, Mapred Learn <ma...@gmail.com>
> wrote:
> > Hi,
> > Could anyone point me to an example of custom counters with new map
> reduce
> > API ?
> >
> > Thanks,
> > -JJ
>

Re: sample usage of custom counters with new map Reduce API

Posted by Mapred Learn <ma...@gmail.com>.

Hi Shrijeet,
Is there a way to do it in main class instead of mappers and reducers ?


On Fri, Jun 24, 2011 at 2:27 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:

> public class MyReducer
>        extends Reducer<Text, Text, Text, Text> {
>
>  private enum MyCounters {
>    INPUT_UNIQUE_USERS
>  }
>
>  @Override
>  protected void setup(Context context) throws java.io.IOException,
>          InterruptedException {
>    Configuration conf = context.getConfiguration();
>    doSetup(conf);
>  }
>
>  @Override
>  public void reduce(Text key, Iterable<Text> values, Context context)
>          throws IOException, InterruptedException {
>    context.getCounter(MyCounters.INPUT_UNIQUE_USERS).increment(1);
>    doReduce();
>  }
>
>  @Override
>  protected void cleanup(Context context) throws java.io.IOException,
>          InterruptedException {
>      doClean();
>   }
> }
>
> On Fri, Jun 24, 2011 at 2:15 PM, Mapred Learn <ma...@gmail.com>
> wrote:
> > Hi,
> > Could anyone point me to an example of custom counters with new map
> reduce
> > API ?
> >
> > Thanks,
> > -JJ
>

Re: sample usage of custom counters with new map Reduce API

Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.

public class MyReducer
        extends Reducer<Text, Text, Text, Text> {

  private enum MyCounters {
    INPUT_UNIQUE_USERS
  }

  @Override
  protected void setup(Context context) throws java.io.IOException,
          InterruptedException {
    Configuration conf = context.getConfiguration();
    doSetup(conf);
  }

  @Override
  public void reduce(Text key, Iterable<Text> values, Context context)
          throws IOException, InterruptedException {
    context.getCounter(MyCounters.INPUT_UNIQUE_USERS).increment(1);
    doReduce();
  }

  @Override
  protected void cleanup(Context context) throws java.io.IOException,
          InterruptedException {
      doClean();
  }
}

On Fri, Jun 24, 2011 at 2:15 PM, Mapred Learn <ma...@gmail.com> wrote:
> Hi,
> Could anyone point me to an example of custom counters with new map reduce
> API ?
>
> Thanks,
> -JJ