You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Harsh J <ha...@cloudera.com> on 2011/10/20 06:30:43 UTC
Re: fixing the mapper percentage viewer
Hello Mark,
Moving to mapreduce-dev@ (bcc'd common-user@).
You need to control whatever calls Reporter#setProgress(…). Mostly its
just the RecordReader implementation doing it via
RecordReader#getProgress(…).
Counters are not part of the progress reports.
On Thu, Oct 20, 2011 at 3:35 AM, Mark question <ma...@gmail.com> wrote:
> Hi all,
>
> I'm written a custom mapRunner, but it seems to have ruined the percentage
> shown for maps on console. I want to know which part of code is responsible
> for adjusting the percentage of maps ... Is it the following in MapRunner:
>
> if(incrProcCount) {
>
> reporter.incrCounter(SkipBadRecords.COUNTER_GROUP,
>
> SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS, 1);
>
>
> Thank you,
> Mark
>
--
Harsh J
Re: fixing the mapper percentage viewer
Posted by Harsh J <ha...@cloudera.com>.
Er, I meant to type mapreduce-user, not -dev :)
On Thu, Oct 20, 2011 at 10:00 AM, Harsh J <ha...@cloudera.com> wrote:
> Hello Mark,
>
> Moving to mapreduce-dev@ (bcc'd common-user@).
>
> You need to control whatever calls Reporter#setProgress(…). Mostly its
> just the RecordReader implementation doing it via
> RecordReader#getProgress(…).
>
> Counters are not part of the progress reports.
>
> On Thu, Oct 20, 2011 at 3:35 AM, Mark question <ma...@gmail.com> wrote:
>> Hi all,
>>
>> I'm written a custom mapRunner, but it seems to have ruined the percentage
>> shown for maps on console. I want to know which part of code is responsible
>> for adjusting the percentage of maps ... Is it the following in MapRunner:
>>
>> if(incrProcCount) {
>>
>> reporter.incrCounter(SkipBadRecords.COUNTER_GROUP,
>>
>> SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS, 1);
>>
>>
>> Thank you,
>> Mark
>>
>
>
>
> --
> Harsh J
>
--
Harsh J