Hadoop 0.21 :: job.getCounters() returns null?

[Felix Halim <fe...@gmail.com>]

I tried hadoop 0.21 release candidate.

job.waitForCompletion(true);
Counters ctrs = job.getCounters();
// here ctrs is null


In the previous hadoop version 0.20.2 it worked fine for all times.

Is this a bug in 0.21 ?
Or i'm missing some settings?

Thanks,

Felix Halim

Re: Hadoop 0.21 :: job.getCounters() returns null?

[Felix Halim <fe...@gmail.com>]

Hi Tom,

Thanks for notifying.

This issue really gives me a hard time circumventing the problem...
I even tried the job.submit() and then read the counter asynchronously...
But found out that the values are always zero...

What Amareshwari Sriramadasu suggested works!

I added these 3 parameters in mapred-site.xml :

	<property>
		<name>mapreduce.jobtracker.retirejobs</name>
		<value>false</value>
	</property>

	<property>
		<name>mapreduce.jobtracker.persist.jobstatus.active</name>
		<value>true</value>
	</property>

	<property>
		<name>mapreduce.jobtracker.persist.jobstatus.hours</name>
		<value>1</value>
	</property>


All work fine now.

I'm now testing it with a very large input to see what's the
performance improvement over the 0.20.2

I heard the 0.21 has a new shuffling :), I'm keeping my hope high...


Felix Halim



On Wed, Jul 7, 2010 at 7:19 PM, Tom White <to...@cloudera.com> wrote:
> Hi Felix,
>
> Aaron Kimball hit the same problem - it's being discussed at
> https://issues.apache.org/jira/browse/MAPREDUCE-1920.
>
> Thanks for reporting this.
>
> Cheers,
> Tom
>
> On Tue, Jul 6, 2010 at 11:26 AM, Felix Halim <fe...@gmail.com> wrote:
>> I tried hadoop 0.21 release candidate.
>>
>> job.waitForCompletion(true);
>> Counters ctrs = job.getCounters();
>> // here ctrs is null
>>
>>
>> In the previous hadoop version 0.20.2 it worked fine for all times.
>>
>> Is this a bug in 0.21 ?
>> Or i'm missing some settings?
>>
>> Thanks,
>>
>> Felix Halim
>>
>

Re: Hadoop 0.21 :: job.getCounters() returns null?

[Tom White <to...@cloudera.com>]

Hi Felix,

Aaron Kimball hit the same problem - it's being discussed at
https://issues.apache.org/jira/browse/MAPREDUCE-1920.

Thanks for reporting this.

Cheers,
Tom

On Tue, Jul 6, 2010 at 11:26 AM, Felix Halim <fe...@gmail.com> wrote:
> I tried hadoop 0.21 release candidate.
>
> job.waitForCompletion(true);
> Counters ctrs = job.getCounters();
> // here ctrs is null
>
>
> In the previous hadoop version 0.20.2 it worked fine for all times.
>
> Is this a bug in 0.21 ?
> Or i'm missing some settings?
>
> Thanks,
>
> Felix Halim
>