You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Shi Yu <sh...@uchicago.edu> on 2011/02/01 20:17:30 UTC

Reduce progress goes backward?

Hi,

I observe that sometimes the map/reduce progress is going backward. What 
does this mean?


11/02/01 12:57:51 INFO mapred.JobClient:  map 100% reduce 99%
11/02/01 12:59:14 INFO mapred.JobClient:  map 100% reduce 98%
11/02/01 12:59:45 INFO mapred.JobClient:  map 100% reduce 99%
11/02/01 13:03:24 INFO mapred.JobClient:  map 100% reduce 98%
11/02/01 13:09:16 INFO mapred.JobClient:  map 100% reduce 97%
11/02/01 13:09:19 INFO mapred.JobClient:  map 100% reduce 96%
11/02/01 13:11:14 INFO mapred.JobClient:  map 100% reduce 97%
11/02/01 13:12:33 INFO mapred.JobClient:  map 100% reduce 96%
11/02/01 13:13:05 INFO mapred.JobClient:  map 100% reduce 95%


Shi

Re: Reduce progress goes backward?

Posted by James Seigel <ja...@tynt.com>.

It means that the scheduler is killing off some of your reducer jobs or some of them are dieing.  Maybe they are taking too long.  You should check out your job tracker and look at some of the details and then drill down to see if you are getting any errors in some of your reducers.

Cheers
James
On 2011-02-01, at 12:17 PM, Shi Yu wrote:

> Hi,
> 
> I observe that sometimes the map/reduce progress is going backward. What does this mean?
> 
> 
> 11/02/01 12:57:51 INFO mapred.JobClient:  map 100% reduce 99%
> 11/02/01 12:59:14 INFO mapred.JobClient:  map 100% reduce 98%
> 11/02/01 12:59:45 INFO mapred.JobClient:  map 100% reduce 99%
> 11/02/01 13:03:24 INFO mapred.JobClient:  map 100% reduce 98%
> 11/02/01 13:09:16 INFO mapred.JobClient:  map 100% reduce 97%
> 11/02/01 13:09:19 INFO mapred.JobClient:  map 100% reduce 96%
> 11/02/01 13:11:14 INFO mapred.JobClient:  map 100% reduce 97%
> 11/02/01 13:12:33 INFO mapred.JobClient:  map 100% reduce 96%
> 11/02/01 13:13:05 INFO mapred.JobClient:  map 100% reduce 95%
> 
> 
> Shi
>