You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2012/10/15 23:32:42 UTC

33%, 66%, and 100% *reducer* optimization

Hi guys !

We all know that there are major milestones in reducers (33%, 66%)

In particular:

Im wondering: What should we optimize for in case a job hangs at 33%, as
compared with 66% ?

-- 
Jay Vyas
http://jayunit100.blogspot.com

Re: 33%, 66%, and 100% *reducer* optimization

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Reduce has three phases - shuffle, sort and reduce.

So, 33% would imply the shuffle phase end, and 66% would refer to the end of sort phase.

Thanks,
+Vinod

On Oct 15, 2012, at 2:32 PM, Jay Vyas wrote:

> Hi guys !
> 
> We all know that there are major milestones in reducers (33%, 66%)
> 
> In particular:
> 
> Im wondering: What should we optimize for in case a job hangs at 33%, as
> compared with 66% ?
> 
> -- 
> Jay Vyas
> http://jayunit100.blogspot.com