You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2012/10/15 23:32:42 UTC
33%, 66%, and 100% *reducer* optimization
Hi guys !
We all know that there are major milestones in reducers (33%, 66%)
In particular:
Im wondering: What should we optimize for in case a job hangs at 33%, as
compared with 66% ?
--
Jay Vyas
http://jayunit100.blogspot.com
Re: 33%, 66%, and 100% *reducer* optimization
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Reduce has three phases - shuffle, sort and reduce.
So, 33% would imply the shuffle phase end, and 66% would refer to the end of sort phase.
Thanks,
+Vinod
On Oct 15, 2012, at 2:32 PM, Jay Vyas wrote:
> Hi guys !
>
> We all know that there are major milestones in reducers (33%, 66%)
>
> In particular:
>
> Im wondering: What should we optimize for in case a job hangs at 33%, as
> compared with 66% ?
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com