You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Samaneh Shokuhi <sa...@gmail.com> on 2012/11/25 23:18:53 UTC

Straggler map tasks

Hi ,

I need to detect the straggler map tasks specifically the last one to do
some experiments .As i undestood job tracker is aware of all map tasks ,so
is there any way to know the current  running map task is the
last(straggler)  one  ?


Samaneh

Re: Straggler map tasks

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Yes, you should look at JobInProgress class, and search for 'speculative'. JobTracker already has logic to launch speculative copies for tasks which are progressing at a degraded rate relative to other tasks.

HTH,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Nov 25, 2012, at 2:18 PM, Samaneh Shokuhi wrote:

> Hi ,
> 
> I need to detect the straggler map tasks specifically the last one to do
> some experiments .As i undestood job tracker is aware of all map tasks ,so
> is there any way to know the current  running map task is the
> last(straggler)  one  ?
> 
> 
> Samaneh