You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Jay Vyas <ja...@gmail.com> on 2012/12/07 17:28:04 UTC
hadoop task assigner
Hi : Where are the hooks in hadoop for the implementation of locality? I
assume that Mappers are blind to locality - they read directly via their
Context objects, whatever splits they are given.
But the MapperContext is created by some kind of "smart", locality aware
Task assignment class.
What is this class and where is it? In particular, im interested in seeing
how to implement it for different InputFormat mechanisms
Thanks!!!!!!!!!!!!!
--
Jay Vyas
http://jayunit100.blogspot.com
Re: hadoop task assigner
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
In 1.0 line, you should look at JobTracker side, specifically JobInProgress.java.
Thanks,
+Vinod
On Dec 7, 2012, at 8:28 AM, Jay Vyas wrote:
> Hi : Where are the hooks in hadoop for the implementation of locality? I assume that Mappers are blind to locality - they read directly via their Context objects, whatever splits they are given.
>
> But the MapperContext is created by some kind of "smart", locality aware Task assignment class.
>
> What is this class and where is it? In particular, im interested in seeing how to implement it for different InputFormat mechanisms
>
> Thanks!!!!!!!!!!!!!
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
Re: hadoop task assigner
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
In 1.0 line, you should look at JobTracker side, specifically JobInProgress.java.
Thanks,
+Vinod
On Dec 7, 2012, at 8:28 AM, Jay Vyas wrote:
> Hi : Where are the hooks in hadoop for the implementation of locality? I assume that Mappers are blind to locality - they read directly via their Context objects, whatever splits they are given.
>
> But the MapperContext is created by some kind of "smart", locality aware Task assignment class.
>
> What is this class and where is it? In particular, im interested in seeing how to implement it for different InputFormat mechanisms
>
> Thanks!!!!!!!!!!!!!
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
Re: hadoop task assigner
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
In 1.0 line, you should look at JobTracker side, specifically JobInProgress.java.
Thanks,
+Vinod
On Dec 7, 2012, at 8:28 AM, Jay Vyas wrote:
> Hi : Where are the hooks in hadoop for the implementation of locality? I assume that Mappers are blind to locality - they read directly via their Context objects, whatever splits they are given.
>
> But the MapperContext is created by some kind of "smart", locality aware Task assignment class.
>
> What is this class and where is it? In particular, im interested in seeing how to implement it for different InputFormat mechanisms
>
> Thanks!!!!!!!!!!!!!
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
Re: hadoop task assigner
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
In 1.0 line, you should look at JobTracker side, specifically JobInProgress.java.
Thanks,
+Vinod
On Dec 7, 2012, at 8:28 AM, Jay Vyas wrote:
> Hi : Where are the hooks in hadoop for the implementation of locality? I assume that Mappers are blind to locality - they read directly via their Context objects, whatever splits they are given.
>
> But the MapperContext is created by some kind of "smart", locality aware Task assignment class.
>
> What is this class and where is it? In particular, im interested in seeing how to implement it for different InputFormat mechanisms
>
> Thanks!!!!!!!!!!!!!
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com