You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mike S <mi...@gmail.com> on 2012/07/10 00:37:41 UTC
Hadoop Job Profiling Tool
Is there any Hadoop specific tool that can profile a job time? For
instance, if a job took x amount of time, the tool can profile how
this time was consumed at different stage likes time spent to read, to
map, to reduce reduce , shuttle move that and the different steps of
a Hadoop M/R job?
If not, how do you recommend finding the map/reduce job bottle neck?
Re: Hadoop Job Profiling Tool
Posted by Nitin Pawar <ni...@gmail.com>.
job history has most of the details you are looking for at each task levels
On Tue, Jul 10, 2012 at 4:07 AM, Mike S <mi...@gmail.com> wrote:
> Is there any Hadoop specific tool that can profile a job time? For
> instance, if a job took x amount of time, the tool can profile how
> this time was consumed at different stage likes time spent to read, to
> map, to reduce reduce , shuttle move that and the different steps of
> a Hadoop M/R job?
>
> If not, how do you recommend finding the map/reduce job bottle neck?
>
--
Nitin Pawar