You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by bikash sharma <sh...@gmail.com> on 2011/09/28 20:45:39 UTC
getting the process id of mapreduce tasks
Hi,
Is it possible to get the process id of each task in a MapReduce job?
When I run a mapreduce job and do a monitoring in linux using ps, i just see
the id of the mapreduce job process but not its constituent map/reduce
tasks.
The use case is to monitor the resource usage of each task by using sar
utility in linux with specific process id of task.
Thanks,
Bikash
Re: getting the process id of mapreduce tasks
Posted by bikash sharma <sh...@gmail.com>.
Thanks Varad.
On Wed, Sep 28, 2011 at 9:35 PM, Varad Meru <me...@gmail.com> wrote:
> The process ids of each individual task can be seen using jps and jconsole
> commands provided by java.
>
> jconsole command on command-line interface provides a GUI screen for
> monitoring running tasks within java.
>
> The tasks are only visible as java virtual machine instance in the os
> system monitoring tool.
>
>
> Regards,
> Varad Meru
> -----------------------------------
> Sent from my iPod
>
> On 29-Sep-2011, at 0:15, bikash sharma <sh...@gmail.com> wrote:
>
> > Hi,
> > Is it possible to get the process id of each task in a MapReduce job?
> > When I run a mapreduce job and do a monitoring in linux using ps, i just
> see
> > the id of the mapreduce job process but not its constituent map/reduce
> > tasks.
> > The use case is to monitor the resource usage of each task by using sar
> > utility in linux with specific process id of task.
> >
> > Thanks,
> > Bikash
>
Re: getting the process id of mapreduce tasks
Posted by Varad Meru <me...@gmail.com>.
The process ids of each individual task can be seen using jps and jconsole commands provided by java.
jconsole command on command-line interface provides a GUI screen for monitoring running tasks within java.
The tasks are only visible as java virtual machine instance in the os system monitoring tool.
Regards,
Varad Meru
-----------------------------------
Sent from my iPod
On 29-Sep-2011, at 0:15, bikash sharma <sh...@gmail.com> wrote:
> Hi,
> Is it possible to get the process id of each task in a MapReduce job?
> When I run a mapreduce job and do a monitoring in linux using ps, i just see
> the id of the mapreduce job process but not its constituent map/reduce
> tasks.
> The use case is to monitor the resource usage of each task by using sar
> utility in linux with specific process id of task.
>
> Thanks,
> Bikash
Re: getting the process id of mapreduce tasks
Posted by bikash sharma <sh...@gmail.com>.
Thanks so much Harsh!
On Thu, Sep 29, 2011 at 12:42 AM, Harsh J <ha...@cloudera.com> wrote:
> Hello Bikash,
>
> The tasks run on the tasktracker, so that is where you'll need to look
> for the process ID -- not the JobTracker/client.
>
> Crudely speaking,
> $ ssh tasktracker01 # or whichever.
> $ jps | grep Child | cut -d " " -f 1
> # And lo, PIDs to play with.
>
> On Thu, Sep 29, 2011 at 12:15 AM, bikash sharma <sh...@gmail.com>
> wrote:
> > Hi,
> > Is it possible to get the process id of each task in a MapReduce job?
> > When I run a mapreduce job and do a monitoring in linux using ps, i just
> see
> > the id of the mapreduce job process but not its constituent map/reduce
> > tasks.
> > The use case is to monitor the resource usage of each task by using sar
> > utility in linux with specific process id of task.
> >
> > Thanks,
> > Bikash
> >
>
>
>
> --
> Harsh J
>
Re: getting the process id of mapreduce tasks
Posted by Harsh J <ha...@cloudera.com>.
Hello Bikash,
The tasks run on the tasktracker, so that is where you'll need to look
for the process ID -- not the JobTracker/client.
Crudely speaking,
$ ssh tasktracker01 # or whichever.
$ jps | grep Child | cut -d " " -f 1
# And lo, PIDs to play with.
On Thu, Sep 29, 2011 at 12:15 AM, bikash sharma <sh...@gmail.com> wrote:
> Hi,
> Is it possible to get the process id of each task in a MapReduce job?
> When I run a mapreduce job and do a monitoring in linux using ps, i just see
> the id of the mapreduce job process but not its constituent map/reduce
> tasks.
> The use case is to monitor the resource usage of each task by using sar
> utility in linux with specific process id of task.
>
> Thanks,
> Bikash
>
--
Harsh J