You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Something Something <ma...@gmail.com> on 2013/06/21 02:21:01 UTC
How many machines did my MR job used?
Hello,
I am running a Pig script which internally starts several jobs. For one of
the jobs that uses maximum no. of mappers & reducers, I need to find out
how many machines it's running on & which machines are those.
I looked around the JobTracker UI, but couldn't find this information. Is
it there somewhere? Is it in any of the logs?
Would appreciate any help on this. Thanks.
Re: How many machines did my MR job used?
Posted by Cheolsoo Park <pi...@gmail.com>.
When Pig launches each job, it prints a JobTracker URL on console. For
example,
More information at:
http://<host>:<port>/jobdetails.jsp?jobid=job_X<http://10.11.46.215:9100/jobdetails.jsp?jobid=job_201306032155_59754>
Now you can find the jobtracker log and grep the job id. Then, you will see
all the task attempts associated with that job as well as machines where
the tasks are running. For example,
'attempt_X_m_000888_0' to tip task_X_m_000888, for tracker
'tracker_<host>:localhost/127.0.0.1: <http://127.0.0.1:33768><port>'
Alternatively, on the JobTracker UI, you can follow the link and will find
the job page. Click on the number of running tasks will take you to tasks
page. Lastly, click on each task link will show you which machine it is
running on.
On Thu, Jun 20, 2013 at 5:21 PM, Something Something <
mailinglists19@gmail.com> wrote:
> Hello,
>
> I am running a Pig script which internally starts several jobs. For one of
> the jobs that uses maximum no. of mappers & reducers, I need to find out
> how many machines it's running on & which machines are those.
>
> I looked around the JobTracker UI, but couldn't find this information. Is
> it there somewhere? Is it in any of the logs?
>
> Would appreciate any help on this. Thanks.
>
Re: How many machines did my MR job used?
Posted by Arun C Murthy <ac...@hortonworks.com>.
I believe Apache Ambari has the information you want via REST web-service…
http://incubator.apache.org/ambari/
Arun
On Jun 20, 2013, at 5:21 PM, Something Something <ma...@gmail.com> wrote:
> Hello,
>
> I am running a Pig script which internally starts several jobs. For one of the jobs that uses maximum no. of mappers & reducers, I need to find out how many machines it's running on & which machines are those.
>
> I looked around the JobTracker UI, but couldn't find this information. Is it there somewhere? Is it in any of the logs?
>
> Would appreciate any help on this. Thanks.
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: How many machines did my MR job used?
Posted by Arun C Murthy <ac...@hortonworks.com>.
I believe Apache Ambari has the information you want via REST web-service…
http://incubator.apache.org/ambari/
Arun
On Jun 20, 2013, at 5:21 PM, Something Something <ma...@gmail.com> wrote:
> Hello,
>
> I am running a Pig script which internally starts several jobs. For one of the jobs that uses maximum no. of mappers & reducers, I need to find out how many machines it's running on & which machines are those.
>
> I looked around the JobTracker UI, but couldn't find this information. Is it there somewhere? Is it in any of the logs?
>
> Would appreciate any help on this. Thanks.
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/