You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by prasenjit mukherjee <pr...@gmail.com> on 2009/09/15 02:48:34 UTC
hadoop/pig debugging
It is probably more appropriate for hadoop forum. I am looking for a
web-based debugging interface for hadoop on Amazon/EC2. Is there a way
I can see ( in Amazon/EC2 ) what is the current status of my mapreduce
jobs for a pig-run.
Re: hadoop/pig debugging
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
There are two tickets, even.
There is 948, which helps figure out which of the many MR jobs that
might be running on your cluster actually belong to the Pig job you
are running (and which pig job).
Then there is also https://issues.apache.org/jira/browse/PIG-908 which
is about relating which stage of your pig script is getting processed
with a given MR job (948 will just tell you that they are related, not
what operators are being processed).
As far as what access AWS gives you to monitor your hadoop jobs when
running through their hadoop services (as opposed to using EC2 to
start up your own Hadoop cluster in the cloud).. I have no idea,
probably a good question for the AWS forums.
If you are running your own cluster, yes there is an interface. I
always forget which port is which for the web interfaces, so I have it
bookmarked. For people who don't have it bookmarked this helps:
http://www.cloudera.com/blog/2009/08/14/hadoop-default-ports-quick-reference/
-D
On Mon, Sep 14, 2009 at 9:49 PM, prasenjit mukherjee
<pr...@gmail.com> wrote:
> wasn't there a web-interface for jobtracker for hadoop ? how do I
> access that web-interface ?
>
> On Mon, Sep 14, 2009 at 8:58 PM, Ashutosh Chauhan
> <as...@gmail.com> wrote:
>> I am not aware of any such interface. There might be one. But, currently
>> there is no easy way to figure out which map-reduce jobs got generated
>> because of a pig script. There is an open jira on this:
>> https://issues.apache.org/jira/browse/PIG-948
>>
>> Ashutosh
>> On Mon, Sep 14, 2009 at 20:48, prasenjit mukherjee <pr...@gmail.com>wrote:
>>
>>> It is probably more appropriate for hadoop forum. I am looking for a
>>> web-based debugging interface for hadoop on Amazon/EC2. Is there a way
>>> I can see ( in Amazon/EC2 ) what is the current status of my mapreduce
>>> jobs for a pig-run.
>>>
>>
>
Re: hadoop/pig debugging
Posted by prasenjit mukherjee <pr...@gmail.com>.
wasn't there a web-interface for jobtracker for hadoop ? how do I
access that web-interface ?
On Mon, Sep 14, 2009 at 8:58 PM, Ashutosh Chauhan
<as...@gmail.com> wrote:
> I am not aware of any such interface. There might be one. But, currently
> there is no easy way to figure out which map-reduce jobs got generated
> because of a pig script. There is an open jira on this:
> https://issues.apache.org/jira/browse/PIG-948
>
> Ashutosh
> On Mon, Sep 14, 2009 at 20:48, prasenjit mukherjee <pr...@gmail.com>wrote:
>
>> It is probably more appropriate for hadoop forum. I am looking for a
>> web-based debugging interface for hadoop on Amazon/EC2. Is there a way
>> I can see ( in Amazon/EC2 ) what is the current status of my mapreduce
>> jobs for a pig-run.
>>
>
Re: hadoop/pig debugging
Posted by Ashutosh Chauhan <as...@gmail.com>.
I am not aware of any such interface. There might be one. But, currently
there is no easy way to figure out which map-reduce jobs got generated
because of a pig script. There is an open jira on this:
https://issues.apache.org/jira/browse/PIG-948
Ashutosh
On Mon, Sep 14, 2009 at 20:48, prasenjit mukherjee <pr...@gmail.com>wrote:
> It is probably more appropriate for hadoop forum. I am looking for a
> web-based debugging interface for hadoop on Amazon/EC2. Is there a way
> I can see ( in Amazon/EC2 ) what is the current status of my mapreduce
> jobs for a pig-run.
>