You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Kyle B <kb...@gmail.com> on 2013/03/06 19:43:30 UTC
Hadoop Jobtracker API
Hello,
I was wondering if the Hadoop job tracker had an API, such as a web service
or xml feed? I'm trying to track Hadoop jobs as they progress. Right now,
I'm parsing the HTML of the "Running Jobs" section at
http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
there is a better way. Is there a simple web service for the data on
jobtracker.php & jobdetails.jsp?
Has anyone run into this problem of trying to track Hadoop progress from a
remote machine programatically?
Any help is appreciated,
-Kyle
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
There is only JobTracker servlet which u can use as web service but you
need to parse HTML response or you can build small java ws using code from
stackoverflow.
Regards
Dino
On Mar 6, 2013 10:06 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hi Dino,
>
> Thanks for the response. I've seen this before, but was hoping to avoid
> getting locked into the Java road. Do you happen to know if there is an
> open API for the job tacker included with Hadoop? Something I could call
> from a variety of languages, like a web service?
>
> -Kyle
>
> On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
>
>> Hi Kyle,
>>
>> Maybe this can help you
>>
>>
>> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>>
>> Regards,
>> Dino
>> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I was wondering if the Hadoop job tracker had an API, such as a web
>>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>>> if there is a better way. Is there a simple web service for the data on
>>> jobtracker.php & jobdetails.jsp?
>>>
>>> Has anyone run into this problem of trying to track Hadoop progress from
>>> a remote machine programatically?
>>>
>>> Any help is appreciated,
>>>
>>> -Kyle
>>>
>>
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
There is only JobTracker servlet which u can use as web service but you
need to parse HTML response or you can build small java ws using code from
stackoverflow.
Regards
Dino
On Mar 6, 2013 10:06 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hi Dino,
>
> Thanks for the response. I've seen this before, but was hoping to avoid
> getting locked into the Java road. Do you happen to know if there is an
> open API for the job tacker included with Hadoop? Something I could call
> from a variety of languages, like a web service?
>
> -Kyle
>
> On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
>
>> Hi Kyle,
>>
>> Maybe this can help you
>>
>>
>> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>>
>> Regards,
>> Dino
>> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I was wondering if the Hadoop job tracker had an API, such as a web
>>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>>> if there is a better way. Is there a simple web service for the data on
>>> jobtracker.php & jobdetails.jsp?
>>>
>>> Has anyone run into this problem of trying to track Hadoop progress from
>>> a remote machine programatically?
>>>
>>> Any help is appreciated,
>>>
>>> -Kyle
>>>
>>
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
There is only JobTracker servlet which u can use as web service but you
need to parse HTML response or you can build small java ws using code from
stackoverflow.
Regards
Dino
On Mar 6, 2013 10:06 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hi Dino,
>
> Thanks for the response. I've seen this before, but was hoping to avoid
> getting locked into the Java road. Do you happen to know if there is an
> open API for the job tacker included with Hadoop? Something I could call
> from a variety of languages, like a web service?
>
> -Kyle
>
> On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
>
>> Hi Kyle,
>>
>> Maybe this can help you
>>
>>
>> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>>
>> Regards,
>> Dino
>> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I was wondering if the Hadoop job tracker had an API, such as a web
>>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>>> if there is a better way. Is there a simple web service for the data on
>>> jobtracker.php & jobdetails.jsp?
>>>
>>> Has anyone run into this problem of trying to track Hadoop progress from
>>> a remote machine programatically?
>>>
>>> Any help is appreciated,
>>>
>>> -Kyle
>>>
>>
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
There is only JobTracker servlet which u can use as web service but you
need to parse HTML response or you can build small java ws using code from
stackoverflow.
Regards
Dino
On Mar 6, 2013 10:06 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hi Dino,
>
> Thanks for the response. I've seen this before, but was hoping to avoid
> getting locked into the Java road. Do you happen to know if there is an
> open API for the job tacker included with Hadoop? Something I could call
> from a variety of languages, like a web service?
>
> -Kyle
>
> On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
>
>> Hi Kyle,
>>
>> Maybe this can help you
>>
>>
>> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>>
>> Regards,
>> Dino
>> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I was wondering if the Hadoop job tracker had an API, such as a web
>>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>>> if there is a better way. Is there a simple web service for the data on
>>> jobtracker.php & jobdetails.jsp?
>>>
>>> Has anyone run into this problem of trying to track Hadoop progress from
>>> a remote machine programatically?
>>>
>>> Any help is appreciated,
>>>
>>> -Kyle
>>>
>>
>
Re: Hadoop Jobtracker API
Posted by Kyle B <kb...@gmail.com>.
Hi Dino,
Thanks for the response. I've seen this before, but was hoping to avoid
getting locked into the Java road. Do you happen to know if there is an
open API for the job tacker included with Hadoop? Something I could call
from a variety of languages, like a web service?
-Kyle
On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
> Hi Kyle,
>
> Maybe this can help you
>
>
> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>
> Regards,
> Dino
> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>
>> Hello,
>>
>> I was wondering if the Hadoop job tracker had an API, such as a web
>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>> if there is a better way. Is there a simple web service for the data on
>> jobtracker.php & jobdetails.jsp?
>>
>> Has anyone run into this problem of trying to track Hadoop progress from
>> a remote machine programatically?
>>
>> Any help is appreciated,
>>
>> -Kyle
>>
>
Re: Hadoop Jobtracker API
Posted by Kyle B <kb...@gmail.com>.
Hi Dino,
Thanks for the response. I've seen this before, but was hoping to avoid
getting locked into the Java road. Do you happen to know if there is an
open API for the job tacker included with Hadoop? Something I could call
from a variety of languages, like a web service?
-Kyle
On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
> Hi Kyle,
>
> Maybe this can help you
>
>
> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>
> Regards,
> Dino
> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>
>> Hello,
>>
>> I was wondering if the Hadoop job tracker had an API, such as a web
>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>> if there is a better way. Is there a simple web service for the data on
>> jobtracker.php & jobdetails.jsp?
>>
>> Has anyone run into this problem of trying to track Hadoop progress from
>> a remote machine programatically?
>>
>> Any help is appreciated,
>>
>> -Kyle
>>
>
Re: Hadoop Jobtracker API
Posted by Kyle B <kb...@gmail.com>.
Hi Dino,
Thanks for the response. I've seen this before, but was hoping to avoid
getting locked into the Java road. Do you happen to know if there is an
open API for the job tacker included with Hadoop? Something I could call
from a variety of languages, like a web service?
-Kyle
On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
> Hi Kyle,
>
> Maybe this can help you
>
>
> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>
> Regards,
> Dino
> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>
>> Hello,
>>
>> I was wondering if the Hadoop job tracker had an API, such as a web
>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>> if there is a better way. Is there a simple web service for the data on
>> jobtracker.php & jobdetails.jsp?
>>
>> Has anyone run into this problem of trying to track Hadoop progress from
>> a remote machine programatically?
>>
>> Any help is appreciated,
>>
>> -Kyle
>>
>
Re: Hadoop Jobtracker API
Posted by Kyle B <kb...@gmail.com>.
Hi Dino,
Thanks for the response. I've seen this before, but was hoping to avoid
getting locked into the Java road. Do you happen to know if there is an
open API for the job tacker included with Hadoop? Something I could call
from a variety of languages, like a web service?
-Kyle
On Wed, Mar 6, 2013 at 12:42 PM, Dino Kečo <di...@gmail.com> wrote:
> Hi Kyle,
>
> Maybe this can help you
>
>
> http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
>
> Regards,
> Dino
> On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
>
>> Hello,
>>
>> I was wondering if the Hadoop job tracker had an API, such as a web
>> service or xml feed? I'm trying to track Hadoop jobs as they progress.
>> Right now, I'm parsing the HTML of the "Running Jobs" section at
>> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired
>> if there is a better way. Is there a simple web service for the data on
>> jobtracker.php & jobdetails.jsp?
>>
>> Has anyone run into this problem of trying to track Hadoop progress from
>> a remote machine programatically?
>>
>> Any help is appreciated,
>>
>> -Kyle
>>
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
Maybe this can help you
http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
Regards,
Dino
On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web
> service or xml feed? I'm trying to track Hadoop jobs as they progress.
> Right now, I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
Maybe this can help you
http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
Regards,
Dino
On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web
> service or xml feed? I'm trying to track Hadoop jobs as they progress.
> Right now, I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
>
Re: Hadoop Jobtracker API
Posted by Harsh J <ha...@cloudera.com>.
The Java API of JobClient class lets you query all jobs and provides
some task-level info as a public API.
In YARN (2.x onwards), the MRv2's AM publishes a REST API that lets
you query it (the RM lets you get a list of such AMs as well, as a
first step). This sounds more like what you need.
A REST API similar to what was added in YARN for MRv2 was also added
to the 1.x's JobTracker recently via
https://issues.apache.org/jira/browse/MAPREDUCE-4837, appearing in
1.2.0 release onwards.
On Thu, Mar 7, 2013 at 12:13 AM, Kyle B <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web service
> or xml feed? I'm trying to track Hadoop jobs as they progress. Right now,
> I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
--
Harsh J
Re: Hadoop Jobtracker API
Posted by Harsh J <ha...@cloudera.com>.
The Java API of JobClient class lets you query all jobs and provides
some task-level info as a public API.
In YARN (2.x onwards), the MRv2's AM publishes a REST API that lets
you query it (the RM lets you get a list of such AMs as well, as a
first step). This sounds more like what you need.
A REST API similar to what was added in YARN for MRv2 was also added
to the 1.x's JobTracker recently via
https://issues.apache.org/jira/browse/MAPREDUCE-4837, appearing in
1.2.0 release onwards.
On Thu, Mar 7, 2013 at 12:13 AM, Kyle B <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web service
> or xml feed? I'm trying to track Hadoop jobs as they progress. Right now,
> I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
--
Harsh J
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
Maybe this can help you
http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
Regards,
Dino
On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web
> service or xml feed? I'm trying to track Hadoop jobs as they progress.
> Right now, I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
>
Re: Hadoop Jobtracker API
Posted by Dino Kečo <di...@gmail.com>.
Hi Kyle,
Maybe this can help you
http://stackoverflow.com/questions/2616524/tracking-hadoop-job-status-via-web-interface-exposing-hadoop-to-internal-clien/4156387#4156387
Regards,
Dino
On Mar 6, 2013 7:43 PM, "Kyle B" <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web
> service or xml feed? I'm trying to track Hadoop jobs as they progress.
> Right now, I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
>
Re: Hadoop Jobtracker API
Posted by Harsh J <ha...@cloudera.com>.
The Java API of JobClient class lets you query all jobs and provides
some task-level info as a public API.
In YARN (2.x onwards), the MRv2's AM publishes a REST API that lets
you query it (the RM lets you get a list of such AMs as well, as a
first step). This sounds more like what you need.
A REST API similar to what was added in YARN for MRv2 was also added
to the 1.x's JobTracker recently via
https://issues.apache.org/jira/browse/MAPREDUCE-4837, appearing in
1.2.0 release onwards.
On Thu, Mar 7, 2013 at 12:13 AM, Kyle B <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web service
> or xml feed? I'm trying to track Hadoop jobs as they progress. Right now,
> I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
--
Harsh J
Re: Hadoop Jobtracker API
Posted by Harsh J <ha...@cloudera.com>.
The Java API of JobClient class lets you query all jobs and provides
some task-level info as a public API.
In YARN (2.x onwards), the MRv2's AM publishes a REST API that lets
you query it (the RM lets you get a list of such AMs as well, as a
first step). This sounds more like what you need.
A REST API similar to what was added in YARN for MRv2 was also added
to the 1.x's JobTracker recently via
https://issues.apache.org/jira/browse/MAPREDUCE-4837, appearing in
1.2.0 release onwards.
On Thu, Mar 7, 2013 at 12:13 AM, Kyle B <kb...@gmail.com> wrote:
> Hello,
>
> I was wondering if the Hadoop job tracker had an API, such as a web service
> or xml feed? I'm trying to track Hadoop jobs as they progress. Right now,
> I'm parsing the HTML of the "Running Jobs" section at
> http://hadoop:50030/jobtracker.jsp, but this is definitely not desired if
> there is a better way. Is there a simple web service for the data on
> jobtracker.php & jobdetails.jsp?
>
> Has anyone run into this problem of trying to track Hadoop progress from a
> remote machine programatically?
>
> Any help is appreciated,
>
> -Kyle
--
Harsh J