You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Kayla Jay <ka...@yahoo.com> on 2008/06/17 19:19:48 UTC
getting hadoop job status/progress outside of hadoop
Hi
Is there a way to grab a hadoop job's status/progress outside of the job and outside of hadoop?
I.e if I have another application running and this application needs to know that a job has ended or the status percentage while the job is running, how can an external app like this get status from the hadoop job or cluster that the job is done and the progress while it's running?
Is there a hook into the status via HTTP or any other interface? How can external apps get progress of the job running and notification when it's done running? I was thinking there might be a hook in since it reports it via the JobTracker.
Thanks.
Re: getting hadoop job status/progress outside of hadoop
Posted by Meng Mao <me...@gmail.com>.
What if I'm not interested in which job is running but simply whether the
current job is not stalled or failed?
Is there a way I can avoid specifying a job by the job ID?
I apologize if there's some commandline documentation I'm missing,
but the commands change a bit from point version to version.
On Tue, Jun 17, 2008 at 1:41 PM, Miles Osborne <mi...@inf.ed.ac.uk> wrote:
> To get this from some other application rather than Hadoop, you just need
> to run this within a shell (I do this kind of thing within perl)
>
> Miles
>
> 2008/6/17 Miles Osborne <mi...@inf.ed.ac.uk>:
>
> > try this:
> >
> > hadoop job -Dmapred.job.tracker=hermitage:9001 -status
> > job_200806160820_0430
> >
> > (and replace my job id with the one you want to track):
> > >
> > hadoop job -Dmapred.job.tracker=hermitage:9001 -status
> > job_200806160820_0430
> >
> > Job: job_200806160820_0430
> > file: /data/tmp/hadoop/mapred/system/job_200806160820_0430/job.xml
> > tracking URL:
> > http://hermitage:50030/jobdetails.jsp?jobid=job_200806160820_0430
> > map() completion: 1.0
> > reduce() completion: 0.20370372
> > >
> >
> > Miles
> >
> > 2008/6/17 Kayla Jay <ka...@yahoo.com>:
> >
> >
> >>
> >> Hi
> >>
> >> Is there a way to grab a hadoop job's status/progress outside of the job
> >> and outside of hadoop?
> >> I.e if I have another application running and this application needs to
> >> know that a job has ended or the status percentage while the job is
> running,
> >> how can an external app like this get status from the hadoop job or
> cluster
> >> that the job is done and the progress while it's running?
> >>
> >> Is there a hook into the status via HTTP or any other interface? How
> can
> >> external apps get progress of the job running and notification when it's
> >> done running? I was thinking there might be a hook in since it reports
> it
> >> via the JobTracker.
> >>
> >> Thanks.
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> > --
> > The University of Edinburgh is a charitable body, registered in Scotland,
> > with registration number SC005336.
>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336.
>
--
hustlin, hustlin, everyday I'm hustlin
Re: getting hadoop job status/progress outside of hadoop
Posted by Miles Osborne <mi...@inf.ed.ac.uk>.
To get this from some other application rather than Hadoop, you just need
to run this within a shell (I do this kind of thing within perl)
Miles
2008/6/17 Miles Osborne <mi...@inf.ed.ac.uk>:
> try this:
>
> hadoop job -Dmapred.job.tracker=hermitage:9001 -status
> job_200806160820_0430
>
> (and replace my job id with the one you want to track):
> >
> hadoop job -Dmapred.job.tracker=hermitage:9001 -status
> job_200806160820_0430
>
> Job: job_200806160820_0430
> file: /data/tmp/hadoop/mapred/system/job_200806160820_0430/job.xml
> tracking URL:
> http://hermitage:50030/jobdetails.jsp?jobid=job_200806160820_0430
> map() completion: 1.0
> reduce() completion: 0.20370372
> >
>
> Miles
>
> 2008/6/17 Kayla Jay <ka...@yahoo.com>:
>
>
>>
>> Hi
>>
>> Is there a way to grab a hadoop job's status/progress outside of the job
>> and outside of hadoop?
>> I.e if I have another application running and this application needs to
>> know that a job has ended or the status percentage while the job is running,
>> how can an external app like this get status from the hadoop job or cluster
>> that the job is done and the progress while it's running?
>>
>> Is there a hook into the status via HTTP or any other interface? How can
>> external apps get progress of the job running and notification when it's
>> done running? I was thinking there might be a hook in since it reports it
>> via the JobTracker.
>>
>> Thanks.
>>
>>
>>
>>
>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336.
--
The University of Edinburgh is a charitable body, registered in Scotland,
with registration number SC005336.
Re: getting hadoop job status/progress outside of hadoop
Posted by Miles Osborne <mi...@inf.ed.ac.uk>.
try this:
hadoop job -Dmapred.job.tracker=hermitage:9001 -status
job_200806160820_0430
(and replace my job id with the one you want to track):
>
hadoop job -Dmapred.job.tracker=hermitage:9001 -status
job_200806160820_0430
Job: job_200806160820_0430
file: /data/tmp/hadoop/mapred/system/job_200806160820_0430/job.xml
tracking URL:
http://hermitage:50030/jobdetails.jsp?jobid=job_200806160820_0430
map() completion: 1.0
reduce() completion: 0.20370372
>
Miles
2008/6/17 Kayla Jay <ka...@yahoo.com>:
>
>
> Hi
>
> Is there a way to grab a hadoop job's status/progress outside of the job
> and outside of hadoop?
> I.e if I have another application running and this application needs to
> know that a job has ended or the status percentage while the job is running,
> how can an external app like this get status from the hadoop job or cluster
> that the job is done and the progress while it's running?
>
> Is there a hook into the status via HTTP or any other interface? How can
> external apps get progress of the job running and notification when it's
> done running? I was thinking there might be a hook in since it reports it
> via the JobTracker.
>
> Thanks.
>
>
>
>
--
The University of Edinburgh is a charitable body, registered in Scotland,
with registration number SC005336.