How do I programmatically get total job execution time?

["W.P. McNeill" <bi...@gmail.com>]

After my Hadoop job has successfully completed I'd like to log the total
amount of time it took. This is the "Finished in" statistic in the web UI.
How do I get this number programmatically? Is there some way I can query
the Job object? I didn't see anything in the API documentation.

Re: How do I programmatically get total job execution time?

[Praveen Sripati <pr...@gmail.com>]

Hi,

Ran a job using new MR API in stand alone mode and 0.21. Both,
Job#getFinishTime and Job#getStartTime are returning 0. Not sure, if this
is a bug.

Thanks,
Praveen

On Sat, Dec 3, 2011 at 6:14 AM, Raj V <ra...@yahoo.com> wrote:

> As Harsh said, I don't think there is a simple way to way to find when the
> job ended, especially after the job is completed.
>
> But cant you just wait for your job to complete and log the time when the
> job completed?
>
> Raj
>
>
>
> >________________________________
> > From: Harsh J <ha...@cloudera.com>
> >To: common-user@hadoop.apache.org
> >Sent: Friday, December 2, 2011 12:53 PM
> >Subject: Re: How do I programmatically get total job execution time?
> >
> >I remember hitting this once in 0.20 - seems like an API limitation. The
> resolution we took back then was to get a list of all tasks, and get the
> end time with the last ended task's completion time (sort and pick). There
> may be other ways though - others can comment on that perhaps (metrics?
> job-history?)
> >
> >On 02-Dec-2011, at 11:27 PM, W.P. McNeill wrote:
> >
> >> After my Hadoop job has successfully completed I'd like to log the total
> >> amount of time it took. This is the "Finished in" statistic in the web
> UI.
> >> How do I get this number programmatically? Is there some way I can query
> >> the Job object? I didn't see anything in the API documentation.
> >
> >02-Dec-2011, at 11:27 PM, W.P. McNeill wrote:
> >
> >> After my Hadoop job has successfully completed I'd like to log the total
> >> amount of time it took. This is the "Finished in" statistic in the web
> UI.
> >> How do I get this number programmatically? Is there some way I can query
> >> the Job object? I didn't see anything in the API documentation.
> >
> >
> >
> >
>

Re: How do I programmatically get total job execution time?

[Raj V <ra...@yahoo.com>]

As Harsh said, I don't think there is a simple way to way to find when the job ended, especially after the job is completed. 

But cant you just wait for your job to complete and log the time when the job completed? 

Raj



>________________________________
> From: Harsh J <ha...@cloudera.com>
>To: common-user@hadoop.apache.org 
>Sent: Friday, December 2, 2011 12:53 PM
>Subject: Re: How do I programmatically get total job execution time?
> 
>I remember hitting this once in 0.20 - seems like an API limitation. The resolution we took back then was to get a list of all tasks, and get the end time with the last ended task's completion time (sort and pick). There may be other ways though - others can comment on that perhaps (metrics? job-history?)
>
>On 02-Dec-2011, at 11:27 PM, W.P. McNeill wrote:
>
>> After my Hadoop job has successfully completed I'd like to log the total
>> amount of time it took. This is the "Finished in" statistic in the web UI.
>> How do I get this number programmatically? Is there some way I can query
>> the Job object? I didn't see anything in the API documentation.
>
>02-Dec-2011, at 11:27 PM, W.P. McNeill wrote:
>
>> After my Hadoop job has successfully completed I'd like to log the total
>> amount of time it took. This is the "Finished in" statistic in the web UI.
>> How do I get this number programmatically? Is there some way I can query
>> the Job object? I didn't see anything in the API documentation.
>
>
>
>

Re: How do I programmatically get total job execution time?

[Harsh J <ha...@cloudera.com>]

I remember hitting this once in 0.20 - seems like an API limitation. The resolution we took back then was to get a list of all tasks, and get the end time with the last ended task's completion time (sort and pick). There may be other ways though - others can comment on that perhaps (metrics? job-history?)

On 02-Dec-2011, at 11:27 PM, W.P. McNeill wrote:

> After my Hadoop job has successfully completed I'd like to log the total
> amount of time it took. This is the "Finished in" statistic in the web UI.
> How do I get this number programmatically? Is there some way I can query
> the Job object? I didn't see anything in the API documentation.

Re: How do I programmatically get total job execution time?

[Tom Melendez <to...@supertom.com>]

On Fri, Dec 2, 2011 at 9:57 AM, W.P. McNeill <bi...@gmail.com> wrote:
> After my Hadoop job has successfully completed I'd like to log the total
> amount of time it took. This is the "Finished in" statistic in the web UI.
> How do I get this number programmatically? Is there some way I can query
> the Job object? I didn't see anything in the API documentation.

This probably *doesn't* help you, but if you're using (or planning on
using) oozie, it has a restful API that can give you this information.

Thanks,

Tom