Getting jobTracker startTime from the JobClient

[Pete Wyckoff <pw...@facebook.com>]

Hi,

I want to get the time the JobTracker was started (Not the time an
individual job was started).

Is there a way from the JobClient? I see you can get a ClusterStatus Object,
but it doesn't include the JobTracker.getStartTime() variable.

Is there another way to do this or should  I open a JIRA and submit a
patch??

Thanks, pete

Re: Getting jobTracker startTime from the JobClient

[Steve Loughran <st...@apache.org>]

Pete Wyckoff wrote:
> I'm looking for the actual #of seconds since (or absolute time) the job
> tracker was ready to start accepting jobs.
> 
> I'm writing a utility to (attempt :)) more robustly run hadoop jobs and one
> thing I want to detect is if the JobTracker has gone down while a job that
> failed was running.
> 
> Since the script will also sleep and poll the job tracker till it comes back
> up (or connectivity is restored), when I finally contact the job tracker, I
> want to know how long its been running.
> 
> So, knowing whether the JT is initializing or not is useful, but also the
> actual time it was ready.
> 
> Thanks, pete

Seems to me something that should be remotely available and visible via 
JMX...always good to use. Absolute time info, if sent as a time_t long 
should be in UTC to avoid confusion when talking to servers in different 
time zones...

-- 
Steve Loughran                  http://www.1060.org/blogxter/publish/5
Author: Ant in Action           http://antbook.org/

Re: Getting jobTracker startTime from the JobClient

[Pete Wyckoff <pw...@facebook.com>]

I'm looking for the actual #of seconds since (or absolute time) the job
tracker was ready to start accepting jobs.

I'm writing a utility to (attempt :)) more robustly run hadoop jobs and one
thing I want to detect is if the JobTracker has gone down while a job that
failed was running.

Since the script will also sleep and poll the job tracker till it comes back
up (or connectivity is restored), when I finally contact the job tracker, I
want to know how long its been running.

So, knowing whether the JT is initializing or not is useful, but also the
actual time it was ready.

Thanks, pete


On 4/30/08 5:07 AM, "Vinod KV" <vi...@yahoo-inc.com> wrote:

> Just wanted to know whether you are talking about the starting time of
> the jobtracker, or you are interested in knowing when jobtracker is
> *really* up and ready to accept jobs. Hadoop-3289 addresses the later.
> 
> Thanks
> -vinod
> 
> 
> 
> Amar Kamat wrote:
>> It can be made a part of ClusterStatus.
>> Amar
>> Devaraj Das wrote:
>>> No, currently, there is no way to get that from the JobClient. Yes,
>>> please
>>> submit a patch.
>>> 
>>>  
>>>> -----Original Message-----
>>>> From: Pete Wyckoff [mailto:pwyckoff@facebook.com] Sent: Wednesday,
>>>> April 30, 2008 8:21 AM
>>>> To: core-user@hadoop.apache.org
>>>> Subject: Getting jobTracker startTime from the JobClient
>>>> 
>>>> 
>>>> Hi,
>>>> 
>>>> I want to get the time the JobTracker was started (Not the time an
>>>> individual job was started).
>>>> 
>>>> Is there a way from the JobClient? I see you can get a ClusterStatus
>>>> Object, but it doesn't include the JobTracker.getStartTime() variable.
>>>> 
>>>> Is there another way to do this or should  I open a JIRA and submit
>>>> a patch??
>>>> 
>>>> Thanks, pete
>>>> 
>>>> 
>>>>     
>>> 
>>>   
>> 
>> 
> 

Re: Getting jobTracker startTime from the JobClient

[Vinod KV <vi...@yahoo-inc.com>]

Just wanted to know whether you are talking about the starting time of 
the jobtracker, or you are interested in knowing when jobtracker is 
*really* up and ready to accept jobs. Hadoop-3289 addresses the later.

Thanks
-vinod



Amar Kamat wrote:
> It can be made a part of ClusterStatus.
> Amar
> Devaraj Das wrote:
>> No, currently, there is no way to get that from the JobClient. Yes, 
>> please
>> submit a patch.
>>
>>  
>>> -----Original Message-----
>>> From: Pete Wyckoff [mailto:pwyckoff@facebook.com] Sent: Wednesday, 
>>> April 30, 2008 8:21 AM
>>> To: core-user@hadoop.apache.org
>>> Subject: Getting jobTracker startTime from the JobClient
>>>
>>>
>>> Hi,
>>>
>>> I want to get the time the JobTracker was started (Not the time an 
>>> individual job was started).
>>>
>>> Is there a way from the JobClient? I see you can get a ClusterStatus 
>>> Object, but it doesn't include the JobTracker.getStartTime() variable.
>>>
>>> Is there another way to do this or should  I open a JIRA and submit 
>>> a patch??
>>>
>>> Thanks, pete
>>>
>>>
>>>     
>>
>>   
>
>

Re: Getting jobTracker startTime from the JobClient

[Amar Kamat <am...@yahoo-inc.com>]

It can be made a part of ClusterStatus.
Amar
Devaraj Das wrote:
> No, currently, there is no way to get that from the JobClient. Yes, please
> submit a patch.
>
>   
>> -----Original Message-----
>> From: Pete Wyckoff [mailto:pwyckoff@facebook.com] 
>> Sent: Wednesday, April 30, 2008 8:21 AM
>> To: core-user@hadoop.apache.org
>> Subject: Getting jobTracker startTime from the JobClient
>>
>>
>> Hi,
>>
>> I want to get the time the JobTracker was started (Not the 
>> time an individual job was started).
>>
>> Is there a way from the JobClient? I see you can get a 
>> ClusterStatus Object, but it doesn't include the 
>> JobTracker.getStartTime() variable.
>>
>> Is there another way to do this or should  I open a JIRA and 
>> submit a patch??
>>
>> Thanks, pete
>>
>>
>>
>>
>>     
>
>   

RE: Getting jobTracker startTime from the JobClient

[Devaraj Das <dd...@yahoo-inc.com>]

No, currently, there is no way to get that from the JobClient. Yes, please
submit a patch.

> -----Original Message-----
> From: Pete Wyckoff [mailto:pwyckoff@facebook.com] 
> Sent: Wednesday, April 30, 2008 8:21 AM
> To: core-user@hadoop.apache.org
> Subject: Getting jobTracker startTime from the JobClient
> 
> 
> Hi,
> 
> I want to get the time the JobTracker was started (Not the 
> time an individual job was started).
> 
> Is there a way from the JobClient? I see you can get a 
> ClusterStatus Object, but it doesn't include the 
> JobTracker.getStartTime() variable.
> 
> Is there another way to do this or should  I open a JIRA and 
> submit a patch??
> 
> Thanks, pete
> 
> 
> 
>