You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Ben Kim <be...@gmail.com> on 2013/06/05 05:02:07 UTC
is time sync required among all nodes?
Hi,
This is very basic & fundamental question.
Is time among all nodes needs to be synced?
I've never even thought of timing in hadoop cluster but recently
experienced my servers going out of sync with time. I know hbase requires
time to by synced due to its timestamp action. But I wonder any of hadoop
functionality requires time sync. Perhaps checkpoint, namenode HA, or
datanode report, etc... hmm
--
*Benjamin Kim*
*benkimkimben at gmail*
Re: is time sync required among all nodes?
Posted by Daryn Sharp <da...@yahoo-inc.com>.
Time sync is always a good idea if for no other reason than being able to collate logs when debugging. Issues may also result if a client is expecting to reliably compare timestamps, ex. checking if a file has been modified. NTP is very easy to use.
Also of note: kerberos is very sensitive to clock skew if you intend to enable security.
Daryn
On Jun 5, 2013, at 12:59 AM, Harsh J wrote:
> Skew in time between nodes is highly confusing, even if not a problem
> to the runtime. I'd strongly recommend running a time sync service for
> operational sanity.
>
> On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
>> Yes, NTPD is your best option.
>>
>> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>>> Hi,
>>> This is very basic & fundamental question.
>>>
>>> Is time among all nodes needs to be synced?
>>>
>>> I've never even thought of timing in hadoop cluster but recently
>>> experienced my servers going out of sync with time. I know hbase requires
>>> time to by synced due to its timestamp action. But I wonder any of hadoop
>>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>>> datanode report, etc... hmm
>>>
>>>
>>> --
>>>
>>> *Benjamin Kim*
>>> *benkimkimben at gmail*
>>>
>
>
>
> --
> Harsh J
Re: is time sync required among all nodes?
Posted by Daryn Sharp <da...@yahoo-inc.com>.
Time sync is always a good idea if for no other reason than being able to collate logs when debugging. Issues may also result if a client is expecting to reliably compare timestamps, ex. checking if a file has been modified. NTP is very easy to use.
Also of note: kerberos is very sensitive to clock skew if you intend to enable security.
Daryn
On Jun 5, 2013, at 12:59 AM, Harsh J wrote:
> Skew in time between nodes is highly confusing, even if not a problem
> to the runtime. I'd strongly recommend running a time sync service for
> operational sanity.
>
> On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
>> Yes, NTPD is your best option.
>>
>> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>>> Hi,
>>> This is very basic & fundamental question.
>>>
>>> Is time among all nodes needs to be synced?
>>>
>>> I've never even thought of timing in hadoop cluster but recently
>>> experienced my servers going out of sync with time. I know hbase requires
>>> time to by synced due to its timestamp action. But I wonder any of hadoop
>>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>>> datanode report, etc... hmm
>>>
>>>
>>> --
>>>
>>> *Benjamin Kim*
>>> *benkimkimben at gmail*
>>>
>
>
>
> --
> Harsh J
Re: is time sync required among all nodes?
Posted by Daryn Sharp <da...@yahoo-inc.com>.
Time sync is always a good idea if for no other reason than being able to collate logs when debugging. Issues may also result if a client is expecting to reliably compare timestamps, ex. checking if a file has been modified. NTP is very easy to use.
Also of note: kerberos is very sensitive to clock skew if you intend to enable security.
Daryn
On Jun 5, 2013, at 12:59 AM, Harsh J wrote:
> Skew in time between nodes is highly confusing, even if not a problem
> to the runtime. I'd strongly recommend running a time sync service for
> operational sanity.
>
> On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
>> Yes, NTPD is your best option.
>>
>> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>>> Hi,
>>> This is very basic & fundamental question.
>>>
>>> Is time among all nodes needs to be synced?
>>>
>>> I've never even thought of timing in hadoop cluster but recently
>>> experienced my servers going out of sync with time. I know hbase requires
>>> time to by synced due to its timestamp action. But I wonder any of hadoop
>>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>>> datanode report, etc... hmm
>>>
>>>
>>> --
>>>
>>> *Benjamin Kim*
>>> *benkimkimben at gmail*
>>>
>
>
>
> --
> Harsh J
Re: is time sync required among all nodes?
Posted by Daryn Sharp <da...@yahoo-inc.com>.
Time sync is always a good idea if for no other reason than being able to collate logs when debugging. Issues may also result if a client is expecting to reliably compare timestamps, ex. checking if a file has been modified. NTP is very easy to use.
Also of note: kerberos is very sensitive to clock skew if you intend to enable security.
Daryn
On Jun 5, 2013, at 12:59 AM, Harsh J wrote:
> Skew in time between nodes is highly confusing, even if not a problem
> to the runtime. I'd strongly recommend running a time sync service for
> operational sanity.
>
> On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
>> Yes, NTPD is your best option.
>>
>> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>>> Hi,
>>> This is very basic & fundamental question.
>>>
>>> Is time among all nodes needs to be synced?
>>>
>>> I've never even thought of timing in hadoop cluster but recently
>>> experienced my servers going out of sync with time. I know hbase requires
>>> time to by synced due to its timestamp action. But I wonder any of hadoop
>>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>>> datanode report, etc... hmm
>>>
>>>
>>> --
>>>
>>> *Benjamin Kim*
>>> *benkimkimben at gmail*
>>>
>
>
>
> --
> Harsh J
Re: is time sync required among all nodes?
Posted by Harsh J <ha...@cloudera.com>.
Skew in time between nodes is highly confusing, even if not a problem
to the runtime. I'd strongly recommend running a time sync service for
operational sanity.
On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
> Yes, NTPD is your best option.
>
> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>> Hi,
>> This is very basic & fundamental question.
>>
>> Is time among all nodes needs to be synced?
>>
>> I've never even thought of timing in hadoop cluster but recently
>> experienced my servers going out of sync with time. I know hbase requires
>> time to by synced due to its timestamp action. But I wonder any of hadoop
>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>> datanode report, etc... hmm
>>
>>
>> --
>>
>> *Benjamin Kim*
>> *benkimkimben at gmail*
>>
--
Harsh J
Re: is time sync required among all nodes?
Posted by Harsh J <ha...@cloudera.com>.
Skew in time between nodes is highly confusing, even if not a problem
to the runtime. I'd strongly recommend running a time sync service for
operational sanity.
On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
> Yes, NTPD is your best option.
>
> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>> Hi,
>> This is very basic & fundamental question.
>>
>> Is time among all nodes needs to be synced?
>>
>> I've never even thought of timing in hadoop cluster but recently
>> experienced my servers going out of sync with time. I know hbase requires
>> time to by synced due to its timestamp action. But I wonder any of hadoop
>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>> datanode report, etc... hmm
>>
>>
>> --
>>
>> *Benjamin Kim*
>> *benkimkimben at gmail*
>>
--
Harsh J
Re: is time sync required among all nodes?
Posted by Harsh J <ha...@cloudera.com>.
Skew in time between nodes is highly confusing, even if not a problem
to the runtime. I'd strongly recommend running a time sync service for
operational sanity.
On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
> Yes, NTPD is your best option.
>
> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>> Hi,
>> This is very basic & fundamental question.
>>
>> Is time among all nodes needs to be synced?
>>
>> I've never even thought of timing in hadoop cluster but recently
>> experienced my servers going out of sync with time. I know hbase requires
>> time to by synced due to its timestamp action. But I wonder any of hadoop
>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>> datanode report, etc... hmm
>>
>>
>> --
>>
>> *Benjamin Kim*
>> *benkimkimben at gmail*
>>
--
Harsh J
Re: is time sync required among all nodes?
Posted by Harsh J <ha...@cloudera.com>.
Skew in time between nodes is highly confusing, even if not a problem
to the runtime. I'd strongly recommend running a time sync service for
operational sanity.
On Wed, Jun 5, 2013 at 8:41 AM, Chris Embree <ce...@gmail.com> wrote:
> Yes, NTPD is your best option.
>
> On 6/4/13, Ben Kim <be...@gmail.com> wrote:
>> Hi,
>> This is very basic & fundamental question.
>>
>> Is time among all nodes needs to be synced?
>>
>> I've never even thought of timing in hadoop cluster but recently
>> experienced my servers going out of sync with time. I know hbase requires
>> time to by synced due to its timestamp action. But I wonder any of hadoop
>> functionality requires time sync. Perhaps checkpoint, namenode HA, or
>> datanode report, etc... hmm
>>
>>
>> --
>>
>> *Benjamin Kim*
>> *benkimkimben at gmail*
>>
--
Harsh J
Re: is time sync required among all nodes?
Posted by Chris Embree <ce...@gmail.com>.
Yes, NTPD is your best option.
On 6/4/13, Ben Kim <be...@gmail.com> wrote:
> Hi,
> This is very basic & fundamental question.
>
> Is time among all nodes needs to be synced?
>
> I've never even thought of timing in hadoop cluster but recently
> experienced my servers going out of sync with time. I know hbase requires
> time to by synced due to its timestamp action. But I wonder any of hadoop
> functionality requires time sync. Perhaps checkpoint, namenode HA, or
> datanode report, etc... hmm
>
>
> --
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>
Re: is time sync required among all nodes?
Posted by Chris Embree <ce...@gmail.com>.
Yes, NTPD is your best option.
On 6/4/13, Ben Kim <be...@gmail.com> wrote:
> Hi,
> This is very basic & fundamental question.
>
> Is time among all nodes needs to be synced?
>
> I've never even thought of timing in hadoop cluster but recently
> experienced my servers going out of sync with time. I know hbase requires
> time to by synced due to its timestamp action. But I wonder any of hadoop
> functionality requires time sync. Perhaps checkpoint, namenode HA, or
> datanode report, etc... hmm
>
>
> --
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>
Re: is time sync required among all nodes?
Posted by Chris Embree <ce...@gmail.com>.
Yes, NTPD is your best option.
On 6/4/13, Ben Kim <be...@gmail.com> wrote:
> Hi,
> This is very basic & fundamental question.
>
> Is time among all nodes needs to be synced?
>
> I've never even thought of timing in hadoop cluster but recently
> experienced my servers going out of sync with time. I know hbase requires
> time to by synced due to its timestamp action. But I wonder any of hadoop
> functionality requires time sync. Perhaps checkpoint, namenode HA, or
> datanode report, etc... hmm
>
>
> --
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>
Re: is time sync required among all nodes?
Posted by Chris Embree <ce...@gmail.com>.
Yes, NTPD is your best option.
On 6/4/13, Ben Kim <be...@gmail.com> wrote:
> Hi,
> This is very basic & fundamental question.
>
> Is time among all nodes needs to be synced?
>
> I've never even thought of timing in hadoop cluster but recently
> experienced my servers going out of sync with time. I know hbase requires
> time to by synced due to its timestamp action. But I wonder any of hadoop
> functionality requires time sync. Perhaps checkpoint, namenode HA, or
> datanode report, etc... hmm
>
>
> --
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>