You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Gokulakannan M <go...@huawei.com> on 2010/03/26 15:31:33 UTC
hadoop conf for dynamically changing ips
Hi,
I have a LAN in which the IPs of the machines will be changed
dynamically by the DHCP sever.
So for namenode, jobtracker, master and slave configurations we
could not give the IP.
can the machine names be given for those configurations???? will
that work??
Thanks,
Gokul
Re: hadoop conf for dynamically changing ips
Posted by Allen Wittenauer <aw...@linkedin.com>.
On 3/26/10 7:55 AM, "Andrew Klochkov" <di...@gmail.com> wrote:
> Guys, doesn't dyndns help here?
Nope. Hadoop uses the IP addresses internally. :(
On 3/26/10 9:39 AM, "Gokulakannan M" <go...@huawei.com> wrote:
>>
>> It's what I do. You just have to make sure that if the IPAddrs change,
>>
>> everything gets restarted.
>
>
> Thanks Steve for the reply. "everything must be restarted" means the
> hadoop cluster or all the systems.
The Hadoop software.
Re: hadoop conf for dynamically changing ips
Posted by Andrew Klochkov <di...@gmail.com>.
Guys, doesn't dyndns help here?
On Fri, Mar 26, 2010 at 5:43 PM, Steve Loughran <st...@apache.org> wrote:
> On 26/03/2010 14:31, Gokulakannan M wrote:
>
>>
>>
>> Hi,
>>
>>
>>
>> I have a LAN in which the IPs of the machines will be changed
>> dynamically by the DHCP sever.
>>
>>
>>
>> So for namenode, jobtracker, master and slave configurations
>> we
>> could not give the IP.
>>
>> can the machine names be given for those configurations????
>> will
>> that work??
>>
>>
> It's what I do. You just have to make sure that if the IPAddrs change,
> everything gets restarted.
>
>
--
Andrew Klochkov
Re: hadoop conf for dynamically changing ips
Posted by Nick Dimiduk <nd...@gmail.com>.
I have set up small, virtual clusters using avahi with success. I'm not sure
I've ever run them though an IP change though. I also don't know how much
overhead the protocol would impose on a "real" cluster. Still, it might be
worth looking into.
Cheers,
-Nick
On Fri, Mar 26, 2010 at 10:16 AM, Steve Loughran <st...@apache.org> wrote:
> On 26/03/2010 16:39, Gokulakannan M wrote:
>
>>
>>> It's what I do. You just have to make sure that if the IPAddrs change,
>>>
>>> everything gets restarted.
>>>
>>
>>
>> Thanks Steve for the reply. "everything must be restarted" means
>> the
>> hadoop cluster or all the systems.
>>
>
> just the JVMs. All JVMs cache mapped IP addresses forever, very annoying.
>
> http://issues.apache.org/jira/browse/HDFS-34
>
> Dyndns and the like don't get a look in. It's for applet security, but
> interferes with long-lived apps. Another reason to automate restarting your
> apps regularly.
>
>
Re: hadoop conf for dynamically changing ips
Posted by Steve Loughran <st...@apache.org>.
On 26/03/2010 16:39, Gokulakannan M wrote:
>>
>> It's what I do. You just have to make sure that if the IPAddrs change,
>>
>> everything gets restarted.
>
>
> Thanks Steve for the reply. "everything must be restarted" means the
> hadoop cluster or all the systems.
just the JVMs. All JVMs cache mapped IP addresses forever, very annoying.
http://issues.apache.org/jira/browse/HDFS-34
Dyndns and the like don't get a look in. It's for applet security, but
interferes with long-lived apps. Another reason to automate restarting
your apps regularly.
Re: hadoop conf for dynamically changing ips
Posted by Owen O'Malley <om...@apache.org>.
On Mar 26, 2010, at 9:39 AM, Gokulakannan M wrote:
>> I have a LAN in which the IPs of the machines will be
>> changed
>> dynamically by the DHCP sever.
I think you'd need to use a NAT translation so that inside your
cluster you have stable IP addrs in 10.x.x.x but the external IP addr
is set by DHCP.
-- Owen
RE: hadoop conf for dynamically changing ips
Posted by Gokulakannan M <go...@huawei.com>.
>
>It's what I do. You just have to make sure that if the IPAddrs change,
>
>everything gets restarted.
Thanks Steve for the reply. "everything must be restarted" means the
hadoop cluster or all the systems.
Thanks,
Gokul
-----Original Message-----
From: Steve Loughran [mailto:stevel@apache.org]
Sent: Friday, March 26, 2010 8:14 PM
To: common-user@hadoop.apache.org
Subject: Re: hadoop conf for dynamically changing ips
On 26/03/2010 14:31, Gokulakannan M wrote:
>
>
> Hi,
>
>
>
> I have a LAN in which the IPs of the machines will be changed
> dynamically by the DHCP sever.
>
>
>
> So for namenode, jobtracker, master and slave configurations
we
> could not give the IP.
>
> can the machine names be given for those configurations????
will
> that work??
>
It's what I do. You just have to make sure that if the IPAddrs change,
everything gets restarted.
Re: hadoop conf for dynamically changing ips
Posted by Steve Loughran <st...@apache.org>.
On 26/03/2010 14:31, Gokulakannan M wrote:
>
>
> Hi,
>
>
>
> I have a LAN in which the IPs of the machines will be changed
> dynamically by the DHCP sever.
>
>
>
> So for namenode, jobtracker, master and slave configurations we
> could not give the IP.
>
> can the machine names be given for those configurations???? will
> that work??
>
It's what I do. You just have to make sure that if the IPAddrs change,
everything gets restarted.