You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Ananth T. Sarathy" <an...@gmail.com> on 2009/09/14 19:31:28 UTC
Region Server on EC2
When we try to connect to hbase running on an ec2 server from a machine
outside ec2, the regionserver on port 60020 is trying to connect to the
internal ip address rather than the external dns
i feel it's the same issue as this
http://issues.apache.org/jira/browse/HBASE-1279
but my client keeps getting
09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
10.244.150.243:60020. Already tried 26 time(s).
09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
10.244.150.243:60020. Already tried 27 time(s).
hbase-site.xml
<name>hbase.master</name>
<value>ec2externaldns:60000</value>
<description>The host and port that the HBase master runs at.
A value of 'local' runs the master and a regionserver in
a single process.
</description>
</property>
<property>
<name>hbase.regionserver</name>
<value>ec2externaldns:60020</value>
<description>The host and port a HBase region server runs at.
</description>
</property>
<property>
<name>hbase.regionserver.dns.interface</name>
<value>0.0.0.0</value>
<description>Name of the network interface which a regionserver
should use to determine it's "real" IP address. This lookup
prevents strings like "localhost" and "127.0.0.1" from being
reported back to the master.
</description>
</property>
Ananth T Sarathy
Re: Region Server on EC2
Posted by Andrew Purtell <ap...@apache.org>.
It should be possible to associate N+1 elastic IPs with the master and N region server instances. Allocate a pool of elastic IPs and associate them as appropriate when (re)starting instances. Be sure to set the security group policy to allow connections to the ZK, master, and region server ports. As J-D mentions, latency will be high. There may be other issues. HRPC is not tuned for WAN operation.
You will have an easier time and better performance using batching and either the Thrift or REST (Stargate) gateway option. Then you need allocate only one elastic IP for HBase access from some remote client.
- Andy
________________________________
From: Jean-Daniel Cryans <jd...@apache.org>
To: hbase-user@hadoop.apache.org
Sent: Monday, September 14, 2009 11:36:03 AM
Subject: Re: Region Server on EC2
Oh no sorry I misled you, this is another problem. You cannot use the
external addresses because they change so HBase _has_ to use the
internal addresses or you could have very weird problems.
So there is another question: why do you want to connect to an EC2
cluster remotely? Any client connection will suffer from high latency.
J-D
On Mon, Sep 14, 2009 at 2:03 PM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Hmm thanks,
> Where exactly am I supposed to find the nameserver? It's not the external
> dns for the box right?
>
> Ananth T Sarathy
>
>
> On Mon, Sep 14, 2009 at 1:35 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> You have to set hbase.regionserver.dns.interface and
>> hbase.regionserver.dns.nameserver with valid values. The interface
>> will probably be something like "eth0" and the nameserver is the one
>> provided by EC2.
>>
>> J-D
>>
>> On Mon, Sep 14, 2009 at 1:31 PM, Ananth T. Sarathy
>> <an...@gmail.com> wrote:
>> > When we try to connect to hbase running on an ec2 server from a machine
>> > outside ec2, the regionserver on port 60020 is trying to connect to the
>> > internal ip address rather than the external dns
>> >
>> > i feel it's the same issue as this
>> >
>> > http://issues.apache.org/jira/browse/HBASE-1279
>> >
>> >
>> > but my client keeps getting
>> >
>> >
>> > 09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
>> > 10.244.150.243:60020. Already tried 26 time(s).
>> > 09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
>> > 10.244.150.243:60020. Already tried 27 time(s).
>> >
>> >
>> >
>> > hbase-site.xml
>> >
>> >
>> > <name>hbase.master</name>
>> > <value>ec2externaldns:60000</value>
>> > <description>The host and port that the HBase master runs at.
>> > A value of 'local' runs the master and a regionserver in
>> > a single process.
>> > </description>
>> > </property>
>> >
>> >
>> > <property>
>> > <name>hbase.regionserver</name>
>> > <value>ec2externaldns:60020</value>
>> > <description>The host and port a HBase region server runs at.
>> > </description>
>> > </property>
>> >
>> > <property>
>> > <name>hbase.regionserver.dns.interface</name>
>> > <value>0.0.0.0</value>
>> > <description>Name of the network interface which a regionserver
>> > should use to determine it's "real" IP address. This lookup
>> > prevents strings like "localhost" and "127.0.0.1" from being
>> > reported back to the master.
>> > </description>
>> > </property>
>> >
>> > Ananth T Sarathy
>> >
>>
>
Re: Region Server on EC2
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Well we assigned and elastic IP.
Well I running remotely to test. We will have apps on other ec2 virtual
machines
I know the external can change if it restarts, but we are willing to
reconfigure if that is case, but I still would like to be able to access
hbase from outside.
Ananth T Sarathy
On Mon, Sep 14, 2009 at 2:36 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Oh no sorry I misled you, this is another problem. You cannot use the
> external addresses because they change so HBase _has_ to use the
> internal addresses or you could have very weird problems.
>
> So there is another question: why do you want to connect to an EC2
> cluster remotely? Any client connection will suffer from high latency.
>
> J-D
>
> On Mon, Sep 14, 2009 at 2:03 PM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
> > Hmm thanks,
> > Where exactly am I supposed to find the nameserver? It's not the
> external
> > dns for the box right?
> >
> > Ananth T Sarathy
> >
> >
> > On Mon, Sep 14, 2009 at 1:35 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> You have to set hbase.regionserver.dns.interface and
> >> hbase.regionserver.dns.nameserver with valid values. The interface
> >> will probably be something like "eth0" and the nameserver is the one
> >> provided by EC2.
> >>
> >> J-D
> >>
> >> On Mon, Sep 14, 2009 at 1:31 PM, Ananth T. Sarathy
> >> <an...@gmail.com> wrote:
> >> > When we try to connect to hbase running on an ec2 server from a
> machine
> >> > outside ec2, the regionserver on port 60020 is trying to connect to
> the
> >> > internal ip address rather than the external dns
> >> >
> >> > i feel it's the same issue as this
> >> >
> >> > http://issues.apache.org/jira/browse/HBASE-1279
> >> >
> >> >
> >> > but my client keeps getting
> >> >
> >> >
> >> > 09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
> >> > 10.244.150.243:60020. Already tried 26 time(s).
> >> > 09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
> >> > 10.244.150.243:60020. Already tried 27 time(s).
> >> >
> >> >
> >> >
> >> > hbase-site.xml
> >> >
> >> >
> >> > <name>hbase.master</name>
> >> > <value>ec2externaldns:60000</value>
> >> > <description>The host and port that the HBase master runs at.
> >> > A value of 'local' runs the master and a regionserver in
> >> > a single process.
> >> > </description>
> >> > </property>
> >> >
> >> >
> >> > <property>
> >> > <name>hbase.regionserver</name>
> >> > <value>ec2externaldns:60020</value>
> >> > <description>The host and port a HBase region server runs at.
> >> > </description>
> >> > </property>
> >> >
> >> > <property>
> >> > <name>hbase.regionserver.dns.interface</name>
> >> > <value>0.0.0.0</value>
> >> > <description>Name of the network interface which a regionserver
> >> > should use to determine it's "real" IP address. This lookup
> >> > prevents strings like "localhost" and "127.0.0.1" from being
> >> > reported back to the master.
> >> > </description>
> >> > </property>
> >> >
> >> > Ananth T Sarathy
> >> >
> >>
> >
>
Re: Region Server on EC2
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Oh no sorry I misled you, this is another problem. You cannot use the
external addresses because they change so HBase _has_ to use the
internal addresses or you could have very weird problems.
So there is another question: why do you want to connect to an EC2
cluster remotely? Any client connection will suffer from high latency.
J-D
On Mon, Sep 14, 2009 at 2:03 PM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> Hmm thanks,
> Where exactly am I supposed to find the nameserver? It's not the external
> dns for the box right?
>
> Ananth T Sarathy
>
>
> On Mon, Sep 14, 2009 at 1:35 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> You have to set hbase.regionserver.dns.interface and
>> hbase.regionserver.dns.nameserver with valid values. The interface
>> will probably be something like "eth0" and the nameserver is the one
>> provided by EC2.
>>
>> J-D
>>
>> On Mon, Sep 14, 2009 at 1:31 PM, Ananth T. Sarathy
>> <an...@gmail.com> wrote:
>> > When we try to connect to hbase running on an ec2 server from a machine
>> > outside ec2, the regionserver on port 60020 is trying to connect to the
>> > internal ip address rather than the external dns
>> >
>> > i feel it's the same issue as this
>> >
>> > http://issues.apache.org/jira/browse/HBASE-1279
>> >
>> >
>> > but my client keeps getting
>> >
>> >
>> > 09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
>> > 10.244.150.243:60020. Already tried 26 time(s).
>> > 09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
>> > 10.244.150.243:60020. Already tried 27 time(s).
>> >
>> >
>> >
>> > hbase-site.xml
>> >
>> >
>> > <name>hbase.master</name>
>> > <value>ec2externaldns:60000</value>
>> > <description>The host and port that the HBase master runs at.
>> > A value of 'local' runs the master and a regionserver in
>> > a single process.
>> > </description>
>> > </property>
>> >
>> >
>> > <property>
>> > <name>hbase.regionserver</name>
>> > <value>ec2externaldns:60020</value>
>> > <description>The host and port a HBase region server runs at.
>> > </description>
>> > </property>
>> >
>> > <property>
>> > <name>hbase.regionserver.dns.interface</name>
>> > <value>0.0.0.0</value>
>> > <description>Name of the network interface which a regionserver
>> > should use to determine it's "real" IP address. This lookup
>> > prevents strings like "localhost" and "127.0.0.1" from being
>> > reported back to the master.
>> > </description>
>> > </property>
>> >
>> > Ananth T Sarathy
>> >
>>
>
Re: Region Server on EC2
Posted by "Ananth T. Sarathy" <an...@gmail.com>.
Hmm thanks,
Where exactly am I supposed to find the nameserver? It's not the external
dns for the box right?
Ananth T Sarathy
On Mon, Sep 14, 2009 at 1:35 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> You have to set hbase.regionserver.dns.interface and
> hbase.regionserver.dns.nameserver with valid values. The interface
> will probably be something like "eth0" and the nameserver is the one
> provided by EC2.
>
> J-D
>
> On Mon, Sep 14, 2009 at 1:31 PM, Ananth T. Sarathy
> <an...@gmail.com> wrote:
> > When we try to connect to hbase running on an ec2 server from a machine
> > outside ec2, the regionserver on port 60020 is trying to connect to the
> > internal ip address rather than the external dns
> >
> > i feel it's the same issue as this
> >
> > http://issues.apache.org/jira/browse/HBASE-1279
> >
> >
> > but my client keeps getting
> >
> >
> > 09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
> > 10.244.150.243:60020. Already tried 26 time(s).
> > 09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
> > 10.244.150.243:60020. Already tried 27 time(s).
> >
> >
> >
> > hbase-site.xml
> >
> >
> > <name>hbase.master</name>
> > <value>ec2externaldns:60000</value>
> > <description>The host and port that the HBase master runs at.
> > A value of 'local' runs the master and a regionserver in
> > a single process.
> > </description>
> > </property>
> >
> >
> > <property>
> > <name>hbase.regionserver</name>
> > <value>ec2externaldns:60020</value>
> > <description>The host and port a HBase region server runs at.
> > </description>
> > </property>
> >
> > <property>
> > <name>hbase.regionserver.dns.interface</name>
> > <value>0.0.0.0</value>
> > <description>Name of the network interface which a regionserver
> > should use to determine it's "real" IP address. This lookup
> > prevents strings like "localhost" and "127.0.0.1" from being
> > reported back to the master.
> > </description>
> > </property>
> >
> > Ananth T Sarathy
> >
>
Re: Region Server on EC2
Posted by Jean-Daniel Cryans <jd...@apache.org>.
You have to set hbase.regionserver.dns.interface and
hbase.regionserver.dns.nameserver with valid values. The interface
will probably be something like "eth0" and the nameserver is the one
provided by EC2.
J-D
On Mon, Sep 14, 2009 at 1:31 PM, Ananth T. Sarathy
<an...@gmail.com> wrote:
> When we try to connect to hbase running on an ec2 server from a machine
> outside ec2, the regionserver on port 60020 is trying to connect to the
> internal ip address rather than the external dns
>
> i feel it's the same issue as this
>
> http://issues.apache.org/jira/browse/HBASE-1279
>
>
> but my client keeps getting
>
>
> 09/09/14 13:31:16 INFO ipc.HBaseClass: Retrying connect to server: /
> 10.244.150.243:60020. Already tried 26 time(s).
> 09/09/14 13:31:37 INFO ipc.HBaseClass: Retrying connect to server: /
> 10.244.150.243:60020. Already tried 27 time(s).
>
>
>
> hbase-site.xml
>
>
> <name>hbase.master</name>
> <value>ec2externaldns:60000</value>
> <description>The host and port that the HBase master runs at.
> A value of 'local' runs the master and a regionserver in
> a single process.
> </description>
> </property>
>
>
> <property>
> <name>hbase.regionserver</name>
> <value>ec2externaldns:60020</value>
> <description>The host and port a HBase region server runs at.
> </description>
> </property>
>
> <property>
> <name>hbase.regionserver.dns.interface</name>
> <value>0.0.0.0</value>
> <description>Name of the network interface which a regionserver
> should use to determine it's "real" IP address. This lookup
> prevents strings like "localhost" and "127.0.0.1" from being
> reported back to the master.
> </description>
> </property>
>
> Ananth T Sarathy
>