[EC2] connecting to hbase from outside

[Patrick Salami <pa...@temboo.com>]

Hi all,

I just ran into the following issue:

I have an hbase cluster running in EC2 that was launched with the 0.20.4 EC2 launch scripts. I'd like to connect to that cluster using an hbase client running locally (outside of EC2).
I pulled down my hbase-site.xml file from the master, which references the internal hostnames of the master and zookeeper nodes. I changed those to the public hostnames.
When running my client, it connects to the zk node without any problem; however, the zk node only knows about the internal hostnames of the region servers (IPs in the 10.193.x.x range).
Needless to say, I can't connect to the region servers at their 10.193 addresses from my local machine, so my client doesn't work.

Does anyone know if there is a configuration flag I can set somewhere to get the public hostnames out of zookeeper instead of the private ones?

Best,

Patrick Salami - Senior Software Engineer
[ p. (858) 449-2241 e. patrick.salami@temboo.com ]
Temboo Inc - 6935 West Bernardo Dr - San Diego, CA 92127

www.temboo.com

Re: [EC2] connecting to hbase from outside

[Andrew Purtell <ap...@apache.org>]

This is a problem with how the Hadoop DNS utility class, which HBase uses, works internally on EC2. The nodes can only know their internal host names. EC2 rDNS does not reveal the external hostname. 

Search for earlier discussion on hbase-user@ in this regard. Some options include:

- Use Thrift or REST gateways. Run as many as you want, either on separate instances or colocated with the region servers. Connect round robin from client side to balance load. They will access the HBase cluster from up on EC2 where internal DNS names are resolvable on behalf of your clients.

- Use SSH as a SOCKS 5 proxy, which will proxy DNS requests as well as TCP connections. 

- Use a VPN solution (e.g. EC2's "Cloud VPN") and delegate .internal in your local DNS to the EC2 nameservers. 

- Run your clients up on EC2. 

> From: Patrick Salami <pa...@temboo.com>
> Subject: [EC2] connecting to hbase from outside
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> Cc: "Adam Purtell" <ap...@apache.org>
> Date: Wednesday, June 2, 2010, 2:22 PM
> Hi all,
> 
> I just ran into the following issue:
> 
> I have an hbase cluster running in EC2 that was launched
> with the 0.20.4 EC2 launch scripts. I'd like to connect to
> that cluster using an hbase client running locally (outside
> of EC2).
> I pulled down my hbase-site.xml file from the master, which
> references the internal hostnames of the master and
> zookeeper nodes. I changed those to the public hostnames.
> When running my client, it connects to the zk node without
> any problem; however, the zk node only knows about the
> internal hostnames of the region servers (IPs in the
> 10.193.x.x range).
> Needless to say, I can't connect to the region servers at
> their 10.193 addresses from my local machine, so my client
> doesn't work.
> 
> Does anyone know if there is a configuration flag I can set
> somewhere to get the public hostnames out of zookeeper
> instead of the private ones?
> 
> Best,
> 
> Patrick Salami - Senior Software Engineer
> [ p. (858) 449-2241 e. patrick.salami@temboo.com
> ]
> Temboo Inc - 6935 West Bernardo Dr - San Diego, CA 92127
> 
> www.temboo.com
> 
>