You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by st...@gmail.com on 2013/09/06 01:14:06 UTC
Tunneling over SSH
I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
installation. The internal IP of the machine is on a local subnet behind a
SSH-only firewall - 192.168.182.22.. I use static host names in all of the
config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for all
the servers. There is no problem connecting when I'm directly connected to
the subnet inside the firewall.
However, when I try to connect via the JAVA API from outside the firewall,
I get an error: Failed to find an available server in the list of servers:
[192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
interface that allows me to forward unlimited ports directly through the
SSH tunnel to the internal network - there is no issue with connecting to
Hadoop via Java or the web interface, and I can view the Accumuoo status
page at 50095 by just setting my Windows box to resolve the hostname to the
loopback local IP -> SSH -> 192.168.182.22:50095.
I think the problem is that Zookeeper is telling my Java process to try and
make a connection directly to 192.168.22.9997. If Zoo would use the
hostname, there'd be no problem as it'd resolve to the loopback, and get
tunneled along with everything else. But since it uses the actual IP, the
Windows box won't route that back through the SSH tunnel as it considers it
a local subnet outside of the firewall.
Anyone experienced this issue and have a solution? I guess one solution
might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
through the loopback (-> SSH), but I'm not seeing a good way to do that.
Thanks
Re: Tunneling over SSH
Posted by Mike Drob <md...@mdrob.com>.
There is some development going on as part of
ACCUMULO-1585<https://issues.apache.org/jira/browse/ACCUMULO-1585>[1]
to allow tservers to store the hostname instead of the ip address.
That
seems like a good place to start, although I'm not sure if this is the same
problem that you're seeing.
[1]: https://issues.apache.org/jira/browse/ACCUMULO-1585
Mike
<https://issues.apache.org/jira/browse/ACCUMULO-1585>
On Thu, Sep 5, 2013 at 7:14 PM, <st...@gmail.com> wrote:
> I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
> installation. The internal IP of the machine is on a local subnet behind a
> SSH-only firewall - 192.168.182.22.. I use static host names in all of the
> config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for all
> the servers. There is no problem connecting when I'm directly connected to
> the subnet inside the firewall.
>
> However, when I try to connect via the JAVA API from outside the firewall,
> I get an error: Failed to find an available server in the list of servers:
> [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
> interface that allows me to forward unlimited ports directly through the
> SSH tunnel to the internal network - there is no issue with connecting to
> Hadoop via Java or the web interface, and I can view the Accumuoo status
> page at 50095 by just setting my Windows box to resolve the hostname to the
> loopback local IP -> SSH -> 192.168.182.22:50095.
>
> I think the problem is that Zookeeper is telling my Java process to try
> and make a connection directly to 192.168.22.9997. If Zoo would use the
> hostname, there'd be no problem as it'd resolve to the loopback, and get
> tunneled along with everything else. But since it uses the actual IP, the
> Windows box won't route that back through the SSH tunnel as it considers it
> a local subnet outside of the firewall.
>
> Anyone experienced this issue and have a solution? I guess one solution
> might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
> through the loopback (-> SSH), but I'm not seeing a good way to do that.
>
> Thanks
>
Re: Tunneling over SSH
Posted by Christopher <ct...@apache.org>.
That's a good point. You can run the thrift proxy on the remote node,
and use port forwarding over SSH to talk to it.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Thu, Sep 5, 2013 at 11:33 PM, Eric Newton <er...@gmail.com> wrote:
> Speaking of Proxy... there's a Thrift Proxy that would accommodate a single
> port connection to do all client operations if hosted on the subnet. Bonus:
> you can use any thrift-supported language.
>
> Without the proxy, however, the data model (inherent to the BigTable design)
> is that the client can reach every tablet server, as well as the pointer to
> the root tablet on the fault-tolerant register (zookeeper/chubby). Without
> a network that supports this connectivity, you are fighting the
> architecture.
>
> -Eric
>
>
>
> On Thu, Sep 5, 2013 at 10:36 PM, Christopher <ct...@apache.org> wrote:
>>
>> You're right that ZK is instructing the client to talk directly to
>> 192.168.182.22:9997 (tablet server). As Mike suggested, this could be
>> resolved if we stored hostnames rather than IPs, and you had hostnames
>> mapped to the external IP, and ports forwarded over SSH.
>>
>> A more robust solution would be to have a client-side configuration
>> setting that allowed you to specify a SOCKS proxy. The standard system
>> properties "socksProxyHost" and "socksProxyPort" may even work today,
>> if you set them up as system properties in your client code before you
>> open a thrift connection... I haven't tested this myself.
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Thu, Sep 5, 2013 at 7:14 PM, <st...@gmail.com> wrote:
>> > I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo
>> > stand-alone
>> > installation. The internal IP of the machine is on a local subnet behind
>> > a
>> > SSH-only firewall - 192.168.182.22.. I use static host names in all of
>> > the
>> > config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for
>> > all
>> > the servers. There is no problem connecting when I'm directly connected
>> > to
>> > the subnet inside the firewall.
>> >
>> > However, when I try to connect via the JAVA API from outside the
>> > firewall, I
>> > get an error: Failed to find an available server in the list of servers:
>> > [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
>> > interface that allows me to forward unlimited ports directly through the
>> > SSH
>> > tunnel to the internal network - there is no issue with connecting to
>> > Hadoop
>> > via Java or the web interface, and I can view the Accumuoo status page
>> > at
>> > 50095 by just setting my Windows box to resolve the hostname to the
>> > loopback
>> > local IP -> SSH -> 192.168.182.22:50095.
>> >
>> > I think the problem is that Zookeeper is telling my Java process to try
>> > and
>> > make a connection directly to 192.168.22.9997. If Zoo would use the
>> > hostname, there'd be no problem as it'd resolve to the loopback, and get
>> > tunneled along with everything else. But since it uses the actual IP,
>> > the
>> > Windows box won't route that back through the SSH tunnel as it considers
>> > it
>> > a local subnet outside of the firewall.
>> >
>> > Anyone experienced this issue and have a solution? I guess one solution
>> > might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
>> > through the loopback (-> SSH), but I'm not seeing a good way to do that.
>> >
>> > Thanks
>
>
Re: Tunneling over SSH
Posted by Eric Newton <er...@gmail.com>.
Speaking of Proxy... there's a Thrift Proxy that would accommodate a single
port connection to do all client operations if hosted on the subnet.
Bonus: you can use any thrift-supported language.
Without the proxy, however, the data model (inherent to the BigTable
design) is that the client can reach every tablet server, as well as the
pointer to the root tablet on the fault-tolerant register
(zookeeper/chubby). Without a network that supports this connectivity, you
are fighting the architecture.
-Eric
On Thu, Sep 5, 2013 at 10:36 PM, Christopher <ct...@apache.org> wrote:
> You're right that ZK is instructing the client to talk directly to
> 192.168.182.22:9997 (tablet server). As Mike suggested, this could be
> resolved if we stored hostnames rather than IPs, and you had hostnames
> mapped to the external IP, and ports forwarded over SSH.
>
> A more robust solution would be to have a client-side configuration
> setting that allowed you to specify a SOCKS proxy. The standard system
> properties "socksProxyHost" and "socksProxyPort" may even work today,
> if you set them up as system properties in your client code before you
> open a thrift connection... I haven't tested this myself.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Thu, Sep 5, 2013 at 7:14 PM, <st...@gmail.com> wrote:
> > I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
> > installation. The internal IP of the machine is on a local subnet behind
> a
> > SSH-only firewall - 192.168.182.22.. I use static host names in all of
> the
> > config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for
> all
> > the servers. There is no problem connecting when I'm directly connected
> to
> > the subnet inside the firewall.
> >
> > However, when I try to connect via the JAVA API from outside the
> firewall, I
> > get an error: Failed to find an available server in the list of servers:
> > [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
> > interface that allows me to forward unlimited ports directly through the
> SSH
> > tunnel to the internal network - there is no issue with connecting to
> Hadoop
> > via Java or the web interface, and I can view the Accumuoo status page at
> > 50095 by just setting my Windows box to resolve the hostname to the
> loopback
> > local IP -> SSH -> 192.168.182.22:50095.
> >
> > I think the problem is that Zookeeper is telling my Java process to try
> and
> > make a connection directly to 192.168.22.9997. If Zoo would use the
> > hostname, there'd be no problem as it'd resolve to the loopback, and get
> > tunneled along with everything else. But since it uses the actual IP, the
> > Windows box won't route that back through the SSH tunnel as it considers
> it
> > a local subnet outside of the firewall.
> >
> > Anyone experienced this issue and have a solution? I guess one solution
> > might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
> > through the loopback (-> SSH), but I'm not seeing a good way to do that.
> >
> > Thanks
>
Re: Tunneling over SSH
Posted by Christopher <ct...@apache.org>.
The easiest solution may be to set up a VPN, though.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Thu, Sep 5, 2013 at 10:36 PM, Christopher <ct...@apache.org> wrote:
> You're right that ZK is instructing the client to talk directly to
> 192.168.182.22:9997 (tablet server). As Mike suggested, this could be
> resolved if we stored hostnames rather than IPs, and you had hostnames
> mapped to the external IP, and ports forwarded over SSH.
>
> A more robust solution would be to have a client-side configuration
> setting that allowed you to specify a SOCKS proxy. The standard system
> properties "socksProxyHost" and "socksProxyPort" may even work today,
> if you set them up as system properties in your client code before you
> open a thrift connection... I haven't tested this myself.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Thu, Sep 5, 2013 at 7:14 PM, <st...@gmail.com> wrote:
>> I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
>> installation. The internal IP of the machine is on a local subnet behind a
>> SSH-only firewall - 192.168.182.22.. I use static host names in all of the
>> config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for all
>> the servers. There is no problem connecting when I'm directly connected to
>> the subnet inside the firewall.
>>
>> However, when I try to connect via the JAVA API from outside the firewall, I
>> get an error: Failed to find an available server in the list of servers:
>> [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
>> interface that allows me to forward unlimited ports directly through the SSH
>> tunnel to the internal network - there is no issue with connecting to Hadoop
>> via Java or the web interface, and I can view the Accumuoo status page at
>> 50095 by just setting my Windows box to resolve the hostname to the loopback
>> local IP -> SSH -> 192.168.182.22:50095.
>>
>> I think the problem is that Zookeeper is telling my Java process to try and
>> make a connection directly to 192.168.22.9997. If Zoo would use the
>> hostname, there'd be no problem as it'd resolve to the loopback, and get
>> tunneled along with everything else. But since it uses the actual IP, the
>> Windows box won't route that back through the SSH tunnel as it considers it
>> a local subnet outside of the firewall.
>>
>> Anyone experienced this issue and have a solution? I guess one solution
>> might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
>> through the loopback (-> SSH), but I'm not seeing a good way to do that.
>>
>> Thanks
Re: Tunneling over SSH
Posted by Christopher <ct...@apache.org>.
You're right that ZK is instructing the client to talk directly to
192.168.182.22:9997 (tablet server). As Mike suggested, this could be
resolved if we stored hostnames rather than IPs, and you had hostnames
mapped to the external IP, and ports forwarded over SSH.
A more robust solution would be to have a client-side configuration
setting that allowed you to specify a SOCKS proxy. The standard system
properties "socksProxyHost" and "socksProxyPort" may even work today,
if you set them up as system properties in your client code before you
open a thrift connection... I haven't tested this myself.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Thu, Sep 5, 2013 at 7:14 PM, <st...@gmail.com> wrote:
> I'm trying to tunnel via SSH to a single Hadoop,Zoo, Accumulo stand-alone
> installation. The internal IP of the machine is on a local subnet behind a
> SSH-only firewall - 192.168.182.22.. I use static host names in all of the
> config files (Accumulo, Zoo, Hadoop) that resolve to 192.168.182.22 for all
> the servers. There is no problem connecting when I'm directly connected to
> the subnet inside the firewall.
>
> However, when I try to connect via the JAVA API from outside the firewall, I
> get an error: Failed to find an available server in the list of servers:
> [192.168.182.22:9997:9997 (120000)]. I've created a Windows Loopback
> interface that allows me to forward unlimited ports directly through the SSH
> tunnel to the internal network - there is no issue with connecting to Hadoop
> via Java or the web interface, and I can view the Accumuoo status page at
> 50095 by just setting my Windows box to resolve the hostname to the loopback
> local IP -> SSH -> 192.168.182.22:50095.
>
> I think the problem is that Zookeeper is telling my Java process to try and
> make a connection directly to 192.168.22.9997. If Zoo would use the
> hostname, there'd be no problem as it'd resolve to the loopback, and get
> tunneled along with everything else. But since it uses the actual IP, the
> Windows box won't route that back through the SSH tunnel as it considers it
> a local subnet outside of the firewall.
>
> Anyone experienced this issue and have a solution? I guess one solution
> might be to 'trick' Windows into forwarding the 192.168.x.y subnet back
> through the loopback (-> SSH), but I'm not seeing a good way to do that.
>
> Thanks