[jira] Issue Comment Edited: (HADOOP-3249) Browsing DFS behind gateway

["Martin Boeker (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589759#action_12589759 ] 

mboeker edited comment on HADOOP-3249 at 4/16/08 3:45 PM:
----------------------------------------------------------------

Hey guys,

Thanks for all the suggestions. I've been trying to get the ssh proxy to work, but nothing I've tried has been successful. Here's more specific information about the situation:

Internet
     to
Firewall: Ports 22, 50010-50080 open.
     to
Gateway/SSH: Running httpd, listening to port 50080. Forwarding ports 50020-50030 to machines inside the network (using iptables). The httpd and iptables forwarding works fine.
      to
Cluster: Standard hadoop network, nodes have IP addresses from 10.1.1.10 to 10.1.1.30

No matter what I try to do, I can't get the proxy to work. I tried the following:

[me@gateway~]$ ssh -D 50010 localhost     --> then:
[me@gateway~]$ telnet localhost 50010     --> works
[me@gateway~]$ telnet gw_internal_ip 50010     --> does not work, connection refused.
[me@gateway~]$ telnet gw_external_ip 50010    --> does not work, connection refused.
[me@cluster~]$ telnet gw_internal_ip 50010   --> does not work, connection refused.
[me@internet~]$ telnet gw_external_ip 50010   --> does not work, connection refused.

so I tried variations:
[me@gateway~]$ ssh -D 50010 gw_external_ip    --> same
[me@gateway~]$ ssh -D 50010 gw_internal_ip    --> same

[me@gateway~]$ ssh -gND 50010 localhost    --> error: bind, port in use (no matter what port I use) but the error isn't terminal, see:
[me@cluster~]$ telnet gw_internal_ip 50010    --> works
[me@internet~]$ telnet gw_external_ip 50010   --> does not work, connection refused

Keep in mind that all this time, the iptables forwarded ports are working fine..
[me@internet~]$ telnet gw_external_ip 50020    --> works

I don't know what the problem is.. I even tried tunneling on the gateway with ssh -L but that didn't do me any good either..

I hope someone can help me! Thanks again for all the feedback.

-Martin

      was (Author: mboeker):
    Hey guys,

Thanks for all the suggestions. I've been trying to get the ssh proxy to work, but nothing I've tried has been successful. Here's more specific information about the situation:

*Internet*
       |
Firewall: Ports 22, 50010-50080 open.
       |
Gateway/SSH: Running httpd, listening to port 50080. Forwarding ports 50020-50030 to machines inside the network (using iptables). The httpd and iptables forwarding works fine.
       |
Cluster: Standard hadoop network, nodes have IP addresses from 10.1.1.10 to 10.1.1.30

No matter what I try to do, I can't get the proxy to work. I tried the following:

[me@gateway~]$ ssh -D 50010 localhost     --> then:
[me@gateway~]$ telnet localhost 50010     --> works
[me@gateway~]$ telnet gw_internal_ip 50010     --> does not work, connection refused.
[me@gateway~]$ telnet gw_external_ip 50010    --> does not work, connection refused.
[me@cluster~]$ telnet gw_internal_ip 50010   --> does not work, connection refused.
[me@internet~]$ telnet gw_external_ip 50010   --> does not work, connection refused.

so I tried variations:
[me@gateway~]$ ssh -D 50010 gw_external_ip    --> same
[me@gateway~]$ ssh -D 50010 gw_internal_ip    --> same

[me@gateway~]$ ssh -gND 50010 localhost    --> error: bind, port in use (no matter what port I use) but the error isn't terminal, see:
[me@cluster~]$ telnet gw_internal_ip 50010    --> works
[me@internet~]$ telnet gw_external_ip 50010   --> does not work, connection refused

Keep in mind that all this time, the iptables forwarded ports are working fine..
[me@internet~]$ telnet gw_external_ip 50020    --> works

I don't know what the problem is.. I even tried tunneling on the gateway with ssh -L but that didn't do me any good either..

I hope someone can help me! Thanks again for all the feedback.

-Martin
  
> Browsing DFS behind gateway
> ---------------------------
>
>                 Key: HADOOP-3249
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3249
>             Project: Hadoop Core
>          Issue Type: Wish
>          Components: dfs
>    Affects Versions: 0.16.0, 0.16.1, 0.16.2
>         Environment: Red-Hat cluster
>            Reporter: Martin Boeker
>   Original Estimate: 5h
>  Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.