You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Martin Boeker (JIRA)" <ji...@apache.org> on 2008/04/14 21:57:05 UTC
[jira] Created: (HADOOP-3249) Browsing DFS behind gateway
Browsing DFS behind gateway
---------------------------
Key: HADOOP-3249
URL: https://issues.apache.org/jira/browse/HADOOP-3249
Project: Hadoop Core
Issue Type: Wish
Components: dfs
Affects Versions: 0.16.2, 0.16.1, 0.16.0
Environment: Red-Hat cluster
Reporter: Martin Boeker
Dear Hadoop guys,
I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
[gateway_external_IP]:50070 >> [node_internal_IP]:50070
This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
Thanks a lot in advance,
-Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Martin Boeker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589759#action_12589759 ]
Martin Boeker commented on HADOOP-3249:
---------------------------------------
Hey guys,
Thanks for all the suggestions. I've been trying to get the ssh proxy to work, but nothing I've tried has been successful. Here's more specific information about the situation:
*Internet*
|
Firewall: Ports 22, 50010-50080 open.
|
Gateway/SSH: Running httpd, listening to port 50080. Forwarding ports 50020-50030 to machines inside the network (using iptables). The httpd and iptables forwarding works fine.
|
Cluster: Standard hadoop network, nodes have IP addresses from 10.1.1.10 to 10.1.1.30
No matter what I try to do, I can't get the proxy to work. I tried the following:
[me@gateway~]$ ssh -D 50010 localhost --> then:
[me@gateway~]$ telnet localhost 50010 --> works
[me@gateway~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@gateway~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
[me@cluster~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
so I tried variations:
[me@gateway~]$ ssh -D 50010 gw_external_ip --> same
[me@gateway~]$ ssh -D 50010 gw_internal_ip --> same
[me@gateway~]$ ssh -gND 50010 localhost --> error: bind, port in use (no matter what port I use) but the error isn't terminal, see:
[me@cluster~]$ telnet gw_internal_ip 50010 --> works
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused
Keep in mind that all this time, the iptables forwarded ports are working fine..
[me@internet~]$ telnet gw_external_ip 50020 --> works
I don't know what the problem is.. I even tried tunneling on the gateway with ssh -L but that didn't do me any good either..
I hope someone can help me! Thanks again for all the feedback.
-Martin
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589784#action_12589784 ]
Raghu Angadi commented on HADOOP-3249:
--------------------------------------
With "-g" option, my RHEL4 ssh correctly binds to "0.0.0.0".
I don't see any need for iptables and httpd redirects with "ssh -gD". I would try this set up without the iptable configured on the gw machine. Once 'you@internet: telnet gw_external_ip 50010' works, then you just need to use "gw_external_ip:50010" as your browser socks proxy on 'internet'.
This is not a bug in Hadoop in any case . Better place is hadoop-user but for now it ok, you can update this jira.
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Christophe Taton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589238#action_12589238 ]
Christophe Taton commented on HADOOP-3249:
------------------------------------------
If you are able to open one port on your gateway, then you can probably open a socks proxy server on it (have a look at dante:http://www.inet.no/dante/), in which case all your users would have to setup the proxy address in their browser to point to your gateway.
Concerning the JSPs, you have to rebuild the hadoop-core jar to make your changes available. The JSPs are statically compiled into servlets during the build process.
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Christophe Taton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588764#action_12588764 ]
Christophe Taton commented on HADOOP-3249:
------------------------------------------
You should be able to do this using a SOCKS proxy:
- create the proxy using SSH: ssh -D 1080 gateway (you can also do this with some windows SSH clients such as putty): this will create a proxy listening on localhost:1080 and forwarding connections as if they were created from your gateway;
- tell your browser to use your proxy (if you use firefox, edit the preferences, advanced tab, network tab, settings button, enable SOCKS to localhost, port 1080; there is also a plug-in named foxyproxy).
Hope this helps...
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-3249:
---------------------------------
Component/s: (was: dfs)
Priority: Minor (was: Major)
Affects Version/s: (was: 0.16.2)
(was: 0.16.1)
(was: 0.16.0)
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Priority: Minor
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Martin Boeker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589227#action_12589227 ]
Martin Boeker commented on HADOOP-3249:
---------------------------------------
Christophe,
Thanks for the suggestion. It's acceptable to have people enter a proxy server into their browser settings to view the DFS report page, but I can't ask them to use ssh to create their own proxy 'server' -- one of the reasons for that is that ssh-ing to the gateway requires a user account which is not an option (security reasons, not something I can do anything about).
Can I run a proxy server on the gateway somehow? Another option would be if I could tell dfshealth.jsp that the url for each node is something like this:
http://gateway_external_IP:10001/, http://gateway_external_IP:10002/, etc instead of http://xenhost-123.../, but of course I would want this change to only show up in the html interface, the rest of the hadoop setup should still keep the hostnames and IP addresses unchanged.
One thing I've noticed is that the only place I find a dfshealth.jsp file is under /hadoop/src/webapps/dfs, but making changes to that file doesn't affect the port 50070 interface.. I've noticed though that /hadoop/webapps/dfs/index.html is the page that forwards at port 50070, but I can't figure out where the dfshealth.jsp is that is shown through the web interface.
Sorry if that's kind of confusing, I'm in the middle of a meeting while writing this! Thank you again for your feedback.
-Martin
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3249) Browsing DFS behind
gateway
Posted by "Martin Boeker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589759#action_12589759 ]
mboeker edited comment on HADOOP-3249 at 4/16/08 3:45 PM:
----------------------------------------------------------------
Hey guys,
Thanks for all the suggestions. I've been trying to get the ssh proxy to work, but nothing I've tried has been successful. Here's more specific information about the situation:
Internet
to
Firewall: Ports 22, 50010-50080 open.
to
Gateway/SSH: Running httpd, listening to port 50080. Forwarding ports 50020-50030 to machines inside the network (using iptables). The httpd and iptables forwarding works fine.
to
Cluster: Standard hadoop network, nodes have IP addresses from 10.1.1.10 to 10.1.1.30
No matter what I try to do, I can't get the proxy to work. I tried the following:
[me@gateway~]$ ssh -D 50010 localhost --> then:
[me@gateway~]$ telnet localhost 50010 --> works
[me@gateway~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@gateway~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
[me@cluster~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
so I tried variations:
[me@gateway~]$ ssh -D 50010 gw_external_ip --> same
[me@gateway~]$ ssh -D 50010 gw_internal_ip --> same
[me@gateway~]$ ssh -gND 50010 localhost --> error: bind, port in use (no matter what port I use) but the error isn't terminal, see:
[me@cluster~]$ telnet gw_internal_ip 50010 --> works
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused
Keep in mind that all this time, the iptables forwarded ports are working fine..
[me@internet~]$ telnet gw_external_ip 50020 --> works
I don't know what the problem is.. I even tried tunneling on the gateway with ssh -L but that didn't do me any good either..
I hope someone can help me! Thanks again for all the feedback.
-Martin
was (Author: mboeker):
Hey guys,
Thanks for all the suggestions. I've been trying to get the ssh proxy to work, but nothing I've tried has been successful. Here's more specific information about the situation:
*Internet*
|
Firewall: Ports 22, 50010-50080 open.
|
Gateway/SSH: Running httpd, listening to port 50080. Forwarding ports 50020-50030 to machines inside the network (using iptables). The httpd and iptables forwarding works fine.
|
Cluster: Standard hadoop network, nodes have IP addresses from 10.1.1.10 to 10.1.1.30
No matter what I try to do, I can't get the proxy to work. I tried the following:
[me@gateway~]$ ssh -D 50010 localhost --> then:
[me@gateway~]$ telnet localhost 50010 --> works
[me@gateway~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@gateway~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
[me@cluster~]$ telnet gw_internal_ip 50010 --> does not work, connection refused.
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused.
so I tried variations:
[me@gateway~]$ ssh -D 50010 gw_external_ip --> same
[me@gateway~]$ ssh -D 50010 gw_internal_ip --> same
[me@gateway~]$ ssh -gND 50010 localhost --> error: bind, port in use (no matter what port I use) but the error isn't terminal, see:
[me@cluster~]$ telnet gw_internal_ip 50010 --> works
[me@internet~]$ telnet gw_external_ip 50010 --> does not work, connection refused
Keep in mind that all this time, the iptables forwarded ports are working fine..
[me@internet~]$ telnet gw_external_ip 50020 --> works
I don't know what the problem is.. I even tried tunneling on the gateway with ssh -L but that didn't do me any good either..
I hope someone can help me! Thanks again for all the feedback.
-Martin
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Martin Boeker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589763#action_12589763 ]
Martin Boeker commented on HADOOP-3249:
---------------------------------------
Update:
I decided I'd test the setup on a different system, and it works, so it's something specific about this gateway... I checked /etc/sshd_config and TCP forwarding is allowed.. anyone have any suggestions as to what other settings or configuration file I should take a look at? The test system where I got the ssh -gD proxy to work is FedCor 8, but the gateway is RHEL ES4... Could there be some default setting I'm not seeing? I checked the firewall, the settings seem to be fine.
Thanks,
Martin
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589239#action_12589239 ]
Raghu Angadi commented on HADOOP-3249:
--------------------------------------
> Can I run a proxy server on the gateway somehow?
What I meant is that, you could do 'ssh -D 1080 localhost' on gateway machine. This will work as long as users can access port 1080 on the gateway.
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3249) Browsing DFS behind gateway
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589235#action_12589235 ]
Raghu Angadi commented on HADOOP-3249:
--------------------------------------
bq. It's acceptable to have people enter a proxy server into their browser settings to view the DFS report page, but I can't ask them to use ssh to create their own proxy 'server' - one of the reasons for that is that ssh-ing to the gateway requires a user account which is not an option (security reasons, not something I can do anything about).
Proxy needs to be set up only once (by one user). SSH is just one way of setting up a SOCKS proxy. You could run some other proxy. If you do use 'ssh -D', make sure SSH accepts connections from external hosts.
> Can I run a proxy server on the gateway somehow?
Yes. SSH -D works very well.
> Browsing DFS behind gateway
> ---------------------------
>
> Key: HADOOP-3249
> URL: https://issues.apache.org/jira/browse/HADOOP-3249
> Project: Hadoop Core
> Issue Type: Wish
> Components: dfs
> Affects Versions: 0.16.0, 0.16.1, 0.16.2
> Environment: Red-Hat cluster
> Reporter: Martin Boeker
> Original Estimate: 5h
> Remaining Estimate: 5h
>
> Dear Hadoop guys,
> I'm urgently trying to make a way for users to be able to see the contents of a Hadoop DFS that is behind a gateway. I'm using port forwarding on the gateway itself to point to the DFS web interface, something like this:
> [gateway_external_IP]:50070 >> [node_internal_IP]:50070
> This works fine, if I go to http://gateway_external_ip:50070/ I can view the DFS cluster html page from the outside world. The problem is that if I click on any of the slave node links, it forwards to http://node_hostname/.., which obviously doesn't work. I really need to get this going, a couple of projects require this to be implemented.
> I'm willing to do this any way possible, I don't really need to use the 50070 web interface, even a simple directory structure would do, but I'm not sure how to implement that either, because I don't know of a way to make an httpd or ftpd use "bin/hadoop dfs -lsr /" as the root directory. I'd also be willing to make people use a proxy server if that would fix my issue somehow..
> If anyone can help, I would greatly appreciate it, like I said it's kind of urgent and I'm running out of ideas to try..
> Thanks a lot in advance,
> -Martin
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.