[jira] [Commented] (HDFS-4039) Add option for WebHDFS to be a proxy, rather than just 302 to the DNs

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474004#comment-13474004 ] 

Steve Loughran commented on HDFS-4039:
--------------------------------------

Based on my experience using webhdfs against remote EC2-hosted clusters, I can see the benefit of allowing webhdfs to proxy to the DNs, rather than redirect to them. I know this could be done with a separate proxy, but in a small5+ node cluster, this is one more thing to manage -and in the small clusters the NN load is too low to matter.

In a remote cluster, the local user can access the Namenode via its external name if the cluster security settings are set up to allow this. But WebHDFS 302s to the DNs via their internal IPAddrs, not the external addresses, so the redirects don't work. There's also the issue that you may not want to expose the internal DNs to the remote users, any more than you want the IPC ports to be visible. 

If webhdfs could be turned into a proxy, its external IP address could be used for all get/post operations, while the rest of the cluster was isolated from remote users. 
                
> Add option for WebHDFS to be a proxy, rather than just 302 to the DNs
> ---------------------------------------------------------------------
>
>                 Key: HDFS-4039
>                 URL: https://issues.apache.org/jira/browse/HDFS-4039
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 1.0.3
>            Reporter: Steve Loughran
>
> Add an option allowing webhdfs to proxy to the DNs, rather than redirecting to them. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira