You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@whirr.apache.org by "Tom White (JIRA)" <ji...@apache.org> on 2010/09/01 19:09:52 UTC

[jira] Created: (WHIRR-81) Support efficient Hadoop distcp from external clusters

Support efficient Hadoop distcp from external clusters
------------------------------------------------------

                 Key: WHIRR-81
                 URL: https://issues.apache.org/jira/browse/WHIRR-81
             Project: Whirr
          Issue Type: New Feature
          Components: service/hadoop
            Reporter: Tom White


On EC2 currently all external traffic to a Hadoop cluster is proxied through the namenode, which make distcp impractical. This JIRA is to explore ways to improve this operation, possible candidates include a SocketFactory implementation that is aware of the cloud provider's networking (and can supply the public addresses appropriately), or a VPN. Ideally this would support different cloud providers, although it is possible that different providers need different solutions.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.