You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by Charles Menguy <cm...@adobe.com> on 2014/04/30 20:05:36 UTC

Securely transferring data from HDFS to S3 using distcp

Hi,

I wasn't able to find any information regarding the security of using distcp to transfer data to S3 using something like "hadoop distcp hdfs://... s3n://<access-key>:<secret-key>@<bucket>/...".
I know for example in the Python library boto you can specify is_secure=True to use SSL, but it is not clear to me what distcp uses and if there is any way to turn SSL on.

Does anyone have any insight on that?

Thanks,

Charles