You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by David Parks <da...@yahoo.com> on 2013/01/29 04:31:47 UTC

Symbolic links available in 1.0.3?

Is it possible to use symbolic links in 1.0.3?

 

If yes: can I use symbolic links to create a single, final directory
structure of files from many locations; then use DistCp/S3DistCp to copy
that final directory structure to another filesystem such as S3?

 

Usecase:

I currently launch 4 S3DistCp jobs concurrently (a royal pain in the a** to
do) and copy files from 4 different locations to various subfolders of a
single directory on S3 (the final result directory).

 

I'd love to create 1 S3DistCp job to do all of that work, but
S3DistCp/DistCp don't support specifying multiple source/destinations.