You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by jagaran das <ja...@yahoo.co.in> on 2011/08/13 10:00:21 UTC

MR job to copy to hadoop

Hi,

What is the best and fast way to achieve parallel copy to hadoop from an NFS mount?
We have a mount with huge number of files and we need to copy it into hdfs.

Some options:

1. Run copyFromLocal in a multithreaded way
2. Use distcp in an isolated way.
3. Can i write a map only job to do copy?

Regards,
JD