You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by prasenjit mukherjee <pm...@quattrowireless.com> on 2010/01/23 09:59:15 UTC

Techniques to speedup "hadoop fs -moveFromLocal " command

It is taking ~ 2 minutes for hdfs to do  -removeFromLocal  of a 200 MB file
on a 12 node cluster ( in ec2 ). I haven't touched any of the default
configurations ( like replication-config etc. )

I am using the following command : "hadood fs -removefromLocal /mnt/file1
/mnt/file2 ..... /ip/." BTW, there is another thread which keeps putting
tehse files in that directory.

Is 2 minutes normal for a 200 mb file hdfs put ? If not any way I can speed
it up.

-Prasen