You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com> on 2015/07/27 14:15:40 UTC

Hdfs put VS webhdfs

Hi,
I'm loading 28 GB file in hadoop hdfs using webhdfs and it takes ~25 mins to load. I tried loading same file using hdfs put and It took ~6 mins. Why there is so much difference in performance?
What is recommended to use? Can somebody explain or direct me to some good link it will be really helpful.
Below us the command I'm using
curl -i --negotiate -u: -X PUT "http://$hostname:$port/webhdfs/v1/$destination_file_location/$source_filename.temp?op=CREATE&overwrite=true"

this will redirect to a datanode address which I use in next step to write the data

Regards,
Chhaya