You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by cho ju il <tj...@kgrid.co.kr> on 2014/09/11 06:17:45 UTC

Balancing is very slow.

hadoop 2.4.1
Balancing is very slow. 
 
$HADOOP_PREFIX/bin/hdfs dfsadmin -setBalancerBandwidth 52428800
 
It takes long time to move the one block.
 
2014. 09. 11. 11:38:01  Block begins to move
2014-09-11  11:47:20  Complete block move
 
 
#10.2.1.211 netstat, Block begins to move, 10.2.1.210 --&gt;&gt;&gt; 10.2.1.211
2014. 09. 11. 11:38:01
tcp   1110650      0 10.2.1.211:56819            10.2.1.210:40010            ESTABLISHED -
 
 
# datanode log, 10.2.1.211 
2014-09-11 11:47:09,819 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Copied BP-1770955034-0.0.0.0-1401163460236:blk_1077753386_4013196 to /10.2.1.211:56819
 
# namenode balancer log
2014-09-11 11:47:20,782 INFO org.apache.hadoop.hdfs.server.balancer.Balancer: Successfully moved blk_1077753386_4013196 with size=134217728 from 10.2.1.204:40010 to 10.2.1.211:40010 through 10.2.1.210:40010

# check network state, File transfer speed using scp,  76.7MB/sdummy.tar   100%  230MB  76.7MB/s   00:03