You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by cho ju il <tj...@kgrid.co.kr> on 2014/09/11 06:17:45 UTC
Balancing is very slow.
hadoop 2.4.1
Balancing is very slow.
$HADOOP_PREFIX/bin/hdfs dfsadmin -setBalancerBandwidth 52428800
It takes long time to move the one block.
2014. 09. 11. 11:38:01 Block begins to move
2014-09-11 11:47:20 Complete block move
#10.2.1.211 netstat, Block begins to move, 10.2.1.210 -->>> 10.2.1.211
2014. 09. 11. 11:38:01
tcp 1110650 0 10.2.1.211:56819 10.2.1.210:40010 ESTABLISHED -
# datanode log, 10.2.1.211
2014-09-11 11:47:09,819 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Copied BP-1770955034-0.0.0.0-1401163460236:blk_1077753386_4013196 to /10.2.1.211:56819
# namenode balancer log
2014-09-11 11:47:20,782 INFO org.apache.hadoop.hdfs.server.balancer.Balancer: Successfully moved blk_1077753386_4013196 with size=134217728 from 10.2.1.204:40010 to 10.2.1.211:40010 through 10.2.1.210:40010
# check network state, File transfer speed using scp, 76.7MB/sdummy.tar 100% 230MB 76.7MB/s 00:03