答复: HDFS Balancer - Recommended Threshold Value

[Yeqi <ye...@huawei.com>]

Try to set a higher number of DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY will help. Default is 5.
It's won't help much if u only use bandwidthPerSec and threshold.

-----邮件原件-----
发件人: Senthil Kumar [mailto:senthilec566@gmail.com] 
发送时间: 2016年8月10日 21:28
收件人: hdfs-dev@hadoop.apache.org; senthilec566@apache.org
主题: HDFS Balancer - Recommended Threshold Value

Hi Team ,  We are running big cluster ( 3000 nodes cluster ) , many time we are hitting  Median Increasing to 99.99 % ( 80 % of the DN's ) .  Balancer is running all time in cluster ..But still  median is not coming down i.e <
90 % ..

Here is how i start balancer ?
/apache/hadoop/sbin/start-balancer.sh
-Ddfs.balance.bandwidthPerSec=104857600  *-threshold  30*

What the recommended value for thershold ??  Is there any way to pass param only to move blocks from Over Utilized ( 98-100%) to under utilized ?


Pls advise!




Regards,
Senthil