You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Otis Gospodnetic <ot...@yahoo.com> on 2008/05/11 20:36:24 UTC

Balancer not balancing 100%?

Hi,

I have 4 identical nodes in a Hadoop cluster (all functioning as DNs).  One of the 4 nodes is a new node that I recently added.  I ran the balancer a few times and it did move some of the blocks from the other 3 nodes to the new node.  However, the 4 nodes are still not 100% balanced (according to the GUI), even though running bin/hadoop balancer says the cluster is balanced:

Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being Moved
The cluster is balanced. Exiting...
Balancing took 666.0 milliseconds


The 3 old DNs are about 60% full (around 24K blocks), which the 1 new DN is only about 50% full (around 21K blocks).  I restarted the NN and re-ran the balancer, bug got the same output: "The cluster is balanced. Exiting..."

Is this a bug or is it somehow possible for a cluster to be balanced, yet have nodes with different number of blocks?

Thanks,
Otis