You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2006/11/16 23:56:31 UTC

[Lucene-hadoop Wiki] Update of "FAQ" by KonstantinShvachko

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by KonstantinShvachko:
http://wiki.apache.org/lucene-hadoop/FAQ

------------------------------------------------------------------------------
  
  If you find something that you wish were done better, and know how to fix it, read HowToContribute, and contribute a patch.
  
+ == 5. If I add new data-nodes to the cluster will HDFS move the blocks to the newly added nodes in order to balance disk space utilization between the nodes? ==
+ 
+ No, HDFS will not move blocks to new nodes automatically. However, newly created files will likely have their blocks placed on the new nodes.
+ 
+ There are several ways to rebalance the cluster manually.
+  1. Select a subset of files that take up a good percentage of your disk space; copy them to new locations in HDFS; remove the old copies of the files; rename the new copies to their original names.
+  2. A simpler way, with no interruption of service, is to turn up the replication of files, wait for transfers to stabilize, and then turn the replication back down.
+  3. Yet another way to re-balance blocks is to turn off the data-node, which is full, wait until its blocks are replicated, and then bring it back again. The over-replicated blocks will be randomly removed from different nodes, so you really get them rebalanced not just removed from the current node.
+