svn commit: r1293466 - /hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java

[mb...@apache.org]

Author: mbautin
Date: Fri Feb 24 22:47:46 2012
New Revision: 1293466

URL: http://svn.apache.org/viewvc?rev=1293466&view=rev
Log:
[master] Bound the number of non preferred assignment for each
region server

Summary:
This is diff is related to task 934952. Master should assign a bounded number
of non preferred assignment for each region server.
https://our.intern.facebook.com/intern/tasks/?t=934952

When the master starts or failover, it will scan the dfs to compute the
locality mapping. And there 3 time windows for master to assign regions.

Let's set the timer starting at the time when Master successfully initialized
everything, no matter this is a fresh start up time or failover time.

1) In the 1st min, the master will always assign regions to its best locality
region server. If the best locality region server is not online, then the master
will hold its corresponding regions.

2) In later 4 mins, the master will still try to assign regions to its best
locality region server but not holding region any more. Also start to assign
these hold regions to any checked-in region server.

3) Later on, assign region to any checked-in region server. No locality
assignment any more.

The problem today was there were lots of region serves stopped in the first
time window. When the second time window came in, the master had already hold a
large number unassigned regions.

And assume region server A was the first RS to checked-in this time window, the
master was supposed to assign all the best locality regions plus a bounded
number of unassigned regions together to this region server A.

Test Plan: Running all the unit and will test the failover case in titan
migration002 cluster.

Reviewers: kannan, gqchen, kranganathan

Reviewed By: kannan

CC: hbase-eng@lists

Differential Revision: https://phabricator.fb.com/D412069

Task ID: 934952

Modified:
    hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java

Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java?rev=1293466&r1=1293465&r2=1293466&view=diff
==============================================================================
--- hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java (original)
+++ hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Fri Feb 24 22:47:46 2012
@@ -546,6 +546,7 @@ public class RegionManager {
     // lookup hostname of addr if needed
     String hostName = null;
     RegionState rootState = null;
+    int nonPreferredAssignmentCount = 0;
     // Handle if root is unassigned... only assign root if root is offline.
     synchronized (this.regionsInTransition) {
       rootState = regionsInTransition.get(HRegionInfo.ROOT_REGIONINFO
@@ -608,14 +609,21 @@ public class RegionManager {
             if (hostName.startsWith(preferredHost)) {
               LOG.debug("Doing Preferred Region Assignment for : " + name +
                   " to the " + hostName);
+              // add the region to its preferred region server.
+              if (s.isUnassigned()) {
+                regionsToAssign.add(s);
+              }
+              continue;
             } else if (holdRegionForBestRegionserver ||
                 quickStartRegionServerSet.contains(preferredHost)) {
               continue;
             }
           }
         }
-
-        if (s.isUnassigned()) {
+        // Only assign a configured number unassigned region at one time in the
+        // non preferred assignment case.
+        if (s.isUnassigned() &&
+            (nonPreferredAssignmentCount++) < this.maxAssignInOneGo) {
           regionsToAssign.add(s);
         }
       }