svn commit: r1181404 - in /hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master: HMaster.java RegionManager.java

[ns...@apache.org]

Author: nspiegelberg
Date: Tue Oct 11 02:05:42 2011
New Revision: 1181404

URL: http://svn.apache.org/viewvc?rev=1181404&view=rev
Log:
multiply assigned regions: createTable race condition

Summary:
Details explained in:

https://issues.apache.org/jira/browse/HBASE-2881

This also causes TestAdmin to fail intermittently.

Confirmed in a reproducible manner that the race condition genuinely existed by
putting a sleep of 2 mins prior to setting the region unassigned in master's
in-memory state. By this time, the base scanner assigns out the region, and then
setting the region unassigned in master's state after the sleep causes it to get
"unassigned" once again, and hence assigned out again to a new region server.

The fix basically no longer sets the newly created region unassigned in the
master's in-memory state. Instead, after updating META (with region marked
offline) for all the newly created regions, it simply schedules the meta scanner
to be run immediately. So the regions get assigned without delay as before, but
without the race condition.

Test Plan:
TestAdmin ran successfully a couple of times. Running all unit tests now. Will
run bin/hbase verify as well on a small cluster.

DiffCamp Revision: 170814
Reviewed By: jgray
CC: jgray, kannan, hbase@lists
Tasks:
#279154: HBASE-2881: TestAdmin intermittent failures: Race condition during
createTable can result in region double assignment

Revert Plan:
OK

Modified:
    hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
    hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java

Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java?rev=1181404&r1=1181403&r2=1181404&view=diff
==============================================================================
--- hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java (original)
+++ hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Tue Oct 11 02:05:42 2011
@@ -855,6 +855,8 @@ public class HMaster extends Thread impl
     for(HRegionInfo newRegion : newRegions) {
       regionManager.createRegion(newRegion, srvr, metaRegionName);
     }
+    // kick off a meta scan right away to assign the newly created regions
+    regionManager.metaScannerThread.triggerNow();
   }
 
   public void deleteTable(final byte [] tableName) throws IOException {

Modified: hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java?rev=1181404&r1=1181403&r2=1181404&view=diff
==============================================================================
--- hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java (original)
+++ hbase/branches/0.89/src/main/java/org/apache/hadoop/hbase/master/RegionManager.java Tue Oct 11 02:05:42 2011
@@ -755,8 +755,9 @@ public class RegionManager {
     region.close();
     region.getLog().closeAndDelete();
 
-    // 5. Get it assigned to a server
-    setUnassigned(info, true);
+    // After all regions are created, the caller will schedule
+    // the meta scanner to run immediately and assign out the
+    // regions.
   }
 
   /**