You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by st...@duboce.net on 2010/08/03 03:32:57 UTC

Review Request: HBASE-2461 Split doesn't handle IOExceptions when creating new region reference files

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/474/
-----------------------------------------------------------

Review request for hbase.

Summary
-------

Posting this to review board.

Patch that keeps a journal during split transaction. If split fails, call to rollback will restore the parent to original open condition by backing up whatever transaction steps completed.

The transaction spans split checks, closing of parent region and creation of daughters up to the addition of parent offlining to .META. Once the .META. edit has been made, we cannot rollback -- we have to go forward. This means that the basescanner fixup that will add missing daughter regions should the regionserver crash after parent region edit but before its added daughters is still required, in some form at least.

This patch includes a test of the new split code but only run against an HRegion, not in server context. The split code is buried in heart of the regionserver and created on startup. I stared at it for a while and injecting fault was just forbidding. Its like bramble; there are so many spikes in the way of getting your finger down into the running split I ended up passing on it.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
(split): Most of the split code has been moved out to the new SplitTransaction class.
Now this method prepares the split transaction, executes, and if failure does rollback.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(splitLock) Removed. Doesn't seem necessary. Just made close method synchronized.
(SPLITDIR) Moved to new SplitTransaction
Moved cleanup of half-done splits into SplitTransaction. It'll know better how to do this.
Moved split code into SplitTransaction class.

+ M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Made this class implement new OnlineRegions interface

+ A src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java
New Interface that allows you add/remove regions from oline regions. This Interface
adds little. Was just trying to make it so I didn't have to have server context doing
tests but in the end I just passed null for the case of no server context. Could remove
this.

+ A src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
New class that encapsulates all to do w/ splitting "transaction".

+ A src/main/java/org/apache/hadoop/hbase/util/PairOfSameType.java
Minor utility class

+M src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
(loadRegion) Added loading a region

+ M src/test/java/org/apache/hadoop/hbase/io/TestImmutableBytesWritable.java
(testSpecificCompare) Unrelated change

+ M src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Change because of new manner in which splits are run. Added a splitRegions method.

+ A src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
Test of region splitting code in region context. Testing in server context would take
a bunch of work making it so could insert mock instance of SplitTransaction.

This addresses bug HBASE-2461.
http://issues.apache.org/jira/browse/HBASE-2461

Diffs
-----

src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 7589db3
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 6dc41a4
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 6a54736
src/main/java/org/apache/hadoop/hbase/regionserver/OnlineRegions.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/util/PairOfSameType.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 4d09fe9
src/test/java/org/apache/hadoop/hbase/io/TestImmutableBytesWritable.java 43fa6dd
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 98bd3e5
src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java PRE-CREATION

Diff: http://review.cloudera.org/r/474/diff

Testing
-------

Basic unit tests are passing.

Thanks,

stack

Re: Review Request: HBASE-2461 Split doesn't handle IOExceptions when creating new region reference files

Posted by st...@duboce.net.

(Updated 2010-08-03 13:10:44.610359)

Review request for hbase.

Changes
-------

Found minor issue where wasnt' returning if failed rollback.