You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by Jonathan Gray <jg...@apache.org> on 2010/10/18 22:32:30 UTC

Review Request: HBASE-2819 HBCK repair problems (new master remix)

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1036/
-----------------------------------------------------------

Review request for hbase and stack.

Summary
-------

Migration of latest HBCK patch made against old master by Alex.

I'm not sure this really makes sense anymore on the new master. Dupe assignment and non-assignment in the "traditional" cases should not happen anymore, at least not in the way it used to happen.

This is mostly a straight migration and adds stuff to clear things out of transition on master, close a region on an RS w/o using ZK (already supported just exposed via rpc now), etc...

I don't think it hurts to have this code in there but mostly the detection code should be used for now. Once we trip over assignment issues / hbck fixable issues, we can add fixup to this new stuff.

Can just rip out the repair for now but figured would be simpler just to leave it and continue improving it rather than going back a step.

This addresses bug HBSAE-2819.
http://issues.apache.org/jira/browse/HBSAE-2819

Diffs
-----

trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsck.java 1023961
trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsckRepair.java PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1023961
trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1023961
trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1023961
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1023961

Diff: http://review.cloudera.org/r/1036/diff

Testing
-------

Thanks,

Jonathan

Re: Review Request: HBASE-2819 HBCK repair problems (new master remix)

Posted by st...@duboce.net.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1036/#review1622
-----------------------------------------------------------



trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
<http://review.cloudera.org/r/1036/#comment5479>

    Should we do a notifyall on rit after this remove?  I'm looking at regionOnline.  It does a notify all if the remove actually removed something.  In fact, can we do a little refactoring so we share the first part of regionOnline between regionOnline and here?



trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
<http://review.cloudera.org/r/1036/#comment5480>

    Is this safe?  We need a sync on regions above, but not here?  And if you sync, is there danger of deadlock?



trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<http://review.cloudera.org/r/1036/#comment5474>

    this is an odd check?  why not have the check done inside in clearRegionFromTransition... Its a little racy?


- stack


On 2010-10-20 17:47:11, Jonathan Gray wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1036/
> -----------------------------------------------------------
> 
> (Updated 2010-10-20 17:47:11)
> 
> 
> Review request for hbase and stack.
> 
> 
> Summary
> -------
> 
> Migration of latest HBCK patch made against old master by Alex.
> 
> I'm not sure this really makes sense anymore on the new master.  Dupe assignment and non-assignment in the "traditional" cases should not happen anymore, at least not in the way it used to happen.
> 
> This is mostly a straight migration and adds stuff to clear things out of transition on master, close a region on an RS w/o using ZK (already supported just exposed via rpc now), etc...
> 
> I don't think it hurts to have this code in there but mostly the detection code should be used for now.  Once we trip over assignment issues / hbck fixable issues, we can add fixup to this new stuff.
> 
> Can just rip out the repair for now but figured would be simpler just to leave it and continue improving it rather than going back a step.
> 
> 
> This addresses bug HBSAE-2819.
>     http://issues.apache.org/jira/browse/HBSAE-2819
> 
> 
> Diffs
> -----
> 
>   trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsck.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsckRepair.java PRE-CREATION 
>   trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1025790 
> 
> Diff: http://review.cloudera.org/r/1036/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jonathan
> 
>

Re: Review Request: HBASE-2819 HBCK repair problems (new master remix)

Posted by st...@duboce.net.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1036/#review1729
-----------------------------------------------------------


I tried this patch.  Its doing something odd around root region.  Here is exception I get:

Exception in thread "main" org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for table: -ROOT-, row=-ROOT-,,99999999999999
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:107)
        at org.apache.hadoop.hbase.client.HBaseFsck.getMetaEntries(HBaseFsck.java:674)
        at org.apache.hadoop.hbase.client.HBaseFsck.doWork(HBaseFsck.java:117)
        at org.apache.hadoop.hbase.client.HBaseFsck.main(HBaseFsck.java:904)

Its doing this which seems odd given metaScan looks for the passed table -ROOT- in .META.

    // Scan -ROOT- to pick up META regions
    MetaScanner.metaScan(conf, visitor,
      HConstants.ROOT_TABLE_NAME, HConstants.EMPTY_START_ROW,
      Integer.MAX_VALUE);



- stack


On 2010-10-20 17:47:11, Jonathan Gray wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1036/
> -----------------------------------------------------------
> 
> (Updated 2010-10-20 17:47:11)
> 
> 
> Review request for hbase and stack.
> 
> 
> Summary
> -------
> 
> Migration of latest HBCK patch made against old master by Alex.
> 
> I'm not sure this really makes sense anymore on the new master.  Dupe assignment and non-assignment in the "traditional" cases should not happen anymore, at least not in the way it used to happen.
> 
> This is mostly a straight migration and adds stuff to clear things out of transition on master, close a region on an RS w/o using ZK (already supported just exposed via rpc now), etc...
> 
> I don't think it hurts to have this code in there but mostly the detection code should be used for now.  Once we trip over assignment issues / hbck fixable issues, we can add fixup to this new stuff.
> 
> Can just rip out the repair for now but figured would be simpler just to leave it and continue improving it rather than going back a step.
> 
> 
> This addresses bug HBSAE-2819.
>     http://issues.apache.org/jira/browse/HBSAE-2819
> 
> 
> Diffs
> -----
> 
>   trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsck.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsckRepair.java PRE-CREATION 
>   trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1025790 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1025790 
> 
> Diff: http://review.cloudera.org/r/1036/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jonathan
> 
>

Re: Review Request: HBASE-2819 HBCK repair problems (new master remix)

Posted by Jonathan Gray <jg...@apache.org>.

(Updated 2010-10-20 17:47:11.597267)

Review request for hbase and stack.

Changes
-------

Latest version against trunk.

Summary
-------

Migration of latest HBCK patch made against old master by Alex.

I'm not sure this really makes sense anymore on the new master. Dupe assignment and non-assignment in the "traditional" cases should not happen anymore, at least not in the way it used to happen.

This is mostly a straight migration and adds stuff to clear things out of transition on master, close a region on an RS w/o using ZK (already supported just exposed via rpc now), etc...

I don't think it hurts to have this code in there but mostly the detection code should be used for now. Once we trip over assignment issues / hbck fixable issues, we can add fixup to this new stuff.

Can just rip out the repair for now but figured would be simpler just to leave it and continue improving it rather than going back a step.

This addresses bug HBSAE-2819.
http://issues.apache.org/jira/browse/HBSAE-2819

Diffs (updated)
-----

trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsck.java 1025790
trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseFsckRepair.java PRE-CREATION
trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1025790
trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1025790
trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1025790
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1025790

Diff: http://review.cloudera.org/r/1036/diff

Testing
-------

Thanks,

Jonathan