You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (Resolved) (JIRA)" <ji...@apache.org> on 2011/10/06 06:04:29 UTC
[jira] [Resolved] (HBASE-4416) OpenedRegionHandler running for a
dead assignment will kill the master
[ https://issues.apache.org/jira/browse/HBASE-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-4416.
--------------------------
Resolution: Duplicate
Resolving as duplicate of HBASE-4540 (Ram is working on this issue over there).
> OpenedRegionHandler running for a dead assignment will kill the master
> ----------------------------------------------------------------------
>
> Key: HBASE-4416
> URL: https://issues.apache.org/jira/browse/HBASE-4416
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.4
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.90.5
>
>
> It goes like this:
> - Master balances region R from server A to server B
> - B reports OPENED of R
> - Master queues a OpenedRegionHandler for R on B
> - B dies (sad)
> - Master processes the splits and reassigns R to C
> - C reports OPENED of R
> - Master queues a OpenedRegionHandler for R on C
> - OpenedRegionHandler for R on B is finally processed, but leaves the region in a weird state (log #1)
> - OpenedRegionHandler for R on C is processed, fails when it tries to delete the znode, kills the master (log #2)
> If the master didn't commit seppuku, it would have had a wrong view of the state of that region because it wouldn't link it to the region server that really opened it in the end.
> I'm not sure how I would go fixing this though...
> Log 1:
> {quote}
> 2011-09-15 01:57:47,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: The server is not in online servers, ServerName=sv4r23s44,60020,1316076811761, region=6e22b45f4288ea4d73f612ccf111aea6
> 2011-09-15 01:57:47,430 DEBUG org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Opened region 6e22b45f4288ea4d73f612ccf111aea6 on sv4r23s44,60020,1316076811761
> {quote}
> Log 2:
> {quote}
> 2011-09-15 01:58:10,171 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x231d7b21aba0480 Deleting existing unassigned node for 6e22b45f4288ea4d73f612ccf111aea6 that is in expected state RS_ZK_REGION_OPENED
> 2011-09-15 01:58:10,204 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: master:60000-0x231d7b21aba0480 Unable to get data of znode /hbase/unassigned/6e22b45f4288ea4d73f612ccf111aea6 because node does not exist (not necessarily an error)
> 2011-09-15 01:58:10,204 FATAL org.apache.hadoop.hbase.master.HMaster: Error deleting OPENED node in ZK for transition ZK node (6e22b45f4288ea4d73f612ccf111aea6)
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira