You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org> on 2012/06/28 15:24:44 UTC
[jira] [Comment Edited] (HBASE-6289) ROOT region doesn't get
re-assigned in ServerShutdownHandler if the RS is still working but only
the RS's ZK node expires.
[ https://issues.apache.org/jira/browse/HBASE-6289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403085#comment-13403085 ]
ramkrishna.s.vasudevan edited comment on HBASE-6289 at 6/28/12 1:24 PM:
------------------------------------------------------------------------
@Maryann
Good one Maraynn. Nice catch.
I have one suggestion. Can we just say assignRoot instead of verifying the root location like how we do assignMeta.
{code}
this.services.getAssignmentManager().assignMeta();
{code}
similarly can we say
{code}
if (isCarryingRoot()) { // -ROOT-
LOG.info("Server " + serverName +
" was carrying ROOT. Trying to assign.");
this.services.getAssignmentManager().
regionOffline(HRegionInfo.ROOT_REGIONINFO);
this.services.getAssignmentManager().assignRoot();
}
{code}
Because we are sure that the root is down here. What do you feel?
{Edit}
But am not sure if that verification step was added for some specific reasons.
{Edit}
was (Author: ram_krish):
@Maryann
Good one Maraynn. Nice catch.
I have one suggestion. Can we just say assignRoot instead of verifying the root location like how we do assignMeta.
{code}
this.services.getAssignmentManager().assignMeta();
{code}
similarly can we say
{code}
if (isCarryingRoot()) { // -ROOT-
LOG.info("Server " + serverName +
" was carrying ROOT. Trying to assign.");
this.services.getAssignmentManager().
regionOffline(HRegionInfo.ROOT_REGIONINFO);
this.services.getAssignmentManager().assignRoot();
}
{code}
Because we are sure that the root is down here. What do you feel?
> ROOT region doesn't get re-assigned in ServerShutdownHandler if the RS is still working but only the RS's ZK node expires.
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6289
> URL: https://issues.apache.org/jira/browse/HBASE-6289
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.6, 0.94.0
> Reporter: Maryann Xue
> Assignee: Maryann Xue
> Priority: Critical
> Attachments: HBASE-6289.patch
>
>
> The ROOT RS has some network problem and its ZK node expires first, which kicks off the ServerShutdownHandler. it calls verifyAndAssignRoot() to try to re-assign ROOT. At that time, the RS is actually still working and passes the verifyRootRegionLocation() check, so the ROOT region is skipped from re-assignment.
> private void verifyAndAssignRoot()
> throws InterruptedException, IOException, KeeperException {
> long timeout = this.server.getConfiguration().
> getLong("hbase.catalog.verification.timeout", 1000);
> if (!this.server.getCatalogTracker().verifyRootRegionLocation(timeout)) {
> this.services.getAssignmentManager().assignRoot();
> }
> }
> After a few moments, this RS encounters DFS write problem and decides to abort. The RS then soon gets restarted from commandline, and constantly report:
> 2012-06-27 23:13:08,627 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException; Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,627 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException; Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,628 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException; Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,628 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException; Region is not online: -ROOT-,,0
> 2012-06-27 23:13:08,630 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: NotServingRegionException; Region is not online: -ROOT-,,0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira