You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Duo Zhang (JIRA)" <ji...@apache.org> on 2019/06/08 06:40:00 UTC

[jira] [Updated] (HBASE-22477) Throwing exception when meta region is not in OPEN state in client registry may crash a master

     [ https://issues.apache.org/jira/browse/HBASE-22477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Duo Zhang updated HBASE-22477:
------------------------------
    Fix Version/s: 2.1.6
                   2.2.1
                   2.3.0
                   3.0.0

> Throwing exception when meta region is not in OPEN state in client registry may crash a master
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-22477
>                 URL: https://issues.apache.org/jira/browse/HBASE-22477
>             Project: HBase
>          Issue Type: Bug
>          Components: Client, master, meta
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>            Priority: Major
>             Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
>         Attachments: HBASE-22477-test.patch
>
>
> {noformat}
> 2019-05-26 17:10:31,195 ERROR [master/asf906:0:becomeActiveMaster] helpers.MarkerIgnoringBase(159): Failed to become active master
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Cannot get the location for replica0 of region for  in hbase:meta
> 	at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:335)
> 	at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
> 	at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58)
> 	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:192)
> 	at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:263)
> 	at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:405)
> 	at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:285)
> 	at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:564)
> 	at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:766)
> 	at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:734)
> 	at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:690)
> 	at org.apache.hadoop.hbase.MetaTableAccessor.fullScanRegions(MetaTableAccessor.java:220)
> 	at org.apache.hadoop.hbase.master.assignment.RegionStateStore.visitMeta(RegionStateStore.java:77)
> 	at org.apache.hadoop.hbase.master.assignment.AssignmentManager.loadMeta(AssignmentManager.java:1294)
> 	at org.apache.hadoop.hbase.master.assignment.AssignmentManager.joinCluster(AssignmentManager.java:1255)
> 	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1100)
> 	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2375)
> 	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:605)
> 	at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Meta region is in state OPENING
> 	at org.apache.hadoop.hbase.client.ZKAsyncRegistry.lambda$getMetaRegionLocation$1(ZKAsyncRegistry.java:162)
> 	at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:70)
> 	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> 	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> 	at org.apache.hadoop.hbase.client.ZKAsyncRegistry.lambda$getAndConvert$0(ZKAsyncRegistry.java:81)
> 	at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:70)
> 	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> 	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> 	at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:174)
> 	at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342)
> 	... 1 more
> {noformat}
> I think the logic is introduced at the time that we do not change the state of meta region on zk. But now, we will change the state of meta region, so maybe we should remove the logic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)