[jira] [Created] (HBASE-27508) Hbase master is not up due to NotServingRegionException: hbase:meta,,1 is not online

["kaushik mandal (Jira)" <ji...@apache.org>]

kaushik mandal created HBASE-27508:
--------------------------------------

             Summary: Hbase master is not up due to NotServingRegionException: hbase:meta,,1 is not online
                 Key: HBASE-27508
                 URL: https://issues.apache.org/jira/browse/HBASE-27508
             Project: HBase
          Issue Type: Bug
    Affects Versions: 2.4.13
         Environment: we are using 

hbase 2.4.13 and hdfs 3.3.0
            Reporter: kaushik mandal


Hbase master is in initializing state and never become ready.

it is not up due to following error

"NotServingRegionException: hbase:meta,,1 is not online on hbase-regionserver-0.hbase-regionserver.default.svc.cluster.local, ..."

when we observe this:

when region server split into multiple region and there is restart of hbase regionserver and master.
{code:java}
INFO  [main] util.HBaseFsck: Validating mapping using HDFS state
Number of live region servers: 1
Number of dead region servers: 2
Master: hbase-master-0.hbase-master.default.svc.cluster.local,....
Number of backup masters: 0
Average load: 0.0
Number of requests: 0 
Number of regions: 0
Number of regions in transition: 0
INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=16, started=4333 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
    at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2830)
    at org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1075)
    at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:384)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:371)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:351)
, details=, see https://s.apache.org/timeout {code}
workaround:

to make it up, we are deleting /hbase/meta-regionserver using "hbase zkcli delete   /hbase/meta-regionserver"

 

is there any way to prevent this to occur by setting some properties in hbase-site.xml 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)