You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2023/09/20 00:23:00 UTC

[jira] [Created] (HDDS-9326) HBase HMaster unable to start due to IllegalArgumentException: NO_REPLICA_FOUND

Wei-Chiu Chuang created HDDS-9326:
-------------------------------------

             Summary: HBase HMaster unable to start due to IllegalArgumentException: NO_REPLICA_FOUND
                 Key: HDDS-9326
                 URL: https://issues.apache.org/jira/browse/HDDS-9326
             Project: Apache Ozone
          Issue Type: Bug
            Reporter: Wei-Chiu Chuang


Ran ycsb for extended period of time. Restart the cluster.

First of all, I am aware that HMaster doesn't start because Ozone DN takes a long time to replay Ratis transaction logs (due to the redundant metadata in PutBlock) and SCM was in safe mode.

 

But even after SCM gets out of safe mode, HMaster is still failing due to  IllegalArgumentException: NO_REPLICA_FOUND
{noformat}
11:43:34.199 PMERRORHMasterFailed to become active master
java.lang.IllegalArgumentException: NO_REPLICA_FOUND
	at org.apache.hadoop.ozone.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
	at org.apache.hadoop.hdds.scm.XceiverClientManager.acquireClient(XceiverClientManager.java:170)
	at org.apache.hadoop.hdds.scm.XceiverClientManager.acquireClientForReadData(XceiverClientManager.java:163)
	at org.apache.hadoop.hdds.scm.storage.BlockInputStream.acquireClient(BlockInputStream.java:285)
	at org.apache.hadoop.hdds.scm.storage.BlockInputStream.getChunkInfos(BlockInputStream.java:238)
	at org.apache.hadoop.hdds.scm.storage.BlockInputStream.initialize(BlockInputStream.java:146)
	at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readWithStrategy(BlockInputStream.java:308)
	at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:56)
	at org.apache.hadoop.hdds.scm.storage.ByteArrayReader.readFromBlock(ByteArrayReader.java:57)
	at org.apache.hadoop.hdds.scm.storage.MultipartInputStream.readWithStrategy(MultipartInputStream.java:96)
	at org.apache.hadoop.hdds.scm.storage.ExtendedInputStream.read(ExtendedInputStream.java:56)
	at org.apache.hadoop.fs.ozone.OzoneFSInputStream.read(OzoneFSInputStream.java:64)
	at java.io.DataInputStream.readFully(DataInputStream.java:195)
	at java.io.DataInputStream.readFully(DataInputStream.java:169)
	at org.apache.hadoop.hbase.util.FSUtils.getClusterId(FSUtils.java:526)
	at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:267)
	at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:135)
	at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:112)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:814)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2216)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528)
	at java.lang.Thread.run(Thread.java:748) {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org