You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Yu Li (JIRA)" <ji...@apache.org> on 2018/07/18 03:41:00 UTC
[jira] [Commented] (HBASE-20907) Fix Intermittent failure on
TestProcedurePriority
[ https://issues.apache.org/jira/browse/HBASE-20907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547336#comment-16547336 ]
Yu Li commented on HBASE-20907:
-------------------------------
More information from UT output:
{noformat}
-------------------------------------------------------------------------------
Test set: org.apache.hadoop.hbase.master.procedure.TestProcedurePriority
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 275.09 s <<< FAILURE! - in org.apache.hadoop.hbase.master.procedure.TestProcedurePriority
org.apache.hadoop.hbase.master.procedure.TestProcedurePriority Time elapsed: 275.09 s <<< ERROR!
java.io.IOException: Shutting down
at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority.setUp(TestProcedurePriority.java:110)
Caused by: java.lang.RuntimeException: Master not initialized after 200000ms seconds
at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority.setUp(TestProcedurePriority.java:110)
Process Thread Dump: Thread dump because: Master not initialized after 200000ms seconds
Thread 5882 (Thread-4003):
State: TIMED_WAITING
Blocked count: 159
Waited count: 270
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:167)
org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1078)
org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:403)
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:94)
org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:63)
org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:226)
org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1136)
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:984)
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2110)
org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:567)
org.apache.hadoop.hbase.master.HMaster$$Lambda$35/397595326.run(Unknown Source)
java.lang.Thread.run(Thread.java:745)
{noformat}
> Fix Intermittent failure on TestProcedurePriority
> -------------------------------------------------
>
> Key: HBASE-20907
> URL: https://issues.apache.org/jira/browse/HBASE-20907
> Project: HBase
> Issue Type: Test
> Reporter: Yu Li
> Assignee: Yu Li
> Priority: Major
>
> From a local UT check against 2.1.0-RC1, HMaster failed to initialize before time out. Checking the test log we could see below message:
> {noformat}
> 2018-07-17 20:06:37,142 DEBUG [Thread-4003] client.RpcRetryingCallerImpl(131): Call exception, tries=6, retries=6, started=4173 ms ago, cancelled=false, msg=java.io.IOException: Inject error
> at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
> at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
> at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:838)
> at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:540)
> at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:614)
> at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:838)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2520)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2460)
> at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> , details=row 'hbase:namespace' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hdpdevm1.et2sqa.tbsite.net,59254,1531829189215, seqNum=-1, exception=java.io.IOException: java.io.IOException: Inject error
> at org.apache.hadoop.hbase.master.procedure.TestProcedurePriority$MyCP.preGetOp(TestProcedurePriority.java:92)
> at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:841)
> ...
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
> at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1078)
> at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:403)
> at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:94)
> {noformat}
> In current test code we will set {{FAIL}} to true w/o checking whether namespace manager is already up, and if not lucky we will run into the above case and get a timeout.
> The fix will be quite straight forward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)