You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Nilotpal Nandi (JIRA)" <ji...@apache.org> on 2019/02/18 09:02:00 UTC
[jira] [Created] (HDDS-1126) datanode is trying to qausi-close a
container which is already closed
Nilotpal Nandi created HDDS-1126:
------------------------------------
Summary: datanode is trying to qausi-close a container which is already closed
Key: HDDS-1126
URL: https://issues.apache.org/jira/browse/HDDS-1126
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
--------------------
# created 12 datanodes cluster and running workload on all the nodes
# running failure injection/restart on 1 datanode at a time periodically and randomly.
Error seen in ozone.log :
--------------------------
{noformat}
2019-02-18 06:06:32,780 [Datanode State Machine Thread - 0] DEBUG (DatanodeStateMachine.java:176) - Executing cycle Number : 30
2019-02-18 06:06:32,784 [Command processor thread] DEBUG (CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,785 [Datanode State Machine Thread - 0] DEBUG (DatanodeStateMachine.java:176) - Executing cycle Number : 31
2019-02-18 06:06:32,785 [Command processor thread] ERROR (CloseContainerCommandHandler.java:118) - Can't close container #37
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: Cannot quasi close container #37 while in CLOSED state.
at org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.quasiCloseContainer(KeyValueHandler.java:903)
at org.apache.hadoop.ozone.container.ozoneimpl.ContainerController.quasiCloseContainer(ContainerController.java:93)
at org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:110)
at org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:413)
at java.lang.Thread.run(Thread.java:748)
2019-02-18 06:06:32,785 [Command processor thread] DEBUG (CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,788 [Command processor thread] DEBUG (CloseContainerCommandHandler.java:71) - Processing Close Container command.
2019-02-18 06:06:32,788 [Datanode State Machine Thread - 0] DEBUG (DatanodeStateMachine.java:176) - Executing cycle Number : 32
2019-02-18 06:06:34,430 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:36,608 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:38,876 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:41,084 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:43,297 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:45,469 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:47,684 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:49,958 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:52,124 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:54,344 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:56,499 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:06:58,764 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:00,969 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:02,788 [Datanode State Machine Thread - 0] DEBUG (DatanodeStateMachine.java:176) - Executing cycle Number : 33
2019-02-18 06:07:03,240 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
2019-02-18 06:07:05,486 [main] DEBUG (OzoneClientFactory.java:287) - Using org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org