You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Nilotpal Nandi (JIRA)" <ji...@apache.org> on 2019/02/17 18:57:00 UTC
[jira] [Created] (HDDS-1125) java.lang.InterruptedException seen in
datanode logs
Nilotpal Nandi created HDDS-1125:
------------------------------------
Summary: java.lang.InterruptedException seen in datanode logs
Key: HDDS-1125
URL: https://issues.apache.org/jira/browse/HDDS-1125
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
--------------------
# created 12 datanodes cluster and running workload on all the nodes
exception seen :
-------------------------
{noformat}
2019-02-15 10:16:48,713 ERROR org.apache.ratis.server.impl.LogAppender: 943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3, i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-15 10:16:48,714 ERROR org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(943007c8-4fdd-4926-89e2-2c8c52c05073 -> 8c77b16b-8054-49e3-b669-1ff759cfd271) hit IOException while loading raft log
org.apache.ratis.server.storage.RaftLogIOException: 943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3, i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:440)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
... 6 more
2019-02-15 10:16:48,715 ERROR org.apache.ratis.server.impl.LogAppender: 943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3, i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-15 10:16:48,715 ERROR org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(943007c8-4fdd-4926-89e2-2c8c52c05073 -> a40a7b01-a30b-469c-b373-9fcb20a126ed) hit IOException while loading raft log
org.apache.ratis.server.storage.RaftLogIOException: 943007c8-4fdd-4926-89e2-2c8c52c05073: Failed readStateMachineData for (t:3, i:3084), STATEMACHINELOGENTRY, client-632E77ADA885, cid=6232
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:440)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:347)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
... 6 more
2019-02-15 10:16:48,723 WARN org.apache.ratis.grpc.client.GrpcClientProtocolService: 943007c8-4fdd-4926-89e2-2c8c52c05073-5: onError: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: CANCELLED: cancelled before receiving half close
2019-02-15 10:16:48,724 INFO org.apache.ratis.server.impl.RoleInfo: 943007c8-4fdd-4926-89e2-2c8c52c05073: start FollowerState
2019-02-15 10:16:48,726 INFO org.apache.ratis.server.impl.RaftServerImpl: 943007c8-4fdd-4926-89e2-2c8c52c05073 changes role from FOLLOWER to FOLLOWER at term 4 for recognizeCandidate:8c77b16b-8054-49e3-b669-1ff759cfd271
2019-02-15 10:16:48,726 INFO org.apache.ratis.server.impl.RoleInfo: 943007c8-4fdd-4926-89e2-2c8c52c05073: shutdown FollowerState
2019-02-15 10:16:48,726 INFO org.apache.ratis.server.impl.RoleInfo: 943007c8-4fdd-4926-89e2-2c8c52c05073: start FollowerState
2019-02-15 10:16:48,726 INFO org.apache.ratis.server.impl.FollowerState: 943007c8-4fdd-4926-89e2-2c8c52c05073: FollowerState was interrupted: java.lang.InterruptedException: sleep interrupted
2019-02-15 10:16:48,727 INFO org.apache.ratis.server.impl.RaftServerImpl: 943007c8-4fdd-4926-89e2-2c8c52c05073: change Leader from null to 8c77b16b-8054-49e3-b669-1ff759cfd271 at term 4 for appendEntries, leader elected after 17ms
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org