You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by "Nilotpal Nandi (JIRA)" <ji...@apache.org> on 2019/02/11 09:51:00 UTC
[jira] [Created] (RATIS-483) java.util.concurrent.TimeoutException
exceptions seen in datanode logs
Nilotpal Nandi created RATIS-483:
------------------------------------
Summary: java.util.concurrent.TimeoutException exceptions seen in datanode logs
Key: RATIS-483
URL: https://issues.apache.org/jira/browse/RATIS-483
Project: Ratis
Issue Type: Bug
Reporter: Nilotpal Nandi
steps taken :
--------------------
# created 12 datanode cluster.
# started put key operation with size 100GB.
Seeing following exceptions frequently in datanode logs
--------------------------------------------------------------------------------
{noformat}
2019-02-11 09:43:54,759 INFO org.apache.ratis.server.storage.RaftLogWorker: 541f579c-790a-44b0-9989-80415e119cf4-RaftLogWorker: created new log segment /data/disk1/ozone/meta/ratis/d0c1bf83-bfe5-4646-af49-d735ee312392/current/log_inprogress_3286
2019-02-11 09:44:17,443 WARN org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(541f579c-790a-44b0-9989-80415e119cf4 -> 64b7e2c8-4663-42a4-a145-1bafefbccd4c): Failed get (t:1, i:3288), STATEMACHINELOGENTRY, client-3D1F07BD7204, cid=6942 in 11999532ns
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-11 09:44:18,073 INFO org.apache.ratis.server.storage.RaftLogWorker: 541f579c-790a-44b0-9989-80415e119cf4-RaftLogWorker: Rolling segment log-3286_3295 to index:3295{noformat}
{noformat}
2019-02-11 09:42:58,533 INFO org.apache.ratis.server.storage.RaftLogWorker: 541f579c-790a-44b0-9989-80415e119cf4-RaftLogWorker: created new log segment /data/disk1/ozone/meta/ratis/d0c1bf83-bfe5-4646-af49-d735ee312392/current/log_inprogress_3190
2019-02-11 09:43:20,620 WARN org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(541f579c-790a-44b0-9989-80415e119cf4 -> 4e4626c5-f542-410a-a117-96386cd4e3b5): Failed get (t:1, i:3192), STATEMACHINELOGENTRY, client-AA54864AC41C, cid=6742 in 999326ns
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-11 09:43:21,505 INFO org.apache.ratis.server.storage.RaftLogWorker: 541f579c-790a-44b0-9989-80415e119cf4-RaftLogWorker: Rolling segment log-3190_3199 to index:3199{noformat}
{noformat}
2019-02-11 09:45:13,622 INFO org.apache.ratis.server.storage.RaftLogWorker: 6233367a-34c2-495b-ade4-6f7e11f5eb9e-RaftLogWorker: created new log segment /data/disk1/ozone/meta/ratis/390e6f99-716f-4708-8425-c1cdb4ef4cde/current/log_inprogress_3397
2019-02-11 09:45:13,637 WARN org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(6233367a-34c2-495b-ade4-6f7e11f5eb9e -> 4c976a84-5271-474c-b297-ec9119617da9): Failed get (t:1, i:3397), STATEMACHINELOGENTRY, client-311AFF4454D6, cid=7135 in 74999474ns
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748)
2019-02-11 09:45:13,637 WARN org.apache.ratis.server.impl.LogAppender: GrpcLogAppender(6233367a-34c2-495b-ade4-6f7e11f5eb9e -> e7825b0a-b777-49cb-91e9-8a0b230a44ce): Failed get (t:1, i:3397), STATEMACHINELOGENTRY, client-311AFF4454D6, cid=7135 in 74999312ns
java.util.concurrent.TimeoutException
at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.ratis.server.storage.RaftLog$EntryWithData.getEntry(RaftLog.java:433)
at org.apache.ratis.util.DataQueue.pollList(DataQueue.java:133)
at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:171)
at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:152)
at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:96)
at org.apache.ratis.server.impl.LogAppender.runAppender(LogAppender.java:101)
at java.lang.Thread.run(Thread.java:748){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)