You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ratis.apache.org by "Tsz Wo Nicholas Sze (JIRA)" <ji...@apache.org> on 2018/07/09 08:03:00 UTC
[jira] [Updated] (RATIS-263) Deadlock in ratis because of leader
election
[ https://issues.apache.org/jira/browse/RATIS-263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo Nicholas Sze updated RATIS-263:
--------------------------------------
Priority: Critical (was: Major)
Good catch. Let's change the priority to critical.
> Deadlock in ratis because of leader election
> --------------------------------------------
>
> Key: RATIS-263
> URL: https://issues.apache.org/jira/browse/RATIS-263
> Project: Ratis
> Issue Type: Bug
> Affects Versions: 0.2.0-alpha
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Priority: Critical
> Fix For: 0.2.0-alpha
>
>
> There is a deadlock in Ratis between appendEntries and segmented raft log eviction.
> {code}
> "grpc-default-executor-6":
> at org.apache.ratis.server.impl.LogAppender.notifyAppend(LogAppender.java:466)
> - waiting to lock <0x0000000083a1d398> (a org.apache.ratis.grpc.server.GRpcLogAppender)
> at org.apache.ratis.server.impl.LeaderState$$Lambda$222/1001509199.accept(Unknown Source)
> at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:890)
> at org.apache.ratis.server.impl.LeaderState$SenderList.forEach(LeaderState.java:93)
> at org.apache.ratis.server.impl.LeaderState.notifySenders(LeaderState.java:192)
> at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:476)
> - locked <0x000000008084fac8> (a org.apache.ratis.server.impl.RaftServerImpl)
> at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:524)
> at org.apache.ratis.server.impl.RaftServerProxy.submitClientRequestAsync(RaftServerProxy.java:153)
> at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver.processClientRequestAsync(RaftClientProtocolService.java:125)
> at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver$$Lambda$148/1250401784.accept(Unknown Source)
> at org.apache.ratis.util.SlidingWindow$Server.processRequestsFromHead(SlidingWindow.java:351)
> at org.apache.ratis.util.SlidingWindow$Server.receivedRequest(SlidingWindow.java:343)
> - locked <0x00000000c0f7a060> (a org.apache.ratis.util.SlidingWindow$Server)
> at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver.onNext(RaftClientProtocolService.java:145)
> at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver.onNext(RaftClientProtocolService.java:109)
> at org.apache.ratis.shaded.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:248)
> at org.apache.ratis.shaded.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:252)
> at org.apache.ratis.shaded.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:629)
> at org.apache.ratis.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
> at org.apache.ratis.shaded.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Thread evicting Segmented Raft Log
> {code}
> "Thread-72":
> at org.apache.ratis.server.impl.RaftServerImpl.getFollowerNextIndices(RaftServerImpl.java:1032)
> - waiting to lock <0x000000008084fac8> (a org.apache.ratis.server.impl.RaftServerImpl)
> at org.apache.ratis.server.storage.SegmentedRaftLog.checkAndEvictCache(SegmentedRaftLog.java:199)
> at org.apache.ratis.server.storage.SegmentedRaftLog.get(SegmentedRaftLog.java:190)
> at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:178)
> at org.apache.ratis.grpc.server.GRpcLogAppender.appendLog(GRpcLogAppender.java:153)
> - locked <0x0000000083a1d398> (a org.apache.ratis.grpc.server.GRpcLogAppender)
> at org.apache.ratis.grpc.server.GRpcLogAppender.run(GRpcLogAppender.java:93)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)