[jira] [Created] (RATIS-159) Deadlock between AppendStreamer and SegmentedRaftLog eviction

["Mukul Kumar Singh (JIRA)" <ji...@apache.org>]

Mukul Kumar Singh created RATIS-159:
---------------------------------------

             Summary: Deadlock between AppendStreamer and SegmentedRaftLog eviction
                 Key: RATIS-159
                 URL: https://issues.apache.org/jira/browse/RATIS-159
             Project: Ratis
          Issue Type: Bug
            Reporter: Mukul Kumar Singh
            Assignee: Mukul Kumar Singh


Found the following deadlock while testing ratis with Ozone. I have also attached the stack trace file along with the jira.


Java stack information for the threads listed above:
===================================================
{code}
"Thread-195":
        at org.apache.ratis.server.impl.RaftServerImpl.getFollowerNextIndices(RaftServerImpl.java:946)
        - waiting to lock <0x0000000081bbca08> (a org.apache.ratis.server.impl.RaftServerImpl)
        at org.apache.ratis.server.storage.SegmentedRaftLog.checkAndEvictCache(SegmentedRaftLog.java:199)
        at org.apache.ratis.server.storage.SegmentedRaftLog.get(SegmentedRaftLog.java:190)
        at org.apache.ratis.server.impl.LogAppender.createRequest(LogAppender.java:170)
        at org.apache.ratis.grpc.server.GRpcLogAppender.appendLog(GRpcLogAppender.java:138)
        - locked <0x00000000ad0101f8> (a org.apache.ratis.grpc.server.GRpcLogAppender)
        at org.apache.ratis.grpc.server.GRpcLogAppender.run(GRpcLogAppender.java:85)
{code}

{code}
"grpc-default-executor-32":
        at org.apache.ratis.server.impl.LogAppender.notifyAppend(LogAppender.java:456)
        - waiting to lock <0x00000000ad0101f8> (a org.apache.ratis.grpc.server.GRpcLogAppender)
        at org.apache.ratis.server.impl.LeaderState$$Lambda$124/1715853171.accept(Unknown Source)
        at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:890)
        at org.apache.ratis.server.impl.LeaderState$SenderList.forEach(LeaderState.java:91)
        at org.apache.ratis.server.impl.LeaderState.notifySenders(LeaderState.java:190)
        at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:438)
        - locked <0x0000000081bbca08> (a org.apache.ratis.server.impl.RaftServerImpl)
        at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:481)
        at org.apache.ratis.server.impl.RaftServerProxy.submitClientRequestAsync(RaftServerProxy.java:137)
        at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver.onNext(RaftClientProtocolService.java:113)
        at org.apache.ratis.grpc.client.RaftClientProtocolService$AppendRequestStreamObserver.onNext(RaftClientProtocolService.java:96)
        at org.apache.ratis.shaded.io.grpc.stub.ServerCalls$2$1.onMessage(ServerCalls.java:206)
        at org.apache.ratis.shaded.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messageRead(ServerCallImpl.java:237)
        at org.apache.ratis.shaded.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1.runInContext(ServerImpl.java:485)
        at org.apache.ratis.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
        at org.apache.ratis.shaded.io.grpc.internal.SerializingExecutor$TaskRunner.run(SerializingExecutor.java:154)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)