[jira] [Commented] (RATIS-705) GrpcClientProtocolClient#close throws InterruptedException

["Lokesh Jain (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RATIS-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948584#comment-16948584 ] 

Lokesh Jain commented on RATIS-705:
-----------------------------------

# The issue is fixed by calling TimeoutScheduler.close after channel.awaitTermination is called. This makes sure that the thread is not interrupted before the channel is shutdown.
 # There can be a NPE in TimeoutScheduler#onTaskCompleted - This can happen if this function is called after the scheduler is closed.
 # I have added a unit test to reproduce the issue. Both of these issues should not lead to failure of client writes. The client is able to retry even without the fixes.

> GrpcClientProtocolClient#close throws InterruptedException
> ----------------------------------------------------------
>
>                 Key: RATIS-705
>                 URL: https://issues.apache.org/jira/browse/RATIS-705
>             Project: Ratis
>          Issue Type: Bug
>          Components: gRPC
>            Reporter: Nilotpal Nandi
>            Assignee: Lokesh Jain
>            Priority: Major
>         Attachments: RATIS-705.001.patch
>
>
> GrpcClientProtocolClient#close throws InterruptedException. This happens when GrpcClientProtocolClient#close is called from a TimeoutScheduler thread. GrpcClientProtocolClient#close calls scheduler.close() which interrupts all the timeout scheduler threads including the thread executing the close routine. This leads to InterruptedException when channel.awaitTermination is called.
>  
> {code:java}
> 19/10/09 07:40:33 ERROR client.GrpcClientProtocolClient: Unexpected exception while waiting for channel termination
> java.lang.InterruptedException
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
>         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
>         at org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelImpl.awaitTermination(ManagedChannelImpl.java:763)
>         at org.apache.ratis.thirdparty.io.grpc.internal.ForwardingManagedChannel.awaitTermination(ForwardingManagedChannel.java:57)
>         at org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.awaitTermination(ManagedChannelOrphanWrapper.java:70)
>         at org.apache.ratis.grpc.client.GrpcClientProtocolClient.close(GrpcClientProtocolClient.java:146)
>         at org.apache.ratis.util.PeerProxyMap$PeerAndProxy.lambda$close$1(PeerProxyMap.java:74)
>         at org.apache.ratis.util.LifeCycle.lambda$checkStateAndClose$2(LifeCycle.java:231)
>         at org.apache.ratis.util.LifeCycle.checkStateAndClose(LifeCycle.java:251)
>         at org.apache.ratis.util.LifeCycle.checkStateAndClose(LifeCycle.java:229)
>         at org.apache.ratis.util.PeerProxyMap$PeerAndProxy.close(PeerProxyMap.java:70)
>         at org.apache.ratis.util.PeerProxyMap.resetProxy(PeerProxyMap.java:127)
>         at org.apache.ratis.util.PeerProxyMap.handleException(PeerProxyMap.java:136)
>         at org.apache.ratis.client.impl.RaftClientRpcWithProxy.handleException(RaftClientRpcWithProxy.java:47)
>         at org.apache.ratis.client.impl.RaftClientImpl.handleIOException(RaftClientImpl.java:372)
>         at org.apache.ratis.client.impl.OrderedAsync.lambda$sendRequest$10(OrderedAsync.java:236)
>         at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
>         at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
>         at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>         at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
>         at org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.lambda$timeoutCheck$3(GrpcClientProtocolClient.java:324)
>         at java.util.Optional.ifPresent(Optional.java:159)
>         at org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.handleReplyFuture(GrpcClientProtocolClient.java:329)
>         at org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.timeoutCheck(GrpcClientProtocolClient.java:324)
>         at org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.lambda$onNext$1(GrpcClientProtocolClient.java:318)
>         at org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$0(TimeoutScheduler.java:113)
>         at org.apache.ratis.util.TimeoutScheduler.lambda$onTimeout$1(TimeoutScheduler.java:133)
>         at org.apache.ratis.util.LogUtils.runAndLog(LogUtils.java:50)
>         at org.apache.ratis.util.LogUtils$1.run(LogUtils.java:91)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at java.lang.Thread.run(Thread.java:748)
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)