You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Mark Payne (Jira)" <ji...@apache.org> on 2021/09/13 20:03:00 UTC
[jira] [Comment Edited] (NIFI-9217) Possible deadlock when node is
disconnected
[ https://issues.apache.org/jira/browse/NIFI-9217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17414567#comment-17414567 ]
Mark Payne edited comment on NIFI-9217 at 9/13/21, 8:02 PM:
------------------------------------------------------------
Probably the easiest change here is to update LeaderElectionManager so that it does not synchronize on {{this}} in order to access {{pollTimes}} but rather synchronizes on `pollTimes` itself.
was (Author: markap14):
Probably the easiest change here is to update LeaderElectionManager so that it does not synchronize on `this` in order to access `pollTimes` but rather synchronizes on `pollTimes` itself.
> Possible deadlock when node is disconnected
> -------------------------------------------
>
> Key: NIFI-9217
> URL: https://issues.apache.org/jira/browse/NIFI-9217
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Critical
>
> When offloading a node, I encountered a deadlock. Grabbing a thread dump shows the following two threads are in a deadlock:
> {code}
> "Disconnect from Cluster" Id=167 BLOCKED on org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@1f9bfb51 ** DEADLOCKED THREAD ** ** MONITOR-DEADLOCKED THREAD **
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.setLeader(CuratorLeaderElectionManager.java:530)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.disable(CuratorLeaderElectionManager.java:497)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.unregister(CuratorLeaderElectionManager.java:182)
> - waiting on org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager@7aa4e8dc
> at org.apache.nifi.controller.FlowController.onClusterDisconnect(FlowController.java:2456)
> at org.apache.nifi.controller.FlowController.setClustered(FlowController.java:2434)
> at org.apache.nifi.controller.StandardFlowService.disconnect(StandardFlowService.java:771)
> at org.apache.nifi.controller.StandardFlowService.handleDisconnectionRequest(StandardFlowService.java:752)
> at org.apache.nifi.controller.StandardFlowService.access$400(StandardFlowService.java:112)
> at org.apache.nifi.controller.StandardFlowService$3.run(StandardFlowService.java:425)
> at java.lang.Thread.run(Thread.java:748)
> Number of Locked Synchronizers: 2
> - java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3d3a1057
> - java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3d3cb514
> "Process Cluster Protocol Request-3" Id=165 BLOCKED on org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager@7aa4e8dc ** DEADLOCKED THREAD ** ** MONITOR-DEADLOCKED THREAD **
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.registerPollTime(CuratorLeaderElectionManager.java:304)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.getLeader(CuratorLeaderElectionManager.java:293)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.verifyLeader(CuratorLeaderElectionManager.java:556)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener.isLeader(CuratorLeaderElectionManager.java:510)
> - waiting on org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@1f9bfb51
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$LeaderRole.isLeader(CuratorLeaderElectionManager.java:451)
> at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.isLeader(CuratorLeaderElectionManager.java:261)
> at org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.isActiveClusterCoordinator(NodeClusterCoordinator.java:823)
> at org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.handleNodeStatusChange(NodeClusterCoordinator.java:1168)
> at org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.handle(NodeClusterCoordinator.java:1097)
> at org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:176)
> at org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:131)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Number of Locked Synchronizers: 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@c4ecdba
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)