[jira] [Commented] (CURATOR-3) LeaderLatch race condition causing extra nodes to be added in Zookeeper Edit

["Zili Chen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CURATOR-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558880#comment-17558880 ] 

Zili Chen commented on CURATOR-3:
---------------------------------

After some reasoning I think this issue won't happen in 5.2.1.
 # close is synchronized and it will first set state to CLOSE and then setNode(null);
 # reset() is not synchronized and it may create node after close is called.

However,
 # If reset() creates new node, it will check state before getChildren.
 # If state is CLOSE now, then the node will be removed.
 # If state is still START, then later close will call setNode(null) and remove the node.

Another possible race case is about setLeadership, but since it's synchronized, setLeadership(true) later then close() will not notify any listener since they're removed from close().

> LeaderLatch race condition causing extra nodes to be added in Zookeeper Edit
> ----------------------------------------------------------------------------
>
>                 Key: CURATOR-3
>                 URL: https://issues.apache.org/jira/browse/CURATOR-3
>             Project: Apache Curator
>          Issue Type: Bug
>          Components: Recipes
>    Affects Versions: 2.0.0-incubating
>            Reporter: Jordan Zimmerman
>            Priority: Major
>             Fix For: TBD
>
>
> From https://github.com/Netflix/curator/issues/265
> Looks like there's a race condition in LeaderLatch. If LeaderLatch.close() is called at the right time while the latch's watch handler is running, the latch will place another node in Zookeeper after the latch is closed.
> Basically how it happens is this:
> 1) I have two processes contesting a LeaderLatch, ProcessA and ProcessB. ProcessA is leader.
> 2) ProcessA loses leadership somehow (it releases, its connection goes down, etc.)
> 3) This causes ProcessB's watch to get called, check the state is still STARTED, and if so the LeaderLatch will re-evaluate if it is leader.
> 4) While the watch handler is running, close() is called on the LeaderLatch on ProcessB. This sets the LeaderLatch state to CLOSED, removes the znode from ZK and closes off the LeaderLatch.
> 5) The watch handler has already checked that the state is STARTED, so it does a getChildren() on the latch path, and finds the latch's znode is missing. It goes ahead and calls reset(), which places a new znode in Zookeeper.
> Result: The LeaderLatch is closed, but there is still a node in Zookeeper that isn't associated with any LeaderLatch and won't go away until the session goes down. Subsequent LeaderLatches at this path can never get leadership while that session is up.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)