You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Rakesh R (Created) (JIRA)" <ji...@apache.org> on 2011/09/28 08:54:45 UTC
[jira] [Created] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
------------------------------------------------------------------------------------------------------------------------------
Key: ZOOKEEPER-1209
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
Project: ZooKeeper
Issue Type: Bug
Components: recipes
Affects Versions: 3.3.3
Reporter: Rakesh R
Assignee: Rakesh R
*Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
*Problem:*
Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
*Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
*Problem:*
Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Rakesh R (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rakesh R updated ZOOKEEPER-1209:
--------------------------------
Attachment: ZOOKEEPER-1209.patch
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Rakesh R (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116251#comment-13116251 ]
Rakesh R commented on ZOOKEEPER-1209:
-------------------------------------
Soon will upload patch with the below approach. Please let me know if we have any other better solution to apply.
*Approach:*
IMO better the LES f/w can handle these events ('Disconnected', 'SyncConnected' and 'Expiry' ZooKeeper events), rather than be silent. It will help the users be in a safe state instead of be in the same state (ELECTED/READY).
Provide 'EventProcessor-Thread', one thread per LES. This service will execute the events with a time bounded delay. After choosing the first event, the processor will wait for the configured ‘eventDelayTimeout’ and again pick the latest event present in the queue (if exists). Finally the processor will execute the most recent event. This delay is given in order to avoid slight network fluctuations, wait for some grace period say ‘eventDelayTimeout’ default value could be ‘sessionTimeOut/2’.
All the watchevents (‘Disconnected’, ‘SyncConnected’, ‘Expiry’ events ) from the ZooKeeper server and will be given to this processor. It will have the following logic
+Disconnected logic:+
Introduce new state NEUTRAL to represent the disconnection and the clients will see the node has disconnected from the ZooKeeper can be in a safe mode.
1)If the LeaderElectionSupport state is not STOP, dispatch NEUTRAL event to the user. So the user application can act upon it. This will help to go to a safe state rather than in the ELECTED state.
+SyncConnected logic:+
1)Check if my ephemeral node ‘leaderOffer.getnodePath()’ is present in the ZooKeeper or not
2)If Yes, go to determineElectionStatus(). This will decide the state ELECTED/READY.
3)If No, makeOffer() and determineElectionStatus(). This will first create ephemeral node and go to leader determination phase.
+Expiry logic:+
The serving cluster or standalone ZooKeeper has expired this session. This implies, user must create a new client connection (instantiate a new ZooKeeper instance) if you with to access the ensemble.
1) On receival of Expiry, dispatch STOP event to the client. This will notifies the client and they can restart the LeaderElectionSupport with new ZooKeeper client session.
Thanks,
Rakesh
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-1209:
-------------------------------------
Description:
*Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
*Problem:*
Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
*Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
*Problem:*
Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
was:
*Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
*Problem:*
Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
*Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
*Problem:*
Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
Fix Version/s: 3.5.0
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Rakesh R (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125704#comment-13125704 ]
Rakesh R commented on ZOOKEEPER-1209:
-------------------------------------
Hi folks,
I would like to point out one more difficulty in using the existing ‘LeaderElectionSupport' (LES) recipe.
*Ordered execution of the events is not guaranteed*. The f/w has the logical classification of events (for ex: *_START, *_COMPLETE), but it is not ensuring the execution order of these logical events. This makes the usage of the recipe more complex and may result in erroneous client/user implementation.
*Scenario:*
Consider the scenario where the user node is determined as READY state, now the LES f/w first dispatches the READY_START event to the user node and then watcher will get added to the ELECTED node. Say just before dispatching the READY_COMPLETE event, ELECTED has become expired or stopped. So the 'NodeDeleted' event will be notified to the f/w and will go to the leader determination phase by dispatching the DETERMINE_START and DETERMINE_COMPLETE events.
In this case, there is high chance of dispatching the READY_COMPLETE during the leader determination time or even later.
+Following shows the sample output of Ready node events:+
START
OFFER_START
OFFER_COMPLETE
DETERMINE_START
DETERMINE_COMPLETE
READY_START
DETERMINE_START
DETERMINE_COMPLETE
ELECTED_START
READY_COMPLETE
ELECTED_COMPLETE
+*Proposal*+
The patch which I have uploaded contains the *‘LES-EventProcessor’* service and this will help the recipe to ensure the ordered execution of the events.
Thanks,
Rakesh
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-1209:
---------------------------------------
Assignee: Rakesh R
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121026#comment-13121026 ]
jiraposter@reviews.apache.org commented on ZOOKEEPER-1209:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2213/
-----------------------------------------------------------
Review request for zookeeper and Mahadev Konar.
Summary
-------
The attached patch handles Syncconnected, Disconnected, Expired connection events. Tries to make client to neutral after disconnection and stop after session expiration, rather than in Active mode.
This addresses bug ZOOKEEPER-1209.
https://issues.apache.org/jira/browse/ZOOKEEPER-1209
Diffs
-----
./src/recipes/election/test/org/apache/zookeeper/recipes/leader/LeaderElectionAwareTest.java PRE-CREATION
./src/recipes/election/src/java/org/apache/zookeeper/recipes/leader/LeaderElectionSupport.java 1179108
Diff: https://reviews.apache.org/r/2213/diff
Testing
-------
See test program and description on https://issues.apache.org/jira/browse/ZOOKEEPER-1209
Thanks,
Rakesh
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120838#comment-13120838 ]
Hadoop QA commented on ZOOKEEPER-1209:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497779/ZOOKEEPER-1209.patch
against trunk revision 1177432.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/604//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/604//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/604//console
This message is automatically generated.
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rakesh R updated ZOOKEEPER-1209:
--------------------------------
Assignee: (was: Rakesh R)
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1209) LeaderElection recipe doesn't
handle the split-brain issue, n/w disconnection can bring both the client
nodes to be in ELECTED
Posted by "Rakesh R (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121034#comment-13121034 ]
Rakesh R commented on ZOOKEEPER-1209:
-------------------------------------
Hi,
I have attached the initial patch (with test cases) which will make the client node to 'neutral' mode after disconnection and stop after session expiration, rather than in Elected mode.
The attached patch tries to handle Syncconnected, Disconnected, Expired connection events.
Any review help would be appreciated..
Thanks,
Rakesh
> LeaderElection recipe doesn't handle the split-brain issue, n/w disconnection can bring both the client nodes to be in ELECTED
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-1209
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1209
> Project: ZooKeeper
> Issue Type: Bug
> Components: recipes
> Affects Versions: 3.3.3
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: ZOOKEEPER-1209.patch
>
>
> *Case1-* N/w disconnection can bring both the client nodes to be in ELECTED state. Current LeaderElectionSupport(LES) f/w handles only 'NodeDeletion'.
>
> Consider the scenario where ELECTED and READY nodes are running. Say ELECTED node's n/w got failed and is "Disconnected" from ZooKeeper. But it will behave as ELECTED as it is not getting any events from the LeaderElectionSupport(LES) framework.
> After sessiontimeout, node in READY state will be notified by 'NodeDeleted' event and will go to ELECTED state.
> *Problem:*
> Both the node becomes ELECTED and finally the user sees two Master (ELECTED) node and cause inconsistencies.
> *Case2-* Also in this case, Let's say if user has started only one client node and becomes ELECTED. After sometime n/w has disconnected with the ZooKeeper server and the session got expired.
> *Problem:*
> Still the client node will be in the ELECTED state. After sometime if user has started the second client node. Again both the nodes becomes ELECTED.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira