[flink] branch master updated: [FLINK-28078][tests] Mitigate likelihood to run into test stability issues caused by CURATOR-645

[ma...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/master by this push:
     new 655184cdb08 [FLINK-28078][tests] Mitigate likelihood to run into test stability issues caused by CURATOR-645
655184cdb08 is described below

commit 655184cdb086ac2adec3e743701868f1a55b6129
Author: Matthias Pohl <ma...@aiven.io>
AuthorDate: Mon Aug 29 14:48:38 2022 +0200

    [FLINK-28078][tests] Mitigate likelihood to run into test stability issues caused by CURATOR-645
    
    CURATOR-645 covers a bug in the LeaderLatch implementation that causes a race condition if a child node, participating in the leader election, is removed too fast. This results in a different code branch being executed which triggers a reset of the LeaderLatch instead of re-collecting the children to determine the next leader.
    The issue occurs because LeaderLatch#checkLeadership is not executed transactionally, i.e. retrieving the children and setting up the watcher for the predecessor is not done atomically. This leads to the race condition where a children (the previous leader's node) is removed before setting up the watcher which results in an invalid handling of the situation using reset.
    Adding some sleep here (simulating the leader actually doing something) will reduce the risk of falling into the race condition because it will give the concurrently running LeaderLatch instances more time to set up the watchers properly.
    
    This is only meant as a temporary solution until CURATOR-645 is resolved and the curator dependency on the Flink side is upgraded.
---
 ...eeperMultipleComponentLeaderElectionDriverTest.java | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/leaderelection/ZooKeeperMultipleComponentLeaderElectionDriverTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/leaderelection/ZooKeeperMultipleComponentLeaderElectionDriverTest.java
index 80325506733..822044c4dbf 100644
--- a/flink-runtime/src/test/java/org/apache/flink/runtime/leaderelection/ZooKeeperMultipleComponentLeaderElectionDriverTest.java
+++ b/flink-runtime/src/test/java/org/apache/flink/runtime/leaderelection/ZooKeeperMultipleComponentLeaderElectionDriverTest.java
@@ -262,6 +262,24 @@ class ZooKeeperMultipleComponentLeaderElectionDriverTest {
                                                 ElectionDriver::hasLeadership, Collectors.toSet()));
 
                 assertThat(leaderAndRest.get(true)).hasSize(1);
+
+                // TODO: remove this line after CURATOR-645 is resolved
+                // CURATOR-645 covers a bug in the LeaderLatch implementation that causes a race
+                // condition if a child node, participating in the leader election, is removed too
+                // fast. This results in a different code branch being executed which triggers a
+                // reset of the LeaderLatch instead of re-collecting the children to determine the
+                // next leader.
+                // The issue occurs because LeaderLatch#checkLeadership is not executed
+                // transactionally, i.e. retrieving the children and setting up the watcher for the
+                // predecessor is not done atomically. This leads to the race condition where a
+                // children (the previous leader's node) is removed before setting up the watcher
+                // which results in an invalid handling of the situation using reset.
+                // Adding some sleep here (simulating the leader actually doing something) will
+                // reduce the risk of falling into the race condition because it will give the
+                // concurrently running LeaderLatch instances more time to set up the watchers
+                // properly.
+                Thread.sleep(100);
+
                 Iterables.getOnlyElement(leaderAndRest.get(true)).close();
 
                 electionDrivers = leaderAndRest.get(false);