You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by dd...@apache.org on 2021/03/06 20:01:37 UTC
[zookeeper] branch branch-3.7 updated (679cc2b -> 658a700)
This is an automated email from the ASF dual-hosted git repository.
ddiederen pushed a change to branch branch-3.7
in repository https://gitbox.apache.org/repos/asf/zookeeper.git.
from 679cc2b ZOOKEEPER-3781: Create snapshots on followers when snapshot.trust.empty is true
new 5d47a4e ZOOKEEPER-4200: Widen latency window in WatcherCleanerTest
new d28153b ZOOKEEPER-4201: C client: Disable SASL deprecation warnings on macOS
new 658a700 ZOOKEEPER-4199: Avoid thread leak in QuorumRequestPipelineTest
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
zookeeper-client/zookeeper-client-c/src/cli.c | 16 ++++++++++++++++
zookeeper-client/zookeeper-client-c/src/zk_sasl.c | 15 +++++++++++++++
zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h | 1 +
.../zookeeper-client-c/tests/TestSASLAuth.cc | 14 ++++++++++++++
.../server/quorum/QuorumRequestPipelineTest.java | 8 ++++++++
.../zookeeper/server/watch/WatcherCleanerTest.java | 14 +++++++-------
6 files changed, 61 insertions(+), 7 deletions(-)
[zookeeper] 01/03: ZOOKEEPER-4200: Widen latency window in
WatcherCleanerTest
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddiederen pushed a commit to branch branch-3.7
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
commit 5d47a4ea40e4989841738178952025320cd1c30d
Author: Damien Diederen <dd...@crosstwine.com>
AuthorDate: Wed Feb 17 20:18:55 2021 +0100
ZOOKEEPER-4200: Widen latency window in WatcherCleanerTest
`WatcherCleanerTest` performs latency checks which fail when outside of a 20+Xms window. Before this patch, X was 5ms—whereas 30+ms is frequently seen on an i5 Mac Mini running macOS Catalina.
This "dumb" patch just widens the window to 20ms, which makes it "work on my machine," but could obviously still fail in a loaded environment or VM.
Author: Damien Diederen <dd...@crosstwine.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Mate Szalay-Beko <sy...@apache.org>
Closes #1592 from ztzg/ZOOKEEPER-4200-widen-latency-window
---
.../apache/zookeeper/server/watch/WatcherCleanerTest.java | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/watch/WatcherCleanerTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/watch/WatcherCleanerTest.java
index ec05f14..9ed145a 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/watch/WatcherCleanerTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/watch/WatcherCleanerTest.java
@@ -162,13 +162,13 @@ public class WatcherCleanerTest extends ZKTestCase {
assertEquals(3L, values.get("cnt_dead_watchers_cleaner_latency"));
- //Each latency should be a little over 20 ms, allow 5 ms deviation
- assertEquals(20D, (Double) values.get("avg_dead_watchers_cleaner_latency"), 5);
- assertEquals(20D, ((Long) values.get("min_dead_watchers_cleaner_latency")).doubleValue(), 5);
- assertEquals(20D, ((Long) values.get("max_dead_watchers_cleaner_latency")).doubleValue(), 5);
- assertEquals(20D, ((Long) values.get("p50_dead_watchers_cleaner_latency")).doubleValue(), 5);
- assertEquals(20D, ((Long) values.get("p95_dead_watchers_cleaner_latency")).doubleValue(), 5);
- assertEquals(20D, ((Long) values.get("p99_dead_watchers_cleaner_latency")).doubleValue(), 5);
+ //Each latency should be a little over 20 ms, allow 20 ms deviation
+ assertEquals(20D, (Double) values.get("avg_dead_watchers_cleaner_latency"), 20);
+ assertEquals(20D, ((Long) values.get("min_dead_watchers_cleaner_latency")).doubleValue(), 20);
+ assertEquals(20D, ((Long) values.get("max_dead_watchers_cleaner_latency")).doubleValue(), 20);
+ assertEquals(20D, ((Long) values.get("p50_dead_watchers_cleaner_latency")).doubleValue(), 20);
+ assertEquals(20D, ((Long) values.get("p95_dead_watchers_cleaner_latency")).doubleValue(), 20);
+ assertEquals(20D, ((Long) values.get("p99_dead_watchers_cleaner_latency")).doubleValue(), 20);
}
}
[zookeeper] 02/03: ZOOKEEPER-4201: C client: Disable SASL
deprecation warnings on macOS
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddiederen pushed a commit to branch branch-3.7
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
commit d28153b38799c4516681da7941c50fb94f99935e
Author: Damien Diederen <dd...@crosstwine.com>
AuthorDate: Wed Feb 17 20:20:38 2021 +0100
ZOOKEEPER-4201: C client: Disable SASL deprecation warnings on macOS
This patch works around the numerous deprecation notices added to the CyrusSASL library on macOS. It is a direct "port" of the solution to MESOS-3030, which hit exactly the same problem:
https://issues.apache.org/jira/browse/MESOS-3030
https://reviews.apache.org/r/39230/diff/3/
The PR also includes a fix for the the `clockid_t` compilation issue mentioned in the ticket description, but the test suite as a whole remains broken on macOS as its linker does not support the `--wrap` option.
Author: Damien Diederen <dd...@crosstwine.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Mate Szalay-Beko <sy...@apache.org>
Closes #1593 from ztzg/ZOOKEEPER-4201-catalina-c-client-fixes
---
zookeeper-client/zookeeper-client-c/src/cli.c | 16 ++++++++++++++++
zookeeper-client/zookeeper-client-c/src/zk_sasl.c | 15 +++++++++++++++
zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h | 1 +
.../zookeeper-client-c/tests/TestSASLAuth.cc | 14 ++++++++++++++
4 files changed, 46 insertions(+)
diff --git a/zookeeper-client/zookeeper-client-c/src/cli.c b/zookeeper-client/zookeeper-client-c/src/cli.c
index 1864e56..823ed72 100644
--- a/zookeeper-client/zookeeper-client-c/src/cli.c
+++ b/zookeeper-client/zookeeper-client-c/src/cli.c
@@ -948,6 +948,17 @@ int main(int argc, char **argv) {
zoo_deterministic_conn_order(1); // enable deterministic order
#ifdef HAVE_CYRUS_SASL_H
+ /*
+ * We need to disable the deprecation warnings as Apple has
+ * decided to deprecate all of CyrusSASL's functions with OS 10.11
+ * (see MESOS-3030, ZOOKEEPER-4201). We are using GCC pragmas also
+ * for covering clang.
+ */
+#ifdef __APPLE__
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+
if (mechlist) {
zoo_sasl_params_t sasl_params = { 0 };
int sr;
@@ -977,6 +988,11 @@ int main(int argc, char **argv) {
return errno;
}
}
+
+#ifdef __APPLE__
+#pragma GCC diagnostic pop
+#endif
+
#endif /* HAVE_CYRUS_SASL_H */
if (!zh) {
diff --git a/zookeeper-client/zookeeper-client-c/src/zk_sasl.c b/zookeeper-client/zookeeper-client-c/src/zk_sasl.c
index e0ccfb3..6ae7e12 100644
--- a/zookeeper-client/zookeeper-client-c/src/zk_sasl.c
+++ b/zookeeper-client/zookeeper-client-c/src/zk_sasl.c
@@ -48,6 +48,17 @@
#include "zookeeper_log.h"
/*
+ * We need to disable the deprecation warnings as Apple has decided to
+ * deprecate all of CyrusSASL's functions with OS 10.11 (see
+ * MESOS-3030, ZOOKEEPER-4201). We are using GCC pragmas also for
+ * covering clang.
+ */
+#ifdef __APPLE__
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+
+/*
* Store a duplicate of src, or NULL, into *target. Returns
* ZSYSTEMERROR if no memory could be allocated, ZOK otherwise.
*/
@@ -539,3 +550,7 @@ sasl_callback_t *zoo_sasl_make_basic_callbacks(const char *user,
return xcallbacks;
}
}
+
+#ifdef __APPLE__
+#pragma GCC diagnostic pop
+#endif
diff --git a/zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h b/zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h
index 1b6f9db..08028ab 100644
--- a/zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h
+++ b/zookeeper-client/zookeeper-client-c/tests/LibCSymTable.h
@@ -26,6 +26,7 @@
#include <dlfcn.h>
#include <cassert>
#include <poll.h>
+#include <time.h>
#include <unistd.h> // needed for _POSIX_MONOTONIC_CLOCK
#ifdef THREADED
diff --git a/zookeeper-client/zookeeper-client-c/tests/TestSASLAuth.cc b/zookeeper-client/zookeeper-client-c/tests/TestSASLAuth.cc
index e6aa4cb..c98d4bf 100644
--- a/zookeeper-client/zookeeper-client-c/tests/TestSASLAuth.cc
+++ b/zookeeper-client/zookeeper-client-c/tests/TestSASLAuth.cc
@@ -130,6 +130,16 @@ public:
}
#ifdef HAVE_CYRUS_SASL_H
+
+ // We need to disable the deprecation warnings as Apple has
+ // decided to deprecate all of CyrusSASL's functions with OS 10.11
+ // (see MESOS-3030, ZOOKEEPER-4201). We are using GCC pragmas also
+ // for covering clang.
+#ifdef __APPLE__
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
+#endif
+
void testClientSASLHelper(const char *hostPorts, const char *path) {
startServer();
@@ -260,6 +270,10 @@ public:
stopServer();
}
+#ifdef __APPLE__
+#pragma GCC diagnostic pop
+#endif
+
#endif /* HAVE_CYRUS_SASL_H */
};
[zookeeper] 03/03: ZOOKEEPER-4199: Avoid thread leak in
QuorumRequestPipelineTest
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddiederen pushed a commit to branch branch-3.7
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
commit 658a700c14f304319ad06462310b00ecbc89f392
Author: Damien Diederen <dd...@crosstwine.com>
AuthorDate: Thu Feb 18 08:40:55 2021 +0000
ZOOKEEPER-4199: Avoid thread leak in QuorumRequestPipelineTest
`QuorumRequestPipelineTest` hosts parameterized tests which explicitly call `QuorumBase.setUp(boolean)`.
This patch overrides the argument-less `QuorumBase.setUp()` with an empty body, as the former is annotated `BeforeEach`-otherwise causing the runtime to start a fresh 5-ensemble before each test.
Without the override, one such extraneous ensemble is created and immediately leaked for each combination of test method + parameters.
The test consequently requires 4000+ simultaneous threads to complete, and while Linux happily handles that load, macOS Catalina's per-process limit of 2048 threads effectively causes the JVM to "crash" or lock up.
The solution is copied verbatim from another parameterized subclass of `QuorumBase`, `EagerACLFilterTest`.
Author: Damien Diederen <dd...@crosstwine.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Mate Szalay-Beko <sy...@apache.org>
Closes #1591 from ztzg/ZOOKEEPER-4199-thread-leak-qrp-test
---
.../apache/zookeeper/server/quorum/QuorumRequestPipelineTest.java | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumRequestPipelineTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumRequestPipelineTest.java
index 8a463c0..0888d6f 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumRequestPipelineTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumRequestPipelineTest.java
@@ -36,6 +36,7 @@ import org.apache.zookeeper.data.Stat;
import org.apache.zookeeper.server.quorum.QuorumPeer.ServerState;
import org.apache.zookeeper.test.QuorumBase;
import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
@@ -59,6 +60,13 @@ public class QuorumRequestPipelineTest extends QuorumBase {
Arguments.of(ServerState.OBSERVING));
}
+ @BeforeEach
+ @Override
+ public void setUp() {
+ //since parameterized test methods need a parameterized setUp method
+ //the inherited method has to be overridden with an empty function body
+ }
+
public void setUp(ServerState serverState) throws Exception {
CountdownWatcher clientWatch = new CountdownWatcher();
super.setUp(true);