You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by eo...@apache.org on 2020/02/16 09:28:39 UTC
[zookeeper] branch release-3.6.0 updated (5bf63d7 -> cbb3881)
This is an automated email from the ASF dual-hosted git repository.
eolivelli pushed a change to branch release-3.6.0
in repository https://gitbox.apache.org/repos/asf/zookeeper.git.
from 5bf63d7 Update release notes for 3.6.0rc2
new 9606343 ZOOKEEPER-3710: [trivial bug] fix compile error in PurgeTxnTest introduced by ZOOKEEPER-3231
new cbb3881 ZOOKEEPER-3720: Fix rolling upgrade failure (invalid protocol version)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/main/resources/markdown/zookeeperAdmin.md | 30 ++++++++++++++++++--
.../zookeeper/server/quorum/QuorumCnxManager.java | 33 ++++++++++++++++------
.../apache/zookeeper/server/quorum/QuorumPeer.java | 24 +++++++++++++++-
.../zookeeper/server/quorum/QuorumPeerConfig.java | 16 +++++++----
.../zookeeper/server/quorum/QuorumPeerMain.java | 1 +
.../org/apache/zookeeper/server/PurgeTxnTest.java | 4 +--
.../zookeeper/server/quorum/CnxManagerTest.java | 10 +++----
.../zookeeper/server/quorum/LearnerTest.java | 8 ++++++
.../quorum/QuorumPeerMainMultiAddressTest.java | 31 ++++++++++++++++++++
.../zookeeper/server/quorum/QuorumSSLTest.java | 3 ++
.../zookeeper/server/quorum/QuorumServerTest.java | 19 +++++++++++++
.../server/quorum/auth/QuorumAuthTestBase.java | 8 ++++++
.../server/quorum/auth/QuorumAuthUpgradeTest.java | 2 ++
.../server/quorum/auth/QuorumDigestAuthTest.java | 2 ++
.../server/quorum/auth/QuorumKerberosAuthTest.java | 2 ++
.../auth/QuorumKerberosHostBasedAuthTest.java | 2 ++
16 files changed, 171 insertions(+), 24 deletions(-)
[zookeeper] 01/02: ZOOKEEPER-3710: [trivial bug] fix compile error
in PurgeTxnTest introduced by ZOOKEEPER-3231
Posted by eo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
eolivelli pushed a commit to branch release-3.6.0
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
commit 96063432e880051c5b94c868432aedd07f82e337
Author: maoling <ma...@sina.com>
AuthorDate: Fri Jan 31 12:01:23 2020 +0100
ZOOKEEPER-3710: [trivial bug] fix compile error in PurgeTxnTest introduced by ZOOKEEPER-3231
- Link to [PR-1079](https://github.com/apache/zookeeper/pull/1079#issuecomment-580275886)
- more details in the [ZOOKEEPER-3710](https://issues.apache.org/jira/browse/ZOOKEEPER-3710)
Author: maoling <ma...@sina.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Norbert Kalmar <nk...@apache.org>
Closes #1239 from maoling/ZOOKEEPER-3710
---
.../src/test/java/org/apache/zookeeper/server/PurgeTxnTest.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/PurgeTxnTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/PurgeTxnTest.java
index 29c1027..84d9076 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/PurgeTxnTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/PurgeTxnTest.java
@@ -597,7 +597,7 @@ public class PurgeTxnTest extends ZKTestCase {
private void makeValidSnapshot(File snapFile) throws IOException {
SnapStream.setStreamMode(SnapStream.StreamMode.CHECKED);
- CheckedOutputStream os = SnapStream.getOutputStream(snapFile);
+ CheckedOutputStream os = SnapStream.getOutputStream(snapFile, true);
OutputArchive oa = BinaryOutputArchive.getArchive(os);
FileHeader header = new FileHeader(FileSnap.SNAP_MAGIC, 2, 1);
header.serialize(oa, "fileheader");
@@ -610,7 +610,7 @@ public class PurgeTxnTest extends ZKTestCase {
private void makeInvalidSnapshot(File snapFile) throws IOException {
SnapStream.setStreamMode(SnapStream.StreamMode.CHECKED);
- OutputStream os = SnapStream.getOutputStream(snapFile);
+ OutputStream os = SnapStream.getOutputStream(snapFile, true);
os.write(1);
os.flush();
os.close();
[zookeeper] 02/02: ZOOKEEPER-3720: Fix rolling upgrade failure
(invalid protocol version)
Posted by eo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
eolivelli pushed a commit to branch release-3.6.0
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
commit cbb3881116d4d2ba2195cc3f0eb8d2020678272a
Author: Mate Szalay-Beko <sz...@gmail.com>
AuthorDate: Sun Feb 16 10:25:14 2020 +0100
ZOOKEEPER-3720: Fix rolling upgrade failure (invalid protocol version)
The multi-address feature introduced in ZOOKEEPER-3188 required
changes in the Quorum protocol as we had to send all addresses in
the connection initiation message to enable the receiving side to
choose a reachable address in case of network failure.
The new code can handle both the old and the new protocol versions to
avoid 'invalid protocol' error e.g. during rolling restarts. However,
the new protocol version still can not be used during rolling upgrade
if the old servers are not supporting this protocol. In this case the
old and the new servers would form two distinct partitions until all
the servers get upgraded. To support rolling upgrades too, we want to
disable the MultiAddress feature by default and use the old protocol.
If the user would like enable the MultiAddress feature on a 3.6.0
cluster, she/he can do it either by 1) starting the cluster from
scratch (without rolling upgrade), or 2) performing a rolling upgrade
without the MultiAddress feature enabled then doing a rolling restart
with a new configuration where the MultiAddress feature is enabled.
During the rolling restart there will be no partitions, as all the
servers in the cluster now will run ZooKeeper version 3.6.0 which
understands now both protocols.
The changes in this patch:
- introducing new config property: multiAddress.enabled, disabled
by default
- updating QuorumCnxManager to be able to use both protocol versions
and to use the old one if MultiAddress is disabled
- failing with ConfigException if the user provides multiple
addresses in the config while having MultiAddress disabled
- updating the existing MultiAddress related tests to enable the
feature first
- add some new tests
- update the documentation
Testing:
- I ran all the unit tests
- Using https://github.com/symat/zk-rolling-upgrade-test
- I tested rolling upgrade from 3.5.6
- I tested rolling restart to enable the MultiAddress feature
- Using https://github.com/symat/zookeeper-docker-test
- I tested the MultiAddress feature by disabling some virtual
interfaces and waiting for the cluster to recover
Author: Mate Szalay-Beko <sz...@gmail.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Norbert Kalmar <nk...@apache.org>, Andor Molnar <an...@apache.org>
Closes #1251 from symat/ZOOKEEPER-3720
(cherry picked from commit 3aa922c5737c9ef0879f290181cb281261c965e0)
Signed-off-by: Enrico Olivelli <eo...@apache.org>
---
.../src/main/resources/markdown/zookeeperAdmin.md | 30 ++++++++++++++++++--
.../zookeeper/server/quorum/QuorumCnxManager.java | 33 ++++++++++++++++------
.../apache/zookeeper/server/quorum/QuorumPeer.java | 24 +++++++++++++++-
.../zookeeper/server/quorum/QuorumPeerConfig.java | 16 +++++++----
.../zookeeper/server/quorum/QuorumPeerMain.java | 1 +
.../zookeeper/server/quorum/CnxManagerTest.java | 10 +++----
.../zookeeper/server/quorum/LearnerTest.java | 8 ++++++
.../quorum/QuorumPeerMainMultiAddressTest.java | 31 ++++++++++++++++++++
.../zookeeper/server/quorum/QuorumSSLTest.java | 3 ++
.../zookeeper/server/quorum/QuorumServerTest.java | 19 +++++++++++++
.../server/quorum/auth/QuorumAuthTestBase.java | 8 ++++++
.../server/quorum/auth/QuorumAuthUpgradeTest.java | 2 ++
.../server/quorum/auth/QuorumDigestAuthTest.java | 2 ++
.../server/quorum/auth/QuorumKerberosAuthTest.java | 2 ++
.../auth/QuorumKerberosHostBasedAuthTest.java | 2 ++
15 files changed, 169 insertions(+), 22 deletions(-)
diff --git a/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md b/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
index cfcc3e5..1c6e130 100644
--- a/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
+++ b/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
@@ -1132,7 +1132,8 @@ of servers -- that is, when deploying clusters of servers.
<a name="id_multi_address"></a>
Since ZooKeeper 3.6.0 it is possible to specify **multiple addresses** for each
ZooKeeper server (see [ZOOKEEPER-3188](https://issues.apache.org/jira/projects/ZOOKEEPER/issues/ZOOKEEPER-3188)).
- This helps to increase availability and adds network level
+ To enable this feature, you must set the *multiAddress.enabled* configuration property
+ to *true*. This helps to increase availability and adds network level
resiliency to ZooKeeper. When multiple physical network interfaces are used
for the servers, ZooKeeper is able to bind on all interfaces and runtime switching
to a working interface in case a network error. The different addresses can be specified
@@ -1141,7 +1142,18 @@ of servers -- that is, when deploying clusters of servers.
server.1=zoo1-net1:2888:3888|zoo1-net2:2889:3889
server.2=zoo2-net1:2888:3888|zoo2-net2:2889:3889
server.3=zoo3-net1:2888:3888|zoo3-net2:2889:3889
-
+
+
+ ###### Note
+ >By enabling this feature, the Quorum protocol (ZooKeeper Server-Server protocol) will change.
+ The users will not notice this and when anyone starts a ZooKeeper cluster with the new config,
+ everything will work normally. However, it's not possible to enable this feature and specify
+ multiple addresses during a rolling upgrade if the old ZooKeeper cluster didn't support the
+ *multiAddress* feature (and the new Quorum protocol). In case if you need this feature but you
+ also need to perform a rolling upgrade from a ZooKeeper cluster older than *3.6.0*, then you
+ first need to do the rolling upgrade without enabling the MultiAddress feature and later make
+ a separate rolling restart with the new configuration where **multiAddress.enabled** is set
+ to **true** and multiple addresses are provided.
* *syncLimit* :
(No Java system property)
@@ -1489,6 +1501,16 @@ and [SASL authentication for ZooKeeper](https://cwiki.apache.org/confluence/disp
(e.g. the zk/myhost@EXAMPLE.COM client principal will be authenticated in ZooKeeper as zk/myhost)
Default: false
+* *multiAddress.enabled* :
+ (Java system property: **zookeeper.multiAddress.enabled**)
+ **New in 3.6.0:**
+ Since ZooKeeper 3.6.0 you can also [specify multiple addresses](#id_multi_address)
+ for each ZooKeeper server instance (this can increase availability when multiple physical
+ network interfaces can be used parallel in the cluster). Setting this parameter to
+ **true** will enable this feature. Please note, that you can not enable this feature
+ during a rolling upgrade if the version of the old ZooKeeper cluster is prior to 3.6.0.
+ The default value is **false**.
+
* *multiAddress.reachabilityCheckTimeoutMs* :
(Java system property: **zookeeper.multiAddress.reachabilityCheckTimeoutMs**)
**New in 3.6.0:**
@@ -1501,7 +1523,8 @@ and [SASL authentication for ZooKeeper](https://cwiki.apache.org/confluence/disp
in parallel for the different addresses, so the timeout you set here is the maximum time will be taken
by checking the reachability of all addresses.
The default value is **1000**.
-
+
+ This parameter has no effect, unless you enable the MultiAddress feature by setting *multiAddress.enabled=true*.
<a name="Experimental+Options%2FFeatures"></a>
@@ -1593,6 +1616,7 @@ the variable does.
Please note, disabling the reachability check will cause the cluster not to be able to reconfigure
itself properly during network problems, so the disabling is advised only during testing.
+ This parameter has no effect, unless you enable the MultiAddress feature by setting *multiAddress.enabled=true*.
<a name="Disabling+data+directory+autocreation"></a>
diff --git a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
index a92fc52..45ffeb8 100644
--- a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
+++ b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
@@ -37,6 +37,8 @@ import java.nio.ByteBuffer;
import java.nio.channels.UnresolvedAddressException;
import java.time.Duration;
import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
import java.util.Collections;
import java.util.Enumeration;
import java.util.HashSet;
@@ -113,9 +115,12 @@ public class QuorumCnxManager {
private AtomicLong observerCounter = new AtomicLong(-1);
/*
- * Protocol identifier used among peers
+ * Protocol identifier used among peers (must be a negative number for backward compatibility reasons)
*/
- public static final long PROTOCOL_VERSION = -65535L;
+ // the following protocol version was sent in every connection initiation message since ZOOKEEPER-107 released in 3.5.0
+ public static final long PROTOCOL_VERSION_V1 = -65536L;
+ // ZOOKEEPER-3188 introduced multiple addresses in the connection initiation message, released in 3.6.0
+ public static final long PROTOCOL_VERSION_V2 = -65535L;
/*
* Max buffer size to be read from the network.
@@ -218,7 +223,7 @@ public class QuorumCnxManager {
public static InitialMessage parse(Long protocolVersion, DataInputStream din) throws InitialMessageException, IOException {
Long sid;
- if (protocolVersion != PROTOCOL_VERSION) {
+ if (protocolVersion != PROTOCOL_VERSION_V1 && protocolVersion != PROTOCOL_VERSION_V2) {
throw new InitialMessageException("Got unrecognized protocol version %s", protocolVersion);
}
@@ -236,6 +241,8 @@ public class QuorumCnxManager {
throw new InitialMessageException("Read only %s bytes out of %s sent by server %s", num_read, remaining, sid);
}
+ // in PROTOCOL_VERSION_V1 we expect to get a single address here represented as a 'host:port' string
+ // in PROTOCOL_VERSION_V2 we expect to get multiple addresses like: 'host1:port1|host2:port2|...'
String[] addressStrings = new String(b).split("\\|");
List<InetSocketAddress> addresses = new ArrayList<>(addressStrings.length);
for (String addr : addressStrings) {
@@ -416,10 +423,20 @@ public class QuorumCnxManager {
// Sending id and challenge
- // represents protocol version (in other words - message type)
- dout.writeLong(PROTOCOL_VERSION);
+ // First sending the protocol version (in other words - message type).
+ // For backward compatibility reasons we stick to the old protocol version, unless the MultiAddress
+ // feature is enabled. During rolling upgrade, we must make sure that all the servers can
+ // understand the protocol version we use to avoid multiple partitions. see ZOOKEEPER-3720
+ long protocolVersion = self.isMultiAddressEnabled() ? PROTOCOL_VERSION_V2 : PROTOCOL_VERSION_V1;
+ dout.writeLong(protocolVersion);
dout.writeLong(self.getId());
- String addr = self.getElectionAddress().getAllAddresses().stream()
+
+ // now we send our election address. For the new protocol version, we can send multiple addresses.
+ Collection<InetSocketAddress> addressesToSend = protocolVersion == PROTOCOL_VERSION_V2
+ ? self.getElectionAddress().getAllAddresses()
+ : Arrays.asList(self.getElectionAddress().getOne());
+
+ String addr = addressesToSend.stream()
.map(NetUtils::formatInetAddr).collect(Collectors.joining("|"));
byte[] addr_bytes = addr.getBytes();
dout.writeInt(addr_bytes.length);
@@ -639,7 +656,7 @@ public class QuorumCnxManager {
synchronized boolean connectOne(long sid, MultipleAddresses electionAddr) {
if (senderWorkerMap.get(sid) != null) {
LOG.debug("There is a connection already for server {}", sid);
- if (electionAddr.size() > 1 && self.isMultiAddressReachabilityCheckEnabled()) {
+ if (self.isMultiAddressEnabled() && electionAddr.size() > 1 && self.isMultiAddressReachabilityCheckEnabled()) {
// since ZOOKEEPER-3188 we can use multiple election addresses to reach a server. It is possible, that the
// one we are using is already dead and we need to clean-up, so when we will create a new connection
// then we will choose an other one, which is actually reachable
@@ -710,7 +727,7 @@ public class QuorumCnxManager {
synchronized void connectOne(long sid) {
if (senderWorkerMap.get(sid) != null) {
LOG.debug("There is a connection already for server {}", sid);
- if (self.isMultiAddressReachabilityCheckEnabled()) {
+ if (self.isMultiAddressEnabled() && self.isMultiAddressReachabilityCheckEnabled()) {
// since ZOOKEEPER-3188 we can use multiple election addresses to reach a server. It is possible, that the
// one we are using is already dead and we need to clean-up, so when we will create a new connection
// then we will choose an other one, which is actually reachable
diff --git a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
index 011cc0a..dbd23e9 100644
--- a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
+++ b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
@@ -161,8 +161,22 @@ public class QuorumPeer extends ZooKeeperThread implements QuorumStats.Provider
this.observerMasterPort = observerMasterPort;
}
- private int multiAddressReachabilityCheckTimeoutMs = (int) MultipleAddresses.DEFAULT_TIMEOUT.toMillis();
+ public static final String CONFIG_KEY_MULTI_ADDRESS_ENABLED = "zookeeper.multiAddress.enabled";
+ public static final String CONFIG_DEFAULT_MULTI_ADDRESS_ENABLED = "false";
+
+ private boolean multiAddressEnabled = true;
+ public boolean isMultiAddressEnabled() {
+ return multiAddressEnabled;
+ }
+
+ public void setMultiAddressEnabled(boolean multiAddressEnabled) {
+ this.multiAddressEnabled = multiAddressEnabled;
+ LOG.info("multiAddress.enabled set to {}", multiAddressEnabled);
+ }
+ public static final String CONFIG_KEY_MULTI_ADDRESS_REACHABILITY_CHECK_TIMEOUT_MS = "zookeeper.multiAddress.reachabilityCheckTimeoutMs";
+
+ private int multiAddressReachabilityCheckTimeoutMs = (int) MultipleAddresses.DEFAULT_TIMEOUT.toMillis();
public int getMultiAddressReachabilityCheckTimeoutMs() {
return multiAddressReachabilityCheckTimeoutMs;
}
@@ -172,6 +186,8 @@ public class QuorumPeer extends ZooKeeperThread implements QuorumStats.Provider
LOG.info("multiAddress.reachabilityCheckTimeoutMs set to {}", multiAddressReachabilityCheckTimeoutMs);
}
+ public static final String CONFIG_KEY_MULTI_ADDRESS_REACHABILITY_CHECK_ENABLED = "zookeeper.multiAddress.reachabilityCheckEnabled";
+
private boolean multiAddressReachabilityCheckEnabled = true;
public boolean isMultiAddressReachabilityCheckEnabled() {
@@ -274,6 +290,12 @@ public class QuorumPeer extends ZooKeeperThread implements QuorumStats.Provider
}
}
+ boolean multiAddressEnabled = Boolean.parseBoolean(
+ System.getProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, QuorumPeer.CONFIG_DEFAULT_MULTI_ADDRESS_ENABLED));
+ if (!multiAddressEnabled && serverAddresses.length > 1) {
+ throw new ConfigException("Multiple address feature is disabled, but multiple addresses were specified for sid " + sid);
+ }
+
for (String serverAddress : serverAddresses) {
String serverParts[] = ConfigUtils.getHostAndPort(serverAddress);
if ((serverClientParts.length > 2) || (serverParts.length < 3)
diff --git a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java
index 102d699..ae73ef3 100644
--- a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java
+++ b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java
@@ -121,13 +121,13 @@ public class QuorumPeerConfig {
protected int quorumCnxnThreadsSize;
// multi address related configs
+ private boolean multiAddressEnabled = Boolean.parseBoolean(
+ System.getProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, QuorumPeer.CONFIG_DEFAULT_MULTI_ADDRESS_ENABLED));
private boolean multiAddressReachabilityCheckEnabled =
- Boolean.parseBoolean(System.getProperty("zookeeper.multiAddress.reachabilityCheckEnabled",
- "true"));
+ Boolean.parseBoolean(System.getProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_REACHABILITY_CHECK_ENABLED, "true"));
private int multiAddressReachabilityCheckTimeoutMs =
- Integer.parseInt(System.getProperty("zookeeper.multiAddress.reachabilityCheckTimeoutMs",
- String.valueOf(MultipleAddresses.DEFAULT_TIMEOUT.toMillis())));
-
+ Integer.parseInt(System.getProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_REACHABILITY_CHECK_TIMEOUT_MS,
+ String.valueOf(MultipleAddresses.DEFAULT_TIMEOUT.toMillis())));
/**
* Minimum snapshot retain count.
@@ -398,6 +398,8 @@ public class QuorumPeerConfig {
} else if (key.startsWith("metricsProvider.")) {
String keyForMetricsProvider = key.substring(16);
metricsProviderConfiguration.put(keyForMetricsProvider, value);
+ } else if (key.equals("multiAddress.enabled")) {
+ multiAddressEnabled = Boolean.parseBoolean(value);
} else if (key.equals("multiAddress.reachabilityCheckTimeoutMs")) {
multiAddressReachabilityCheckTimeoutMs = Integer.parseInt(value);
} else if (key.equals("multiAddress.reachabilityCheckEnabled")) {
@@ -939,6 +941,10 @@ public class QuorumPeerConfig {
return quorumListenOnAllIPs;
}
+ public boolean isMultiAddressEnabled() {
+ return multiAddressEnabled;
+ }
+
public boolean isMultiAddressReachabilityCheckEnabled() {
return multiAddressReachabilityCheckEnabled;
}
diff --git a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerMain.java b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerMain.java
index 04aec05..a6f94ec 100644
--- a/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerMain.java
+++ b/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerMain.java
@@ -204,6 +204,7 @@ public class QuorumPeerMain {
if (config.sslQuorumReloadCertFiles) {
quorumPeer.getX509Util().enableCertFileReloading();
}
+ quorumPeer.setMultiAddressEnabled(config.isMultiAddressEnabled());
quorumPeer.setMultiAddressReachabilityCheckEnabled(config.isMultiAddressReachabilityCheckEnabled());
quorumPeer.setMultiAddressReachabilityCheckTimeoutMs(config.getMultiAddressReachabilityCheckTimeoutMs());
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/CnxManagerTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/CnxManagerTest.java
index 371fc39..269b6a8 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/CnxManagerTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/CnxManagerTest.java
@@ -261,7 +261,7 @@ public class CnxManagerTest extends ZKTestCase {
InetSocketAddress otherAddr = peers.get(2L).electionAddr.getReachableOrOne();
DataOutputStream dout = new DataOutputStream(sc.socket().getOutputStream());
- dout.writeLong(QuorumCnxManager.PROTOCOL_VERSION);
+ dout.writeLong(QuorumCnxManager.PROTOCOL_VERSION_V1);
dout.writeLong(2);
String addr = otherAddr.getHostString() + ":" + otherAddr.getPort();
byte[] addr_bytes = addr.getBytes();
@@ -604,7 +604,7 @@ public class CnxManagerTest extends ZKTestCase {
dout.writeBytes(hostport);
din = new DataInputStream(new ByteArrayInputStream(bos.toByteArray()));
- msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION, din);
+ msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION_V1, din);
fail("long message accepted");
} catch (InitialMessage.InitialMessageException ex) {
}
@@ -620,7 +620,7 @@ public class CnxManagerTest extends ZKTestCase {
dout.writeBytes(hostport);
din = new DataInputStream(new ByteArrayInputStream(bos.toByteArray()));
- msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION, din);
+ msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION_V1, din);
fail("bad hostport accepted");
} catch (InitialMessage.InitialMessageException ex) {
}
@@ -637,7 +637,7 @@ public class CnxManagerTest extends ZKTestCase {
// now parse it
din = new DataInputStream(new ByteArrayInputStream(bos.toByteArray()));
- msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION, din);
+ msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION_V1, din);
assertEquals(Long.valueOf(5), msg.sid);
assertEquals(Arrays.asList(new InetSocketAddress("10.0.0.2", 3888)), msg.electionAddr);
} catch (InitialMessage.InitialMessageException ex) {
@@ -656,7 +656,7 @@ public class CnxManagerTest extends ZKTestCase {
// now parse it
din = new DataInputStream(new ByteArrayInputStream(bos.toByteArray()));
- msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION, din);
+ msg = InitialMessage.parse(QuorumCnxManager.PROTOCOL_VERSION_V2, din);
assertEquals(Long.valueOf(5), msg.sid);
assertEquals(Arrays.asList(new InetSocketAddress("1.1.1.1", 9999),
new InetSocketAddress("2.2.2.2", 8888),
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/LearnerTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/LearnerTest.java
index 63381fc..97ab5a2 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/LearnerTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/LearnerTest.java
@@ -53,6 +53,7 @@ import org.apache.zookeeper.test.TestUtils;
import org.apache.zookeeper.txn.CreateTxn;
import org.apache.zookeeper.txn.TxnHeader;
import org.apache.zookeeper.util.ServiceUtils;
+import org.junit.After;
import org.junit.Test;
public class LearnerTest extends ZKTestCase {
@@ -134,6 +135,11 @@ public class LearnerTest extends ZKTestCase {
}
}
+ @After
+ public void cleanup() {
+ System.clearProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED);
+ }
+
@Test(expected = IOException.class)
public void connectionRetryTimeoutTest() throws Exception {
Learner learner = new TestLearner();
@@ -178,6 +184,7 @@ public class LearnerTest extends ZKTestCase {
@Test
public void shouldTryMultipleAddresses() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
TestLearner learner = new TestLearner();
learner.self = new QuorumPeer();
learner.self.setTickTime(2000);
@@ -205,6 +212,7 @@ public class LearnerTest extends ZKTestCase {
@Test
public void multipleAddressesSomeAreFailing() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
TestLearner learner = new TestLearner();
learner.self = new QuorumPeer();
learner.self.setTickTime(2000);
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumPeerMainMultiAddressTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumPeerMainMultiAddressTest.java
index ae35573..9c2906c 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumPeerMainMultiAddressTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumPeerMainMultiAddressTest.java
@@ -19,6 +19,7 @@
package org.apache.zookeeper.server.quorum;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
@@ -52,6 +53,7 @@ public class QuorumPeerMainMultiAddressTest extends QuorumPeerTestBase {
@Before
public void setUp() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
ClientBase.setupTestEnv();
System.setProperty("zookeeper.DigestAuthenticationProvider.superDigest", "super:D/InIHSb7yEEbrWz8b9l71RjZJU="/* password is 'test'*/);
QuorumPeerConfig.setReconfigEnabled(true);
@@ -63,6 +65,7 @@ public class QuorumPeerMainMultiAddressTest extends QuorumPeerTestBase {
@After
public void tearDown() throws Exception {
super.tearDown();
+ System.clearProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED);
System.clearProperty("zookeeper.jmx.log4j.disable");
}
@@ -230,6 +233,34 @@ public class QuorumPeerMainMultiAddressTest extends QuorumPeerTestBase {
checkIfZooKeeperQuorumWorks(newQuorumConfig);
}
+ @Test
+ public void shouldFailToReconfigWithMultipleAddressesWhenFeatureIsDisabled() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "false");
+
+ // we have three ZK servers, each server has a single quorumPort and single electionPort registered
+ QuorumServerConfigBuilder initialQuorumConfig = new QuorumServerConfigBuilder(hostName, 3, 1);
+
+ // we launch the three servers, each server should use the same initial config
+ launchServers(Arrays.asList(initialQuorumConfig, initialQuorumConfig, initialQuorumConfig));
+
+ checkIfZooKeeperQuorumWorks(initialQuorumConfig);
+
+ // we create a new config where we add a new address to one of the servers with random available ports
+ QuorumServerConfigBuilder newQuorumConfig = new QuorumServerConfigBuilder(initialQuorumConfig)
+ .addNewServerAddress(FIRST_SERVER);
+
+ ZooKeeperAdmin zkAdmin = newZooKeeperAdmin(initialQuorumConfig);
+
+ // initiating a new incremental reconfig by using the updated ports, expecting exceptions here
+ try {
+ ReconfigTest.reconfig(zkAdmin, newQuorumConfig.buildAsStringList(), null, null, -1);
+ fail("Reconfig succeeded with multiple addresses without exception when the MultiAddress feature is disabled");
+ } catch (KeeperException.BadArgumentsException e) {
+ // do nothing, this is what we expected
+ } catch (Exception e) {
+ fail("Reconfig failed in a wrong way. We expected KeeperException.BadArgumentsException.");
+ }
+ }
private void launchServers(List<QuorumServerConfigBuilder> builders) throws IOException, InterruptedException {
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumSSLTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumSSLTest.java
index 09245c5..593edbd 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumSSLTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumSSLTest.java
@@ -430,6 +430,7 @@ public class QuorumSSLTest extends QuorumPeerTestBase {
@After
public void cleanUp() throws Exception {
+ System.clearProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED);
clearSSLSystemProperties();
if (q1 != null) {
q1.shutdown();
@@ -480,6 +481,7 @@ public class QuorumSSLTest extends QuorumPeerTestBase {
@Test
public void testQuorumSSLWithMultipleAddresses() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
quorumConfiguration = generateMultiAddressQuorumConfiguration();
q1 = new MainThread(1, clientPortQp1, quorumConfiguration, SSL_QUORUM_ENABLED);
@@ -598,6 +600,7 @@ public class QuorumSSLTest extends QuorumPeerTestBase {
@Test
public void testHostnameVerificationForInvalidMultiAddressServerConfig() throws Exception {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
quorumConfiguration = generateMultiAddressQuorumConfiguration();
String badhostnameKeystorePath = tmpDir + "/badhost.jks";
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumServerTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumServerTest.java
index c8cbada..6f5155b 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumServerTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/QuorumServerTest.java
@@ -23,6 +23,7 @@ import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.junit.After;
import org.junit.Test;
public class QuorumServerTest extends ZKTestCase {
@@ -31,6 +32,11 @@ public class QuorumServerTest extends ZKTestCase {
private String ipv6n2 = "[2600:0:0:0:0:0:1:0]";
private String ipv4config = "127.0.0.1:1234:1236";
+ @After
+ public void tearDown() {
+ System.clearProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED);
+ }
+
@Test
public void testToString() throws ConfigException {
String provided = ipv4config + ":participant;0.0.0.0:1237";
@@ -88,6 +94,19 @@ public class QuorumServerTest extends ZKTestCase {
new QuorumServer(0, ipv4config + ":participant;[::1:1237");
}
+ @Test(expected = ConfigException.class)
+ public void shouldNotAllowMultipleAddressesWhenMultiAddressFeatureIsDisabled() throws ConfigException {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "false");
+ new QuorumServer(0, "127.0.0.1:1234:1236|127.0.0.1:2234:2236");
+ }
+
+ @Test
+ public void shouldAllowMultipleAddressesWhenMultiAddressFeatureIsEnabled() throws ConfigException {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
+ QuorumServer qs = new QuorumServer(0, "127.0.0.1:1234:1236|127.0.0.1:2234:2236");
+ assertEquals("MultiAddress parse error", "127.0.0.1:1234:1236|127.0.0.1:2234:2236:participant", qs.toString());
+ }
+
@Test
public void testWildcard() throws KeeperException.BadArgumentsException {
String[] addrs = new String[]{"127.0.0.1", "[0:0:0:0:0:0:0:1]", "0.0.0.0", "[::]"};
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthTestBase.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthTestBase.java
index 97650d0..d365c9a 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthTestBase.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthTestBase.java
@@ -29,8 +29,10 @@ import javax.security.auth.login.Configuration;
import org.apache.commons.io.FileUtils;
import org.apache.zookeeper.PortAssignment;
import org.apache.zookeeper.ZKTestCase;
+import org.apache.zookeeper.server.quorum.QuorumPeer;
import org.apache.zookeeper.server.quorum.QuorumPeerTestBase.MainThread;
import org.apache.zookeeper.test.ClientBase;
+import org.junit.After;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -69,6 +71,11 @@ public class QuorumAuthTestBase extends ZKTestCase {
}
}
+ @After
+ public void tearDown() throws Exception {
+ System.clearProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED);
+ }
+
protected String startQuorum(final int serverCount, Map<String, String> authConfigs,
int authServerCount) throws IOException {
return this.startQuorum(serverCount, authConfigs, authServerCount, false);
@@ -76,6 +83,7 @@ public class QuorumAuthTestBase extends ZKTestCase {
protected String startMultiAddressQuorum(final int serverCount, Map<String, String> authConfigs,
int authServerCount) throws IOException {
+ System.setProperty(QuorumPeer.CONFIG_KEY_MULTI_ADDRESS_ENABLED, "true");
return this.startQuorum(serverCount, authConfigs, authServerCount, true);
}
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthUpgradeTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthUpgradeTest.java
index b944221..ecdfaeb 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthUpgradeTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumAuthUpgradeTest.java
@@ -67,8 +67,10 @@ public class QuorumAuthUpgradeTest extends QuorumAuthTestBase {
}
@After
+ @Override
public void tearDown() throws Exception {
shutdownAll();
+ super.tearDown();
}
@AfterClass
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumDigestAuthTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumDigestAuthTest.java
index 7462979..cb28389 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumDigestAuthTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumDigestAuthTest.java
@@ -60,11 +60,13 @@ public class QuorumDigestAuthTest extends QuorumAuthTestBase {
}
@After
+ @Override
public void tearDown() throws Exception {
for (MainThread mainThread : mt) {
mainThread.shutdown();
mainThread.deleteBaseDir();
}
+ super.tearDown();
}
@AfterClass
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosAuthTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosAuthTest.java
index e4534df..28b93c7 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosAuthTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosAuthTest.java
@@ -88,11 +88,13 @@ public class QuorumKerberosAuthTest extends KerberosSecurityTestcase {
}
@After
+ @Override
public void tearDown() throws Exception {
for (MainThread mainThread : mt) {
mainThread.shutdown();
mainThread.deleteBaseDir();
}
+ super.tearDown();
}
@AfterClass
diff --git a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosHostBasedAuthTest.java b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosHostBasedAuthTest.java
index fc15049..c9b93c5 100644
--- a/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosHostBasedAuthTest.java
+++ b/zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/auth/QuorumKerberosHostBasedAuthTest.java
@@ -113,11 +113,13 @@ public class QuorumKerberosHostBasedAuthTest extends KerberosSecurityTestcase {
}
@After
+ @Override
public void tearDown() throws Exception {
for (MainThread mainThread : mt) {
mainThread.shutdown();
mainThread.deleteBaseDir();
}
+ super.tearDown();
}
@AfterClass