You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@zookeeper.apache.org by GitBox <gi...@apache.org> on 2021/05/11 15:11:18 UTC
[GitHub] [zookeeper] EagerToLeave opened a new pull request #1695: Comment
EagerToLeave opened a new pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] muse-dev[bot] commented on a change in pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
muse-dev[bot] commented on a change in pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695#discussion_r630278464
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java
##########
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.EOFException;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.text.DateFormat;
+import java.util.Date;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.zookeeper.server.persistence.FileHeader;
+import org.apache.zookeeper.server.persistence.FileTxnLog;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+
+@InterfaceAudience.Public
+public class LogFormatter {
+ private static final Logger LOG = LoggerFactory.getLogger(LogFormatter.class);
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: LogFormatter log_file");
+ System.exit(2);
+ }
+ FileInputStream fis = new FileInputStream(args[0]);
+ BinaryInputArchive logStream = BinaryInputArchive.getArchive(fis);
+ FileHeader fhdr = new FileHeader();
+ fhdr.deserialize(logStream, "fileheader");
+
+ if (fhdr.getMagic() != FileTxnLog.TXNLOG_MAGIC) {
+ System.err.println("Invalid magic number for " + args[0]);
+ System.exit(2);
+ }
+ System.out.println("ZooKeeper Transactional Log File with dbid "
+ + fhdr.getDbid() + " txnlog format version "
+ + fhdr.getVersion());
+
+ int count = 0;
+ while (true) {
+ long crcValue;
+ byte[] bytes;
+ try {
+ crcValue = logStream.readLong("crcvalue");
+
+ bytes = logStream.readBuffer("txnEntry");
+ } catch (EOFException e) {
+ System.out.println("EOF reached after " + count + " txns.");
+ return;
+ }
+ if (bytes.length == 0) {
+ // Since we preallocate, we define EOF to be an
+ // empty transaction
+ System.out.println("EOF reached after " + count + " txns.");
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 52 is not released after line 79.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java
##########
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.EOFException;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.text.DateFormat;
+import java.util.Date;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.zookeeper.server.persistence.FileHeader;
+import org.apache.zookeeper.server.persistence.FileTxnLog;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+
+@InterfaceAudience.Public
+public class LogFormatter {
+ private static final Logger LOG = LoggerFactory.getLogger(LogFormatter.class);
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: LogFormatter log_file");
+ System.exit(2);
+ }
+ FileInputStream fis = new FileInputStream(args[0]);
+ BinaryInputArchive logStream = BinaryInputArchive.getArchive(fis);
+ FileHeader fhdr = new FileHeader();
+ fhdr.deserialize(logStream, "fileheader");
+
+ if (fhdr.getMagic() != FileTxnLog.TXNLOG_MAGIC) {
+ System.err.println("Invalid magic number for " + args[0]);
+ System.exit(2);
+ }
+ System.out.println("ZooKeeper Transactional Log File with dbid "
+ + fhdr.getDbid() + " txnlog format version "
+ + fhdr.getVersion());
+
+ int count = 0;
+ while (true) {
+ long crcValue;
+ byte[] bytes;
+ try {
+ crcValue = logStream.readLong("crcvalue");
+
+ bytes = logStream.readBuffer("txnEntry");
+ } catch (EOFException e) {
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 52 is not released after line 72.
**Note**: potential exception at line 71
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java
##########
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.EOFException;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.text.DateFormat;
+import java.util.Date;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.zookeeper.server.persistence.FileHeader;
+import org.apache.zookeeper.server.persistence.FileTxnLog;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+
+@InterfaceAudience.Public
+public class LogFormatter {
+ private static final Logger LOG = LoggerFactory.getLogger(LogFormatter.class);
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: LogFormatter log_file");
+ System.exit(2);
+ }
+ FileInputStream fis = new FileInputStream(args[0]);
+ BinaryInputArchive logStream = BinaryInputArchive.getArchive(fis);
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.FileInputStream` acquired by call to `FileInputStream(...)` at line 51 is not released after line 52.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
##########
@@ -0,0 +1,128 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+/**
+ * Dump a snapshot file to stdout.
+ */
+@InterfaceAudience.Public
+public class SnapshotFormatter {
+
+ /**
+ * USAGE: SnapshotFormatter snapshot_file
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: SnapshotFormatter snapshot_file");
+ System.exit(2);
+ }
+
+ new SnapshotFormatter().run(args[0]);
+ }
+
+ public void run(String snapshotFileName) throws IOException {
+ InputStream is = new CheckedInputStream(
+ new BufferedInputStream(new FileInputStream(snapshotFileName)),
+ new Adler32());
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long, Integer> sessions = new HashMap<Long, Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 60 is not released after line 67.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
##########
@@ -0,0 +1,128 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+/**
+ * Dump a snapshot file to stdout.
+ */
+@InterfaceAudience.Public
+public class SnapshotFormatter {
+
+ /**
+ * USAGE: SnapshotFormatter snapshot_file
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: SnapshotFormatter snapshot_file");
+ System.exit(2);
+ }
+
+ new SnapshotFormatter().run(args[0]);
+ }
+
+ public void run(String snapshotFileName) throws IOException {
+ InputStream is = new CheckedInputStream(
+ new BufferedInputStream(new FileInputStream(snapshotFileName)),
+ new Adler32());
+ InputArchive ia = BinaryInputArchive.getArchive(is);
Review comment:
*RESOURCE_LEAK:* resource of type `java.util.zip.CheckedInputStream` acquired by call to `new()` at line 57 is not released after line 60.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Leader.java
##########
@@ -0,0 +1,1437 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.IOException;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.net.SocketException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.FinalRequestProcessor;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.RequestProcessor;
+import org.apache.zookeeper.server.ZooKeeperCriticalThread;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class has the control logic for the Leader.
+ */
+public class Leader {
+ private static final Logger LOG = LoggerFactory.getLogger(Leader.class);
+
+ static final private boolean nodelay = System.getProperty("leader.nodelay", "true").equals("true");
+ static {
+ LOG.info("TCP NoDelay set to: " + nodelay);
+ }
+
+ static public class Proposal extends SyncedLearnerTracker {
+ public QuorumPacket packet;
+ public Request request;
+
+ @Override
+ public String toString() {
+ return packet.getType() + ", " + packet.getZxid() + ", " + request;
+ }
+ }
+
+ // Throttle when there are too many concurrent snapshots being sent to observers
+ private static final String MAX_CONCURRENT_SNAPSHOTS = "zookeeper.leader.maxConcurrentSnapshots";
+ private static final int maxConcurrentSnapshots;
+ private static final String MAX_CONCURRENT_SNAPSHOT_TIMEOUT = "zookeeper.leader.maxConcurrentSnapshotTimeout";
+ private static final long maxConcurrentSnapshotTimeout;
+ static {
+ maxConcurrentSnapshots = Integer.getInteger(MAX_CONCURRENT_SNAPSHOTS, 10);
+ LOG.info(MAX_CONCURRENT_SNAPSHOTS + " = " + maxConcurrentSnapshots);
+ maxConcurrentSnapshotTimeout = Long.getLong(MAX_CONCURRENT_SNAPSHOT_TIMEOUT, 5);
+ LOG.info(MAX_CONCURRENT_SNAPSHOT_TIMEOUT + " = " + maxConcurrentSnapshotTimeout);
+ }
+
+ private final LearnerSnapshotThrottler learnerSnapshotThrottler;
+
+ final LeaderZooKeeperServer zk;
+
+ final QuorumPeer self;
+
+ // VisibleForTesting
+ protected boolean quorumFormed = false;
+
+ // the follower acceptor thread
+ volatile LearnerCnxAcceptor cnxAcceptor = null;
+
+ // list of all the followers
+ private final HashSet<LearnerHandler> learners =
+ new HashSet<LearnerHandler>();
+
+ private final BufferStats proposalStats;
+
+ public BufferStats getProposalStats() {
+ return proposalStats;
+ }
+
+ public LearnerSnapshotThrottler createLearnerSnapshotThrottler(
+ int maxConcurrentSnapshots, long maxConcurrentSnapshotTimeout) {
+ return new LearnerSnapshotThrottler(
+ maxConcurrentSnapshots, maxConcurrentSnapshotTimeout);
+ }
+
+ /**
+ * Returns a copy of the current learner snapshot
+ */
+ public List<LearnerHandler> getLearners() {
+ synchronized (learners) {
+ return new ArrayList<LearnerHandler>(learners);
+ }
+ }
+
+ // list of followers that are ready to follow (i.e synced with the leader)
+ private final HashSet<LearnerHandler> forwardingFollowers =
+ new HashSet<LearnerHandler>();
+
+ /**
+ * Returns a copy of the current forwarding follower snapshot
+ */
+ public List<LearnerHandler> getForwardingFollowers() {
+ synchronized (forwardingFollowers) {
+ return new ArrayList<LearnerHandler>(forwardingFollowers);
+ }
+ }
+
+ private void addForwardingFollower(LearnerHandler lh) {
+ synchronized (forwardingFollowers) {
+ forwardingFollowers.add(lh);
+ }
+ }
+
+ private final HashSet<LearnerHandler> observingLearners =
+ new HashSet<LearnerHandler>();
+
+ /**
+ * Returns a copy of the current observer snapshot
+ */
+ public List<LearnerHandler> getObservingLearners() {
+ synchronized (observingLearners) {
+ return new ArrayList<LearnerHandler>(observingLearners);
+ }
+ }
+
+ private void addObserverLearnerHandler(LearnerHandler lh) {
+ synchronized (observingLearners) {
+ observingLearners.add(lh);
+ }
+ }
+
+ // Pending sync requests. Must access under 'this' lock.
+ private final HashMap<Long,List<LearnerSyncRequest>> pendingSyncs =
+ new HashMap<Long,List<LearnerSyncRequest>>();
+
+ synchronized public int getNumPendingSyncs() {
+ return pendingSyncs.size();
+ }
+
+ //Follower counter
+ final AtomicLong followerCounter = new AtomicLong(-1);
+
+ /**
+ * Adds peer to the leader.
+ *
+ * @param learner
+ * instance of learner handle
+ */
+ void addLearnerHandler(LearnerHandler learner) {
+ synchronized (learners) {
+ learners.add(learner);
+ }
+ }
+
+ /**
+ * Remove the learner from the learner list
+ *
+ * @param peer
+ */
+ void removeLearnerHandler(LearnerHandler peer) {
+ synchronized (forwardingFollowers) {
+ forwardingFollowers.remove(peer);
+ }
+ synchronized (learners) {
+ learners.remove(peer);
+ }
+ synchronized (observingLearners) {
+ observingLearners.remove(peer);
+ }
+ }
+
+ boolean isLearnerSynced(LearnerHandler peer){
+ synchronized (forwardingFollowers) {
+ return forwardingFollowers.contains(peer);
+ }
+ }
+
+
+ /**
+ * Returns true if a quorum in qv is connected and synced with the leader
+ * and false otherwise
+ *
+ * @param qv, a QuorumVerifier
+ */
+ public boolean isQuorumSynced(QuorumVerifier qv) {
+ HashSet<Long> ids = new HashSet<Long>();
+ if (qv.getVotingMembers().containsKey(self.getId()))
+ ids.add(self.getId());
+ synchronized (forwardingFollowers) {
+ for (LearnerHandler learnerHandler: forwardingFollowers){
+ if (learnerHandler.synced() && qv.getVotingMembers().containsKey(learnerHandler.getSid())){
+ ids.add(learnerHandler.getSid());
+ }
+ }
+ }
+ return qv.containsQuorum(ids);
+ }
+
+ private final ServerSocket ss;
+
+ Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException {
+ this.self = self;
+ this.proposalStats = new BufferStats();
+ try {
+ if (self.shouldUsePortUnification() || self.isSslQuorum()) {
+ boolean allowInsecureConnection = self.shouldUsePortUnification();
+ if (self.getQuorumListenOnAllIPs()) {
+ ss = new UnifiedServerSocket(self.getX509Util(), allowInsecureConnection, self.getQuorumAddress().getPort());
+ } else {
+ ss = new UnifiedServerSocket(self.getX509Util(), allowInsecureConnection);
+ }
+ } else {
+ if (self.getQuorumListenOnAllIPs()) {
+ ss = new ServerSocket(self.getQuorumAddress().getPort());
+ } else {
+ ss = new ServerSocket();
+ }
+ }
+ ss.setReuseAddress(true);
+ if (!self.getQuorumListenOnAllIPs()) {
+ ss.bind(self.getQuorumAddress());
+ }
+ } catch (BindException e) {
+ if (self.getQuorumListenOnAllIPs()) {
+ LOG.error("Couldn't bind to port " + self.getQuorumAddress().getPort(), e);
+ } else {
+ LOG.error("Couldn't bind to " + self.getQuorumAddress(), e);
+ }
+ throw e;
+ }
+ this.zk = zk;
+ this.learnerSnapshotThrottler = createLearnerSnapshotThrottler(
+ maxConcurrentSnapshots, maxConcurrentSnapshotTimeout);
+ }
+
+ /**
+ * This message is for follower to expect diff
+ */
+ final static int DIFF = 13;
+
+ /**
+ * This is for follower to truncate its logs
+ */
+ final static int TRUNC = 14;
+
+ /**
+ * This is for follower to download the snapshots
+ */
+ final static int SNAP = 15;
+
+ /**
+ * This tells the leader that the connecting peer is actually an observer
+ */
+ final static int OBSERVERINFO = 16;
+
+ /**
+ * This message type is sent by the leader to indicate it's zxid and if
+ * needed, its database.
+ */
+ final static int NEWLEADER = 10;
+
+ /**
+ * This message type is sent by a follower to pass the last zxid. This is here
+ * for backward compatibility purposes.
+ */
+ final static int FOLLOWERINFO = 11;
+
+ /**
+ * This message type is sent by the leader to indicate that the follower is
+ * now uptodate andt can start responding to clients.
+ */
+ final static int UPTODATE = 12;
+
+ /**
+ * This message is the first that a follower receives from the leader.
+ * It has the protocol version and the epoch of the leader.
+ */
+ public static final int LEADERINFO = 17;
+
+ /**
+ * This message is used by the follow to ack a proposed epoch.
+ */
+ public static final int ACKEPOCH = 18;
+
+ /**
+ * This message type is sent to a leader to request and mutation operation.
+ * The payload will consist of a request header followed by a request.
+ */
+ final static int REQUEST = 1;
+
+ /**
+ * This message type is sent by a leader to propose a mutation.
+ */
+ public final static int PROPOSAL = 2;
+
+ /**
+ * This message type is sent by a follower after it has synced a proposal.
+ */
+ final static int ACK = 3;
+
+ /**
+ * This message type is sent by a leader to commit a proposal and cause
+ * followers to start serving the corresponding data.
+ */
+ final static int COMMIT = 4;
+
+ /**
+ * This message type is enchanged between follower and leader (initiated by
+ * follower) to determine liveliness.
+ */
+ final static int PING = 5;
+
+ /**
+ * This message type is to validate a session that should be active.
+ */
+ final static int REVALIDATE = 6;
+
+ /**
+ * This message is a reply to a synchronize command flushing the pipe
+ * between the leader and the follower.
+ */
+ final static int SYNC = 7;
+
+ /**
+ * This message type informs observers of a committed proposal.
+ */
+ final static int INFORM = 8;
+
+ /**
+ * Similar to COMMIT, only for a reconfig operation.
+ */
+ final static int COMMITANDACTIVATE = 9;
+
+ /**
+ * Similar to INFORM, only for a reconfig operation.
+ */
+ final static int INFORMANDACTIVATE = 19;
+
+ final ConcurrentMap<Long, Proposal> outstandingProposals = new ConcurrentHashMap<Long, Proposal>();
+
+ private final ConcurrentLinkedQueue<Proposal> toBeApplied = new ConcurrentLinkedQueue<Proposal>();
+
+ // VisibleForTesting
+ protected final Proposal newLeaderProposal = new Proposal();
+
+ class LearnerCnxAcceptor extends ZooKeeperCriticalThread {
+ private volatile boolean stop = false;
+
+ public LearnerCnxAcceptor() {
+ super("LearnerCnxAcceptor-" + ss.getLocalSocketAddress(), zk
+ .getZooKeeperServerListener());
+ }
+
+ @Override
+ public void run() {
+ try {
+ while (!stop) {
+ Socket s = null;
+ boolean error = false;
+ try {
+ s = ss.accept();
+
+ // start with the initLimit, once the ack is processed
+ // in LearnerHandler switch to the syncLimit
+ s.setSoTimeout(self.tickTime * self.initLimit);
+ s.setTcpNoDelay(nodelay);
+
+ BufferedInputStream is = new BufferedInputStream(
+ s.getInputStream());
+ LearnerHandler fh = new LearnerHandler(s, is, Leader.this);
+ fh.start();
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.BufferedInputStream` acquired by call to `new()` at line 402 is not released after line 404.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Learner.java
##########
@@ -0,0 +1,670 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.ConnectException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.nio.ByteBuffer;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.concurrent.ConcurrentHashMap;
+
+import javax.net.ssl.SSLSocket;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.ServerCnxn;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is the superclass of two of the three main actors in a ZK
+ * ensemble: Followers and Observers. Both Followers and Observers share
+ * a good deal of code which is moved into Peer to avoid duplication.
+ */
+public class Learner {
+ static class PacketInFlight {
+ TxnHeader hdr;
+ Record rec;
+ }
+ QuorumPeer self;
+ LearnerZooKeeperServer zk;
+
+ protected BufferedOutputStream bufferedOutput;
+
+ protected Socket sock;
+
+ /**
+ * Socket getter
+ * @return
+ */
+ public Socket getSocket() {
+ return sock;
+ }
+
+ protected InputArchive leaderIs;
+ protected OutputArchive leaderOs;
+ /** the protocol version of the leader */
+ protected int leaderProtocolVersion = 0x01;
+
+ protected static final Logger LOG = LoggerFactory.getLogger(Learner.class);
+
+ static final private boolean nodelay = System.getProperty("follower.nodelay", "true").equals("true");
+ static {
+ LOG.info("TCP NoDelay set to: " + nodelay);
+ }
+
+ final ConcurrentHashMap<Long, ServerCnxn> pendingRevalidations =
+ new ConcurrentHashMap<Long, ServerCnxn>();
+
+ public int getPendingRevalidationsCount() {
+ return pendingRevalidations.size();
+ }
+
+ /**
+ * validate a session for a client
+ *
+ * @param clientId
+ * the client to be revalidated
+ * @param timeout
+ * the timeout for which the session is valid
+ * @return
+ * @throws IOException
+ */
+ void validateSession(ServerCnxn cnxn, long clientId, int timeout)
+ throws IOException {
+ LOG.info("Revalidating client: 0x" + Long.toHexString(clientId));
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(baos);
+ dos.writeLong(clientId);
+ dos.writeInt(timeout);
+ dos.close();
+ QuorumPacket qp = new QuorumPacket(Leader.REVALIDATE, -1, baos
+ .toByteArray(), null);
+ pendingRevalidations.put(clientId, cnxn);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG,
+ ZooTrace.SESSION_TRACE_MASK,
+ "To validate session 0x"
+ + Long.toHexString(clientId));
+ }
+ writePacket(qp, true);
+ }
+
+ /**
+ * write a packet to the leader
+ *
+ * @param pp
+ * the proposal packet to be sent to the leader
+ * @throws IOException
+ */
+ void writePacket(QuorumPacket pp, boolean flush) throws IOException {
+ synchronized (leaderOs) {
+ if (pp != null) {
+ leaderOs.writeRecord(pp, "packet");
+ }
+ if (flush) {
+ bufferedOutput.flush();
+ }
+ }
+ }
+
+ /**
+ * read a packet from the leader
+ *
+ * @param pp
+ * the packet to be instantiated
+ * @throws IOException
+ */
+ void readPacket(QuorumPacket pp) throws IOException {
+ synchronized (leaderIs) {
+ leaderIs.readRecord(pp, "packet");
+ }
+ if (LOG.isTraceEnabled()) {
+ final long traceMask =
+ (pp.getType() == Leader.PING) ? ZooTrace.SERVER_PING_TRACE_MASK
+ : ZooTrace.SERVER_PACKET_TRACE_MASK;
+
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'i', pp);
+ }
+ }
+
+ /**
+ * send a request packet to the leader
+ *
+ * @param request
+ * the request from the client
+ * @throws IOException
+ */
+ void request(Request request) throws IOException {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream oa = new DataOutputStream(baos);
+ oa.writeLong(request.sessionId);
+ oa.writeInt(request.cxid);
+ oa.writeInt(request.type);
+ if (request.request != null) {
+ request.request.rewind();
+ int len = request.request.remaining();
+ byte b[] = new byte[len];
+ request.request.get(b);
+ request.request.rewind();
+ oa.write(b);
+ }
+ oa.close();
+ QuorumPacket qp = new QuorumPacket(Leader.REQUEST, -1, baos
+ .toByteArray(), request.authInfo);
+ writePacket(qp, true);
+ }
+
+ /**
+ * Returns the address of the node we think is the leader.
+ */
+ protected QuorumServer findLeader() {
+ QuorumServer leaderServer = null;
+ // Find the leader by id
+ Vote current = self.getCurrentVote();
+ for (QuorumServer s : self.getView().values()) {
+ if (s.id == current.getId()) {
+ // Ensure we have the leader's correct IP address before
+ // attempting to connect.
+ s.recreateSocketAddresses();
+ leaderServer = s;
+ break;
+ }
+ }
+ if (leaderServer == null) {
+ LOG.warn("Couldn't find the leader with id = "
+ + current.getId());
+ }
+ return leaderServer;
+ }
+
+ /**
+ * Overridable helper method to return the System.nanoTime().
+ * This method behaves identical to System.nanoTime().
+ */
+ protected long nanoTime() {
+ return System.nanoTime();
+ }
+
+ /**
+ * Overridable helper method to simply call sock.connect(). This can be
+ * overriden in tests to fake connection success/failure for connectToLeader.
+ */
+ protected void sockConnect(Socket sock, InetSocketAddress addr, int timeout)
+ throws IOException {
+ sock.connect(addr, timeout);
+ }
+
+ /**
+ * Establish a connection with the Leader found by findLeader. Retries
+ * until either initLimit time has elapsed or 5 tries have happened.
+ * @param addr - the address of the Leader to connect to.
+ * @throws IOException - if the socket connection fails on the 5th attempt
+ * <li>if there is an authentication failure while connecting to leader</li>
+ * @throws ConnectException
+ * @throws InterruptedException
+ */
+ protected void connectToLeader(InetSocketAddress addr, String hostname)
+ throws IOException, InterruptedException, X509Exception {
+ this.sock = createSocket();
+
+ int initLimitTime = self.tickTime * self.initLimit;
+ int remainingInitLimitTime = initLimitTime;
+ long startNanoTime = nanoTime();
+
+ for (int tries = 0; tries < 5; tries++) {
+ try {
+ // recalculate the init limit time because retries sleep for 1000 milliseconds
+ remainingInitLimitTime = initLimitTime - (int)((nanoTime() - startNanoTime) / 1000000);
+ if (remainingInitLimitTime <= 0) {
+ LOG.error("initLimit exceeded on retries.");
+ throw new IOException("initLimit exceeded on retries.");
+ }
+
+ sockConnect(sock, addr, Math.min(self.tickTime * self.syncLimit, remainingInitLimitTime));
+ if (self.isSslQuorum()) {
+ ((SSLSocket) sock).startHandshake();
+ }
+ sock.setTcpNoDelay(nodelay);
+ break;
+ } catch (IOException e) {
+ remainingInitLimitTime = initLimitTime - (int)((nanoTime() - startNanoTime) / 1000000);
+
+ if (remainingInitLimitTime <= 1000) {
+ LOG.error("Unexpected exception, initLimit exceeded. tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ throw e;
+ } else if (tries >= 4) {
+ LOG.error("Unexpected exception, retries exceeded. tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ throw e;
+ } else {
+ LOG.warn("Unexpected exception, tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ this.sock = createSocket();
+ }
+ }
+ Thread.sleep(1000);
+ }
+
+ self.authLearner.authenticate(sock, hostname);
+
+ leaderIs = BinaryInputArchive.getArchive(new BufferedInputStream(
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.BufferedInputStream` acquired by call to `new()` at line 294 is not released after line 293.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Learner.java
##########
@@ -0,0 +1,670 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.ConnectException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.nio.ByteBuffer;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.concurrent.ConcurrentHashMap;
+
+import javax.net.ssl.SSLSocket;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.ServerCnxn;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is the superclass of two of the three main actors in a ZK
+ * ensemble: Followers and Observers. Both Followers and Observers share
+ * a good deal of code which is moved into Peer to avoid duplication.
+ */
+public class Learner {
+ static class PacketInFlight {
+ TxnHeader hdr;
+ Record rec;
+ }
+ QuorumPeer self;
+ LearnerZooKeeperServer zk;
+
+ protected BufferedOutputStream bufferedOutput;
+
+ protected Socket sock;
+
+ /**
+ * Socket getter
+ * @return
+ */
+ public Socket getSocket() {
+ return sock;
+ }
+
+ protected InputArchive leaderIs;
+ protected OutputArchive leaderOs;
+ /** the protocol version of the leader */
+ protected int leaderProtocolVersion = 0x01;
+
+ protected static final Logger LOG = LoggerFactory.getLogger(Learner.class);
+
+ static final private boolean nodelay = System.getProperty("follower.nodelay", "true").equals("true");
+ static {
+ LOG.info("TCP NoDelay set to: " + nodelay);
+ }
+
+ final ConcurrentHashMap<Long, ServerCnxn> pendingRevalidations =
+ new ConcurrentHashMap<Long, ServerCnxn>();
+
+ public int getPendingRevalidationsCount() {
+ return pendingRevalidations.size();
+ }
+
+ /**
+ * validate a session for a client
+ *
+ * @param clientId
+ * the client to be revalidated
+ * @param timeout
+ * the timeout for which the session is valid
+ * @return
+ * @throws IOException
+ */
+ void validateSession(ServerCnxn cnxn, long clientId, int timeout)
+ throws IOException {
+ LOG.info("Revalidating client: 0x" + Long.toHexString(clientId));
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(baos);
+ dos.writeLong(clientId);
+ dos.writeInt(timeout);
+ dos.close();
+ QuorumPacket qp = new QuorumPacket(Leader.REVALIDATE, -1, baos
+ .toByteArray(), null);
+ pendingRevalidations.put(clientId, cnxn);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG,
+ ZooTrace.SESSION_TRACE_MASK,
+ "To validate session 0x"
+ + Long.toHexString(clientId));
+ }
+ writePacket(qp, true);
+ }
+
+ /**
+ * write a packet to the leader
+ *
+ * @param pp
+ * the proposal packet to be sent to the leader
+ * @throws IOException
+ */
+ void writePacket(QuorumPacket pp, boolean flush) throws IOException {
+ synchronized (leaderOs) {
+ if (pp != null) {
+ leaderOs.writeRecord(pp, "packet");
+ }
+ if (flush) {
+ bufferedOutput.flush();
+ }
+ }
+ }
+
+ /**
+ * read a packet from the leader
+ *
+ * @param pp
+ * the packet to be instantiated
+ * @throws IOException
+ */
+ void readPacket(QuorumPacket pp) throws IOException {
+ synchronized (leaderIs) {
+ leaderIs.readRecord(pp, "packet");
+ }
+ if (LOG.isTraceEnabled()) {
+ final long traceMask =
+ (pp.getType() == Leader.PING) ? ZooTrace.SERVER_PING_TRACE_MASK
+ : ZooTrace.SERVER_PACKET_TRACE_MASK;
+
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'i', pp);
+ }
+ }
+
+ /**
+ * send a request packet to the leader
+ *
+ * @param request
+ * the request from the client
+ * @throws IOException
+ */
+ void request(Request request) throws IOException {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream oa = new DataOutputStream(baos);
+ oa.writeLong(request.sessionId);
+ oa.writeInt(request.cxid);
+ oa.writeInt(request.type);
+ if (request.request != null) {
+ request.request.rewind();
+ int len = request.request.remaining();
+ byte b[] = new byte[len];
+ request.request.get(b);
+ request.request.rewind();
+ oa.write(b);
+ }
+ oa.close();
+ QuorumPacket qp = new QuorumPacket(Leader.REQUEST, -1, baos
+ .toByteArray(), request.authInfo);
+ writePacket(qp, true);
+ }
+
+ /**
+ * Returns the address of the node we think is the leader.
+ */
+ protected QuorumServer findLeader() {
+ QuorumServer leaderServer = null;
+ // Find the leader by id
+ Vote current = self.getCurrentVote();
+ for (QuorumServer s : self.getView().values()) {
+ if (s.id == current.getId()) {
+ // Ensure we have the leader's correct IP address before
+ // attempting to connect.
+ s.recreateSocketAddresses();
+ leaderServer = s;
+ break;
+ }
+ }
+ if (leaderServer == null) {
+ LOG.warn("Couldn't find the leader with id = "
+ + current.getId());
+ }
+ return leaderServer;
+ }
+
+ /**
+ * Overridable helper method to return the System.nanoTime().
+ * This method behaves identical to System.nanoTime().
+ */
+ protected long nanoTime() {
+ return System.nanoTime();
+ }
+
+ /**
+ * Overridable helper method to simply call sock.connect(). This can be
+ * overriden in tests to fake connection success/failure for connectToLeader.
+ */
+ protected void sockConnect(Socket sock, InetSocketAddress addr, int timeout)
+ throws IOException {
+ sock.connect(addr, timeout);
+ }
+
+ /**
+ * Establish a connection with the Leader found by findLeader. Retries
+ * until either initLimit time has elapsed or 5 tries have happened.
+ * @param addr - the address of the Leader to connect to.
+ * @throws IOException - if the socket connection fails on the 5th attempt
+ * <li>if there is an authentication failure while connecting to leader</li>
+ * @throws ConnectException
+ * @throws InterruptedException
+ */
+ protected void connectToLeader(InetSocketAddress addr, String hostname)
+ throws IOException, InterruptedException, X509Exception {
+ this.sock = createSocket();
+
+ int initLimitTime = self.tickTime * self.initLimit;
+ int remainingInitLimitTime = initLimitTime;
+ long startNanoTime = nanoTime();
+
+ for (int tries = 0; tries < 5; tries++) {
+ try {
+ // recalculate the init limit time because retries sleep for 1000 milliseconds
+ remainingInitLimitTime = initLimitTime - (int)((nanoTime() - startNanoTime) / 1000000);
+ if (remainingInitLimitTime <= 0) {
+ LOG.error("initLimit exceeded on retries.");
+ throw new IOException("initLimit exceeded on retries.");
+ }
+
+ sockConnect(sock, addr, Math.min(self.tickTime * self.syncLimit, remainingInitLimitTime));
+ if (self.isSslQuorum()) {
+ ((SSLSocket) sock).startHandshake();
+ }
+ sock.setTcpNoDelay(nodelay);
+ break;
+ } catch (IOException e) {
+ remainingInitLimitTime = initLimitTime - (int)((nanoTime() - startNanoTime) / 1000000);
+
+ if (remainingInitLimitTime <= 1000) {
+ LOG.error("Unexpected exception, initLimit exceeded. tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ throw e;
+ } else if (tries >= 4) {
+ LOG.error("Unexpected exception, retries exceeded. tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ throw e;
+ } else {
+ LOG.warn("Unexpected exception, tries=" + tries +
+ ", remaining init limit=" + remainingInitLimitTime +
+ ", connecting to " + addr,e);
+ this.sock = createSocket();
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.InputStream` acquired to `Learner.sock` by call to `createSocket()` at line 247 is not released after line 285.
**Note**: potential exception at line 259
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
##########
@@ -0,0 +1,1366 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import static org.apache.zookeeper.common.NetUtils.formatInetAddr;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.net.SocketException;
+import java.net.SocketTimeoutException;
+import java.net.UnknownHostException;
+import java.nio.BufferUnderflowException;
+import java.nio.ByteBuffer;
+import java.nio.channels.UnresolvedAddressException;
+import java.util.Collections;
+import java.util.Enumeration;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ArrayBlockingQueue;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.SynchronousQueue;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.NoSuchElementException;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Supplier;
+import javax.net.ssl.SSLSocket;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.ExitCode;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthServer;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class implements a connection manager for leader election using TCP. It
+ * maintains one connection for every pair of servers. The tricky part is to
+ * guarantee that there is exactly one connection for every pair of servers that
+ * are operating correctly and that can communicate over the network.
+ *
+ * If two servers try to start a connection concurrently, then the connection
+ * manager uses a very simple tie-breaking mechanism to decide which connection
+ * to drop based on the IP addressed of the two parties.
+ *
+ * For every peer, the manager maintains a queue of messages to send. If the
+ * connection to any particular peer drops, then the sender thread puts the
+ * message back on the list. As this implementation currently uses a queue
+ * implementation to maintain messages to send to another peer, we add the
+ * message to the tail of the queue, thus changing the order of messages.
+ * Although this is not a problem for the leader election, it could be a problem
+ * when consolidating peer communication. This is to be verified, though.
+ *
+ */
+
+public class QuorumCnxManager {
+ private static final Logger LOG = LoggerFactory.getLogger(QuorumCnxManager.class);
+
+ /*
+ * Maximum capacity of thread queues
+ */
+ static final int RECV_CAPACITY = 100;
+ // Initialized to 1 to prevent sending
+ // stale notifications to peers
+ static final int SEND_CAPACITY = 1;
+
+ static final int PACKETMAXSIZE = 1024 * 512;
+
+ /*
+ * Negative counter for observer server ids.
+ */
+
+ private AtomicLong observerCounter = new AtomicLong(-1);
+
+ /*
+ * Protocol identifier used among peers
+ */
+ public static final long PROTOCOL_VERSION = -65536L;
+
+ /*
+ * Max buffer size to be read from the network.
+ */
+ static public final int maxBuffer = 2048;
+
+ /*
+ * Connection time out value in milliseconds
+ */
+
+ private int cnxTO = 5000;
+
+ final QuorumPeer self;
+
+ /*
+ * Local IP address
+ */
+ final long mySid;
+ final int socketTimeout;
+ final Map<Long, QuorumPeer.QuorumServer> view;
+ final boolean listenOnAllIPs;
+ private ThreadPoolExecutor connectionExecutor;
+ private final Set<Long> inprogressConnections = Collections.synchronizedSet(new HashSet<>());
+ private QuorumAuthServer authServer;
+ private QuorumAuthLearner authLearner;
+ private boolean quorumSaslAuthEnabled;
+ /*
+ * Counter to count connection processing threads.
+ */
+ private AtomicInteger connectionThreadCnt = new AtomicInteger(0);
+
+ /*
+ * Mapping from Peer to Thread number
+ */
+ final ConcurrentHashMap<Long, SendWorker> senderWorkerMap;
+ final ConcurrentHashMap<Long, ArrayBlockingQueue<ByteBuffer>> queueSendMap;
+ final ConcurrentHashMap<Long, ByteBuffer> lastMessageSent;
+
+ /*
+ * Reception queue
+ */
+ public final ArrayBlockingQueue<Message> recvQueue;
+ /*
+ * Object to synchronize access to recvQueue
+ */
+ private final Object recvQLock = new Object();
+
+ /*
+ * Shutdown flag
+ */
+
+ volatile boolean shutdown = false;
+
+ /*
+ * Listener thread
+ */
+ public final Listener listener;
+
+ /*
+ * Counter to count worker threads
+ */
+ private AtomicInteger threadCnt = new AtomicInteger(0);
+
+ /*
+ * Socket options for TCP keepalive
+ */
+ private final boolean tcpKeepAlive = Boolean.getBoolean("zookeeper.tcpKeepAlive");
+
+ /*
+ * Socket factory, allowing the injection of custom socket implementations for testing
+ */
+ static final Supplier<Socket> DEFAULT_SOCKET_FACTORY = () -> new Socket();
+ private static Supplier<Socket> SOCKET_FACTORY = DEFAULT_SOCKET_FACTORY;
+ static void setSocketFactory(Supplier<Socket> factory) {
+ SOCKET_FACTORY = factory;
+ }
+
+
+ static public class Message {
+ Message(ByteBuffer buffer, long sid) {
+ this.buffer = buffer;
+ this.sid = sid;
+ }
+
+ ByteBuffer buffer;
+ long sid;
+ }
+
+ /*
+ * This class parses the initial identification sent out by peers with their
+ * sid & hostname.
+ */
+ static public class InitialMessage {
+ public Long sid;
+ public InetSocketAddress electionAddr;
+
+ InitialMessage(Long sid, InetSocketAddress address) {
+ this.sid = sid;
+ this.electionAddr = address;
+ }
+
+ @SuppressWarnings("serial")
+ public static class InitialMessageException extends Exception {
+ InitialMessageException(String message, Object... args) {
+ super(String.format(message, args));
+ }
+ }
+
+ static public InitialMessage parse(Long protocolVersion, DataInputStream din)
+ throws InitialMessageException, IOException {
+ Long sid;
+
+ if (protocolVersion != PROTOCOL_VERSION) {
+ throw new InitialMessageException(
+ "Got unrecognized protocol version %s", protocolVersion);
+ }
+
+ sid = din.readLong();
+
+ int remaining = din.readInt();
+ if (remaining <= 0 || remaining > maxBuffer) {
+ throw new InitialMessageException(
+ "Unreasonable buffer length: %s", remaining);
+ }
+
+ byte[] b = new byte[remaining];
+ int num_read = din.read(b);
+
+ if (num_read != remaining) {
+ throw new InitialMessageException(
+ "Read only %s bytes out of %s sent by server %s",
+ num_read, remaining, sid);
+ }
+
+ String addr = new String(b);
+ String[] host_port;
+ try {
+ host_port = ConfigUtils.getHostAndPort(addr);
+ } catch (ConfigException e) {
+ throw new InitialMessageException("Badly formed address: %s", addr);
+ }
+
+ if (host_port.length != 2) {
+ throw new InitialMessageException("Badly formed address: %s", addr);
+ }
+
+ int port;
+ try {
+ port = Integer.parseInt(host_port[1]);
+ } catch (NumberFormatException e) {
+ throw new InitialMessageException("Bad port number: %s", host_port[1]);
+ } catch (ArrayIndexOutOfBoundsException e) {
+ throw new InitialMessageException("No port number in: %s", addr);
+ }
+
+ return new InitialMessage(sid, isWildcardAddress(host_port[0]) ? null :
+ new InetSocketAddress(host_port[0], port));
+ }
+
+ /**
+ * Returns true if the specified hostname is a wildcard address,
+ * like 0.0.0.0 for IPv4 or :: for IPv6
+ */
+ public static boolean isWildcardAddress(final String hostname) {
+ try {
+ return InetAddress.getByName(hostname).isAnyLocalAddress();
+ } catch (UnknownHostException e) {
+ // if we can not resolve, it can not be a wildcard address
+ return false;
+ } catch (SecurityException e) {
+ LOG.warn("SecurityException in getByName() for" + hostname);
+ return false;
+ }
+ }
+
+ @Override
+ public String toString() {
+ return "InitialMessage{sid=" + sid + ", electionAddr=" + electionAddr + '}';
+ }
+ }
+
+ public QuorumCnxManager(QuorumPeer self,
+ final long mySid,
+ Map<Long,QuorumPeer.QuorumServer> view,
+ QuorumAuthServer authServer,
+ QuorumAuthLearner authLearner,
+ int socketTimeout,
+ boolean listenOnAllIPs,
+ int quorumCnxnThreadsSize,
+ boolean quorumSaslAuthEnabled) {
+ this.recvQueue = new ArrayBlockingQueue<Message>(RECV_CAPACITY);
+ this.queueSendMap = new ConcurrentHashMap<Long, ArrayBlockingQueue<ByteBuffer>>();
+ this.senderWorkerMap = new ConcurrentHashMap<Long, SendWorker>();
+ this.lastMessageSent = new ConcurrentHashMap<Long, ByteBuffer>();
+
+ String cnxToValue = System.getProperty("zookeeper.cnxTimeout");
+ if(cnxToValue != null){
+ this.cnxTO = Integer.parseInt(cnxToValue);
+ }
+
+ this.self = self;
+
+ this.mySid = mySid;
+ this.socketTimeout = socketTimeout;
+ this.view = view;
+ this.listenOnAllIPs = listenOnAllIPs;
+ this.authServer = authServer;
+ this.authLearner = authLearner;
+ this.quorumSaslAuthEnabled = quorumSaslAuthEnabled;
+
+ initializeConnectionExecutor(mySid, quorumCnxnThreadsSize);
+
+ // Starts listener thread that waits for connection requests
+ listener = new Listener();
+ listener.setName("QuorumPeerListener");
+ }
+
+ // we always use the Connection Executor during connection initiation (to handle connection
+ // timeouts), and optionally use it during receiving connections (as the Quorum SASL authentication
+ // can take extra time)
+ private void initializeConnectionExecutor(final long mySid, final int quorumCnxnThreadsSize) {
+ final AtomicInteger threadIndex = new AtomicInteger(1);
+ SecurityManager s = System.getSecurityManager();
+ final ThreadGroup group = (s != null) ? s.getThreadGroup()
+ : Thread.currentThread().getThreadGroup();
+ final ThreadFactory daemonThFactory = runnable -> new Thread(group, runnable,
+ String.format("QuorumConnectionThread-[myid=%d]-%d", mySid, threadIndex.getAndIncrement()));
+ this.connectionExecutor = new ThreadPoolExecutor(3, quorumCnxnThreadsSize, 60, TimeUnit.SECONDS,
+ new SynchronousQueue<>(), daemonThFactory);
+ this.connectionExecutor.allowCoreThreadTimeOut(true);
+ }
+
+
+ /**
+ * Invokes initiateConnection for testing purposes
+ *
+ * @param sid
+ */
+ public void testInitiateConnection(long sid) {
+ LOG.debug("Opening channel to server " + sid);
+ initiateConnection(self.getVotingView().get(sid).electionAddr, sid);
+ }
+
+ /**
+ * First we create the socket, perform SSL handshake and authentication if needed.
+ * Then we perform the initiation protocol.
+ * If this server has initiated the connection, then it gives up on the
+ * connection if it loses challenge. Otherwise, it keeps the connection.
+ */
+ public void initiateConnection(final InetSocketAddress electionAddr, final Long sid) {
+
+ Socket sock = null;
+ try {
+ LOG.debug("Opening channel to server " + sid);
+ if (self.isSslQuorum()) {
+ SSLSocket sslSock = self.getX509Util().createSSLSocket();
+ setSockOpts(sslSock);
+ sslSock.connect(electionAddr, cnxTO);
+ sslSock.startHandshake();
+ sock = sslSock;
+ LOG.info("SSL handshake complete with {} - {} - {}", sslSock.getRemoteSocketAddress(),
+ sslSock.getSession().getProtocol(), sslSock.getSession().getCipherSuite());
+ } else {
+ sock = SOCKET_FACTORY.get();
+ setSockOpts(sock);
+ sock.connect(electionAddr, cnxTO);
+ }
+ LOG.debug("Connected to server " + sid);
+ } catch (X509Exception e) {
+ LOG.warn("Cannot open secure channel to {} at election address {}", sid, electionAddr, e);
+ closeSocket(sock);
+ return;
+ } catch (UnresolvedAddressException | IOException e) {
+ LOG.warn("Cannot open channel to {} at election address {}", sid, electionAddr, e);
+ closeSocket(sock);
+ return;
+ }
+
+ try {
+ startConnection(sock, sid);
+ } catch (IOException e) {
+ LOG.error("Exception while connecting, id: {}, addr: {}, closing learner connection",
+ new Object[] { sid, sock.getRemoteSocketAddress() }, e);
+ closeSocket(sock);
+ }
+ }
+
+ /**
+ * Server will initiate the connection request to its peer server
+ * asynchronously via separate connection thread.
+ */
+ public boolean initiateConnectionAsync(final InetSocketAddress electionAddr, final Long sid) {
+ if(!inprogressConnections.add(sid)){
+ // simply return as there is a connection request to
+ // server 'sid' already in progress.
+ LOG.debug("Connection request to server id: {} is already in progress, so skipping this request",
+ sid);
+ return true;
+ }
+ try {
+ connectionExecutor.execute(new QuorumConnectionReqThread(electionAddr, sid));
+ connectionThreadCnt.incrementAndGet();
+ } catch (Throwable e) {
+ // Imp: Safer side catching all type of exceptions and remove 'sid'
+ // from inprogress connections. This is to avoid blocking further
+ // connection requests from this 'sid' in case of errors.
+ inprogressConnections.remove(sid);
+ LOG.error("Exception while submitting quorum connection request", e);
+ return false;
+ }
+ return true;
+ }
+
+ /**
+ * Thread to send connection request to peer server.
+ */
+ private class QuorumConnectionReqThread extends ZooKeeperThread {
+ final InetSocketAddress electionAddr;
+ final Long sid;
+ QuorumConnectionReqThread(final InetSocketAddress electionAddr, final Long sid) {
+ super("QuorumConnectionReqThread-" + sid);
+ this.electionAddr = electionAddr;
+ this.sid = sid;
+ }
+
+ @Override
+ public void run() {
+ try{
+ initiateConnection(electionAddr, sid);
+ } finally {
+ inprogressConnections.remove(sid);
+ }
+ }
+ }
+
+ private boolean startConnection(Socket sock, Long sid)
+ throws IOException {
+ DataOutputStream dout = null;
+ DataInputStream din = null;
+ LOG.debug("startConnection (myId:{} --> sid:{})", self.getId(), sid);
+ try {
+ // Use BufferedOutputStream to reduce the number of IP packets. This is
+ // important for x-DC scenarios.
+ BufferedOutputStream buf = new BufferedOutputStream(sock.getOutputStream());
+ dout = new DataOutputStream(buf);
+
+ // Sending id and challenge
+ // represents protocol version (in other words - message type)
+ dout.writeLong(PROTOCOL_VERSION);
+ dout.writeLong(self.getId());
+ String addr = formatInetAddr(self.getElectionAddress());
+ byte[] addr_bytes = addr.getBytes();
+ dout.writeInt(addr_bytes.length);
+ dout.write(addr_bytes);
+ dout.flush();
+
+ din = new DataInputStream(
+ new BufferedInputStream(sock.getInputStream()));
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception reading or writing challenge: ", e);
+ closeSocket(sock);
+ return false;
+ }
+
+ // authenticate learner
+ QuorumPeer.QuorumServer qps = self.getVotingView().get(sid);
+ if (qps != null) {
+ // TODO - investigate why reconfig makes qps null.
+ authLearner.authenticate(sock, qps.hostname);
+ }
+
+ // If lost the challenge, then drop the new connection
+ if (sid > self.getId()) {
+ LOG.info("Have smaller server identifier, so dropping the connection: (myId:{} --> sid:{})", self.getId(), sid);
+ closeSocket(sock);
+ // Otherwise proceed with the connection
+ } else {
+ LOG.debug("Have larger server identifier, so keeping the connection: (myId:{} --> sid:{})", self.getId(), sid);
+ SendWorker sw = new SendWorker(sock, sid);
+ RecvWorker rw = new RecvWorker(sock, din, sid, sw);
+ sw.setRecv(rw);
+
+ SendWorker vsw = senderWorkerMap.get(sid);
+
+ if(vsw != null)
+ vsw.finish();
+
+ senderWorkerMap.put(sid, sw);
+ queueSendMap.putIfAbsent(sid, new ArrayBlockingQueue<ByteBuffer>(
+ SEND_CAPACITY));
+
+ sw.start();
+ rw.start();
+
+ return true;
+
+ }
+ return false;
+ }
+
+ /**
+ * If this server receives a connection request, then it gives up on the new
+ * connection if it wins. Notice that it checks whether it has a connection
+ * to this server already or not. If it does, then it sends the smallest
+ * possible long value to lose the challenge.
+ *
+ */
+ public void receiveConnection(final Socket sock) {
+ DataInputStream din = null;
+ try {
+ din = new DataInputStream(
+ new BufferedInputStream(sock.getInputStream()));
+
+ LOG.debug("Sync handling of connection request received from: {}", sock.getRemoteSocketAddress());
+ handleConnection(sock, din);
+ } catch (IOException e) {
+ LOG.error("Exception handling connection, addr: {}, closing server connection",
+ sock.getRemoteSocketAddress());
+ LOG.debug("Exception details: ", e);
+ closeSocket(sock);
+ }
+ }
+
+ /**
+ * Server receives a connection request and handles it asynchronously via
+ * separate thread.
+ */
+ public void receiveConnectionAsync(final Socket sock) {
+ try {
+ LOG.debug("Async handling of connection request received from: {}", sock.getRemoteSocketAddress());
+ connectionExecutor.execute(
+ new QuorumConnectionReceiverThread(sock));
+ connectionThreadCnt.incrementAndGet();
+ } catch (Throwable e) {
+ LOG.error("Exception handling connection, addr: {}, closing server connection",
+ sock.getRemoteSocketAddress());
+ LOG.debug("Exception details: ", e);
+ closeSocket(sock);
+ }
+ }
+
+ /**
+ * Thread to receive connection request from peer server.
+ */
+ private class QuorumConnectionReceiverThread extends ZooKeeperThread {
+ private final Socket sock;
+ QuorumConnectionReceiverThread(final Socket sock) {
+ super("QuorumConnectionReceiverThread-" + sock.getRemoteSocketAddress());
+ this.sock = sock;
+ }
+
+ @Override
+ public void run() {
+ receiveConnection(sock);
+ }
+ }
+
+ private void handleConnection(Socket sock, DataInputStream din)
+ throws IOException {
+ Long sid = null, protocolVersion = null;
+ InetSocketAddress electionAddr = null;
+
+ try {
+ protocolVersion = din.readLong();
+ if (protocolVersion >= 0) { // this is a server id and not a protocol version
+ sid = protocolVersion;
+ } else {
+ try {
+ InitialMessage init = InitialMessage.parse(protocolVersion, din);
+ sid = init.sid;
+ electionAddr = init.electionAddr;
+ } catch (InitialMessage.InitialMessageException ex) {
+ LOG.error("Initial message parsing error!", ex);
+ closeSocket(sock);
+ return;
+ }
+ }
+
+ if (sid == QuorumPeer.OBSERVER_ID) {
+ /*
+ * Choose identifier at random. We need a value to identify
+ * the connection.
+ */
+ sid = observerCounter.getAndDecrement();
+ LOG.info("Setting arbitrary identifier to observer: " + sid);
+ }
+ } catch (IOException e) {
+ LOG.warn("Exception reading or writing challenge: {}", e);
+ closeSocket(sock);
+ return;
+ }
+
+ // do authenticating learner
+ authServer.authenticate(sock, din);
+ //If wins the challenge, then close the new connection.
+ if (sid < self.getId()) {
+ /*
+ * This replica might still believe that the connection to sid is
+ * up, so we have to shut down the workers before trying to open a
+ * new connection.
+ */
+ SendWorker sw = senderWorkerMap.get(sid);
+ if (sw != null) {
+ sw.finish();
+ }
+
+ /*
+ * Now we start a new connection
+ */
+ LOG.debug("Create new connection to server: {}", sid);
+ closeSocket(sock);
+
+ if (electionAddr != null) {
+ connectOne(sid, electionAddr);
+ } else {
+ connectOne(sid);
+ }
+ } else if (sid == self.getId()) {
+ // we saw this case in ZOOKEEPER-2164
+ LOG.warn("We got a connection request from a server with our own ID. "
+ + "This should be either a configuration error, or a bug.");
+ } else { // Otherwise start worker threads to receive data.
+ SendWorker sw = new SendWorker(sock, sid);
+ RecvWorker rw = new RecvWorker(sock, din, sid, sw);
+ sw.setRecv(rw);
+
+ SendWorker vsw = senderWorkerMap.get(sid);
+
+ if (vsw != null) {
+ vsw.finish();
+ }
+
+ senderWorkerMap.put(sid, sw);
+
+ queueSendMap.putIfAbsent(sid,
+ new ArrayBlockingQueue<ByteBuffer>(SEND_CAPACITY));
+
+ sw.start();
+ rw.start();
+ }
+ }
+
+ /**
+ * Processes invoke this message to queue a message to send. Currently,
+ * only leader election uses it.
+ */
+ public void toSend(Long sid, ByteBuffer b) {
+ /*
+ * If sending message to myself, then simply enqueue it (loopback).
+ */
+ if (this.mySid == sid) {
+ b.position(0);
+ addToRecvQueue(new Message(b.duplicate(), sid));
+ /*
+ * Otherwise send to the corresponding thread to send.
+ */
+ } else {
+ /*
+ * Start a new connection if doesn't have one already.
+ */
+ ArrayBlockingQueue<ByteBuffer> bq = new ArrayBlockingQueue<ByteBuffer>(
+ SEND_CAPACITY);
+ ArrayBlockingQueue<ByteBuffer> oldq = queueSendMap.putIfAbsent(sid, bq);
+ if (oldq != null) {
+ addToSendQueue(oldq, b);
+ } else {
+ addToSendQueue(bq, b);
+ }
+ connectOne(sid);
+
+ }
+ }
+
+ /**
+ * Try to establish a connection to server with id sid using its electionAddr.
+ * The function will return quickly and the connection will be established asynchronously.
+ *
+ * @param sid server id
+ * @return boolean success indication
+ */
+ synchronized private boolean connectOne(long sid, InetSocketAddress electionAddr){
+ if (senderWorkerMap.get(sid) != null) {
+ LOG.debug("There is a connection already for server " + sid);
+ return true;
+ }
+
+ // we are doing connection initiation always asynchronously, since it is possible that
+ // the socket connection timeouts or the SSL handshake takes too long and don't want
+ // to keep the rest of the connections to wait
+ return initiateConnectionAsync(electionAddr, sid);
+ }
+
+ /**
+ * Try to establish a connection to server with id sid.
+ * The function will return quickly and the connection will be established asynchronously.
+ *
+ * @param sid server id
+ */
+ synchronized void connectOne(long sid){
+ if (senderWorkerMap.get(sid) != null) {
+ LOG.debug("There is a connection already for server " + sid);
+ return;
+ }
+ synchronized (self.QV_LOCK) {
+ boolean knownId = false;
+ // Resolve hostname for the remote server before attempting to
+ // connect in case the underlying ip address has changed.
+ self.recreateSocketAddresses(sid);
+ Map<Long, QuorumPeer.QuorumServer> lastCommittedView = self.getView();
+ QuorumVerifier lastSeenQV = self.getLastSeenQuorumVerifier();
+ Map<Long, QuorumPeer.QuorumServer> lastProposedView = lastSeenQV.getAllMembers();
+ if (lastCommittedView.containsKey(sid)) {
+ knownId = true;
+ LOG.debug("Server {} knows {} already, it is in the lastCommittedView", self.getId(), sid);
+ if (connectOne(sid, lastCommittedView.get(sid).electionAddr))
+ return;
+ }
+ if (lastSeenQV != null && lastProposedView.containsKey(sid)
+ && (!knownId || (lastProposedView.get(sid).electionAddr !=
+ lastCommittedView.get(sid).electionAddr))) {
+ knownId = true;
+ LOG.debug("Server {} knows {} already, it is in the lastProposedView", self.getId(), sid);
+ if (connectOne(sid, lastProposedView.get(sid).electionAddr))
+ return;
+ }
+ if (!knownId) {
+ LOG.warn("Invalid server id: " + sid);
+ return;
+ }
+ }
+ }
+
+
+ /**
+ * Try to establish a connection with each server if one
+ * doesn't exist.
+ */
+
+ public void connectAll(){
+ long sid;
+ for(Enumeration<Long> en = queueSendMap.keys();
+ en.hasMoreElements();){
+ sid = en.nextElement();
+ connectOne(sid);
+ }
+ }
+
+
+ /**
+ * Check if all queues are empty, indicating that all messages have been delivered.
+ */
+ boolean haveDelivered() {
+ for (ArrayBlockingQueue<ByteBuffer> queue : queueSendMap.values()) {
+ LOG.debug("Queue size: " + queue.size());
+ if (queue.size() == 0) {
+ return true;
+ }
+ }
+
+ return false;
+ }
+
+ /**
+ * Flag that it is time to wrap up all activities and interrupt the listener.
+ */
+ public void halt() {
+ shutdown = true;
+ LOG.debug("Halting listener");
+ listener.halt();
+
+ // Wait for the listener to terminate.
+ try {
+ listener.join();
+ } catch (InterruptedException ex) {
+ LOG.warn("Got interrupted before joining the listener", ex);
+ }
+ softHalt();
+
+ // clear data structures used for auth
+ if (connectionExecutor != null) {
+ connectionExecutor.shutdown();
+ }
+ inprogressConnections.clear();
+ resetConnectionThreadCount();
+ }
+
+ /**
+ * A soft halt simply finishes workers.
+ */
+ public void softHalt() {
+ for (SendWorker sw : senderWorkerMap.values()) {
+ LOG.debug("Halting sender: " + sw);
+ LOG.debug("Server {} is soft-halting sender towards: {}", self.getId(), sw);
+ sw.finish();
+ }
+ }
+
+ /**
+ * Helper method to set socket options.
+ *
+ * @param sock
+ * Reference to socket
+ */
+ private void setSockOpts(Socket sock) throws SocketException {
+ sock.setTcpNoDelay(true);
+ sock.setKeepAlive(tcpKeepAlive);
+ sock.setSoTimeout(self.tickTime * self.syncLimit);
+ }
+
+ /**
+ * Helper method to close a socket.
+ *
+ * @param sock
+ * Reference to socket
+ */
+ private void closeSocket(Socket sock) {
+ if (sock == null) {
+ return;
+ }
+
+ try {
+ sock.close();
+ } catch (IOException ie) {
+ LOG.error("Exception while closing", ie);
+ }
+ }
+
+ /**
+ * Return number of worker threads
+ */
+ public long getThreadCount() {
+ return threadCnt.get();
+ }
+
+ /**
+ * Return number of connection processing threads.
+ */
+ public long getConnectionThreadCount() {
+ return connectionThreadCnt.get();
+ }
+
+ /**
+ * Reset the value of connection processing threads count to zero.
+ */
+ private void resetConnectionThreadCount() {
+ connectionThreadCnt.set(0);
+ }
+
+ /**
+ * Thread to listen on some port
+ */
+ public class Listener extends ZooKeeperThread {
+
+ private static final String ELECTION_PORT_BIND_RETRY = "zookeeper.electionPortBindRetry";
+ private static final int DEFAULT_PORT_BIND_MAX_RETRY = 3;
+
+ private final int portBindMaxRetry;
+ private Runnable socketBindErrorHandler = () -> System.exit(ExitCode.UNABLE_TO_BIND_QUORUM_PORT.getValue());
+ volatile ServerSocket ss = null;
+
+ public Listener() {
+ // During startup of thread, thread name will be overridden to
+ // specific election address
+ super("ListenerThread");
+
+ // maximum retry count while trying to bind to election port
+ // see ZOOKEEPER-3320 for more details
+ final Integer maxRetry = Integer.getInteger(ELECTION_PORT_BIND_RETRY,
+ DEFAULT_PORT_BIND_MAX_RETRY);
+ if (maxRetry >= 0) {
+ LOG.info("Election port bind maximum retries is {}",
+ maxRetry == 0 ? "infinite" : maxRetry);
+ portBindMaxRetry = maxRetry;
+ } else {
+ LOG.info("'{}' contains invalid value: {}(must be >= 0). "
+ + "Use default value of {} instead.",
+ ELECTION_PORT_BIND_RETRY, maxRetry, DEFAULT_PORT_BIND_MAX_RETRY);
+ portBindMaxRetry = DEFAULT_PORT_BIND_MAX_RETRY;
+ }
+ }
+
+ /**
+ * Change socket bind error handler. Used for testing.
+ */
+ public void setSocketBindErrorHandler(Runnable errorHandler) {
+ this.socketBindErrorHandler = errorHandler;
+ }
+
+ /**
+ * Sleeps on accept().
+ */
+ @Override
+ public void run() {
+ int numRetries = 0;
+ InetSocketAddress addr;
+ Socket client = null;
+ Exception exitException = null;
+ while ((!shutdown) && (portBindMaxRetry == 0 || numRetries < portBindMaxRetry)) {
+ LOG.debug("Listener thread started, myId: {}", self.getId());
+ try {
+ if (self.shouldUsePortUnification()) {
+ LOG.info("Creating TLS-enabled quorum server socket");
+ ss = new UnifiedServerSocket(self.getX509Util(), true);
+ } else if (self.isSslQuorum()) {
+ LOG.info("Creating TLS-only quorum server socket");
+ ss = new UnifiedServerSocket(self.getX509Util(), false);
+ } else {
+ ss = new ServerSocket();
+ }
+
+ ss.setReuseAddress(true);
+
+ if (self.getQuorumListenOnAllIPs()) {
+ int port = self.getElectionAddress().getPort();
+ addr = new InetSocketAddress(port);
+ } else {
+ // Resolve hostname for this server in case the
+ // underlying ip address has changed.
+ self.recreateSocketAddresses(self.getId());
+ addr = self.getElectionAddress();
+ }
+ LOG.info("{} is accepting connections now, my election bind port: {}", QuorumCnxManager.this.mySid, addr.toString());
+ setName(addr.toString());
+ ss.bind(addr);
+ while (!shutdown) {
+ try {
+ client = ss.accept();
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.InputStream` acquired to `client` by call to `accept()` at line 934 is not released after line 934.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/util/SerializeUtils.java
##########
@@ -0,0 +1,171 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.util;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.IOUtils;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.CreateTxnV0;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.EOFException;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Map.Entry;
+
+public class SerializeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(SerializeUtils.class);
+
+ public static Record deserializeTxn(byte txnBytes[], TxnHeader hdr)
+ throws IOException {
+ final ByteArrayInputStream bais = new ByteArrayInputStream(txnBytes);
+ InputArchive ia = BinaryInputArchive.getArchive(bais);
+
+ hdr.deserialize(ia, "hdr");
+ bais.mark(bais.available());
+ Record txn = null;
+ switch (hdr.getType()) {
+ case OpCode.createSession:
+ // This isn't really an error txn; it just has the same
+ // format. The error represents the timeout
+ txn = new CreateSessionTxn();
+ break;
+ case OpCode.closeSession:
+ return null;
+ case OpCode.create:
+ case OpCode.create2:
+ txn = new CreateTxn();
+ break;
+ case OpCode.createTTL:
+ txn = new CreateTTLTxn();
+ break;
+ case OpCode.createContainer:
+ txn = new CreateContainerTxn();
+ break;
+ case OpCode.delete:
+ case OpCode.deleteContainer:
+ txn = new DeleteTxn();
+ break;
+ case OpCode.reconfig:
+ case OpCode.setData:
+ txn = new SetDataTxn();
+ break;
+ case OpCode.setACL:
+ txn = new SetACLTxn();
+ break;
+ case OpCode.error:
+ txn = new ErrorTxn();
+ break;
+ case OpCode.multi:
+ txn = new MultiTxn();
+ break;
+ default:
+ throw new IOException("Unsupported Txn with type=%d" + hdr.getType());
+ }
+ if (txn != null) {
+ try {
+ txn.deserialize(ia, "txn");
+ } catch(EOFException e) {
+ // perhaps this is a V0 Create
+ if (hdr.getType() == OpCode.create) {
+ CreateTxn create = (CreateTxn)txn;
+ bais.reset();
+ CreateTxnV0 createv0 = new CreateTxnV0();
+ createv0.deserialize(ia, "txn");
+ // cool now make it V1. a -1 parentCVersion will
+ // trigger fixup processing in processTxn
+ create.setPath(createv0.getPath());
+ create.setData(createv0.getData());
+ create.setAcl(createv0.getAcl());
+ create.setEphemeral(createv0.getEphemeral());
+ create.setParentCVersion(-1);
+ } else {
+ throw e;
+ }
+ }
+ }
+ return txn;
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 59 is not released after line 124.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/util/SerializeUtils.java
##########
@@ -0,0 +1,171 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.util;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.IOUtils;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.CreateTxnV0;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.EOFException;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Map.Entry;
+
+public class SerializeUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(SerializeUtils.class);
+
+ public static Record deserializeTxn(byte txnBytes[], TxnHeader hdr)
+ throws IOException {
+ final ByteArrayInputStream bais = new ByteArrayInputStream(txnBytes);
+ InputArchive ia = BinaryInputArchive.getArchive(bais);
+
+ hdr.deserialize(ia, "hdr");
+ bais.mark(bais.available());
+ Record txn = null;
+ switch (hdr.getType()) {
+ case OpCode.createSession:
+ // This isn't really an error txn; it just has the same
+ // format. The error represents the timeout
+ txn = new CreateSessionTxn();
+ break;
+ case OpCode.closeSession:
+ return null;
Review comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired to `return` by call to `getArchive(...)` at line 59 is not released after line 71.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNIO.doIO(...)` indirectly reads with synchronization from `this.now`. Potentially races with unsynchronized write in method `ClientCnxnSocketNIO.doTransport(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNIO.doIO(...)` indirectly reads with synchronization from `this.sendThread.this$0.zooKeeperSaslClient.gotLastPacket`. Potentially races with unsynchronized write in method `ClientCnxnSocketNIO.doIO(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNIO.doIO(...)` reads with synchronization from `this.initialized`. Potentially races with unsynchronized write in method `ClientCnxnSocketNIO.doIO(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNIO.doIO(...)` reads without synchronization from `this.incomingBuffer`. Potentially races with write in method `ClientCnxnSocketNIO.doIO(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNIO.doIO(...)` reads without synchronization from `this.sockKey`. Potentially races with write in method `ClientCnxnSocketNIO.cleanup()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doIO(...)` indirectly writes to field `this.lastHeard` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doIO(...)` indirectly writes to field `this.sendThread.this$0.connectTimeout` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doIO(...)` indirectly writes to field `this.sendThread.this$0.zooKeeperSaslClient.gotLastPacket` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doIO(...)` writes to field `this.incomingBuffer` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doIO(...)` writes to field `this.initialized` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
+ // On initial connection, write the complete connect request
+ // packet, but then disable further writes until after
+ // receiving a successful connection response. If the
+ // session is expired, then the server sends the expiration
+ // response and immediately closes its end of the socket. If
+ // the client is simultaneously writing on its end, then the
+ // TCP stack may choose to abort with RST, in which case the
+ // client would never receive the session expired event. See
+ // http://docs.oracle.com/javase/6/docs/technotes/guides/net/articles/connection_release.html
+ disableWrite();
+ } else {
+ // Just in case
+ enableWrite();
+ }
+ }
+ }
+
+ private Packet findSendablePacket(LinkedBlockingDeque<Packet> outgoingQueue,
+ boolean tunneledAuthInProgres) {
+ if (outgoingQueue.isEmpty()) {
+ return null;
+ }
+ // If we've already starting sending the first packet, we better finish
+ if (outgoingQueue.getFirst().bb != null || !tunneledAuthInProgres) {
+ return outgoingQueue.getFirst();
+ }
+ // Since client's authentication with server is in progress,
+ // send only the null-header packet queued by primeConnection().
+ // This packet must be sent so that the SASL authentication process
+ // can proceed, but all other packets should wait until
+ // SASL authentication completes.
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p.requestHeader == null) {
+ // We've found the priming-packet. Move it to the beginning of the queue.
+ iter.remove();
+ outgoingQueue.addFirst(p);
+ return p;
+ } else {
+ // Non-priming packet: defer it until later, leaving it in the queue
+ // until authentication completes.
+ LOG.debug("deferring non-priming packet {} until SASL authentation completes.", p);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ void cleanup() {
+ if (sockKey != null) {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ sockKey.cancel();
+ try {
+ sock.socket().shutdownInput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown input", e);
+ }
+ }
+ try {
+ sock.socket().shutdownOutput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown output",
+ e);
+ }
+ }
+ try {
+ sock.socket().close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during socket close", e);
+ }
+ }
+ try {
+ sock.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during channel close", e);
+ }
+ }
+ }
+ try {
+ Thread.sleep(100);
+ } catch (InterruptedException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SendThread interrupted during sleep, ignoring");
+ }
+ }
+ sockKey = null;
+ }
+
+ @Override
+ void close() {
+ try {
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Doing client selector close");
+ }
+ selector.close();
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closed client selector");
+ }
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during selector close", e);
+ }
+ }
+
+ /**
+ * create a socket channel.
+ * @return the created socket channel
+ * @throws IOException
+ */
+ SocketChannel createSock() throws IOException {
+ SocketChannel sock;
+ sock = SocketChannel.open();
+ sock.configureBlocking(false);
+ sock.socket().setSoLinger(false, -1);
+ sock.socket().setTcpNoDelay(true);
+ return sock;
+ }
+
+ /**
+ * register with the selection and connect
+ * @param sock the {@link SocketChannel}
+ * @param addr the address of remote host
+ * @throws IOException
+ */
+ void registerAndConnect(SocketChannel sock, InetSocketAddress addr)
+ throws IOException {
+ sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
+ boolean immediateConnect = sock.connect(addr);
+ if (immediateConnect) {
+ sendThread.primeConnection();
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ SocketChannel sock = createSock();
+ try {
+ registerAndConnect(sock, addr);
+ } catch (IOException e) {
+ LOG.error("Unable to open socket to " + addr);
+ sock.close();
+ throw e;
+ }
+ initialized = false;
+
+ /*
+ * Reset incomingBuffer
+ */
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ }
+
+ /**
+ * Returns the address to which the socket is connected.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getRemoteSocketAddress() {
+ return remoteSocketAddress;
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getLocalSocketAddress() {
+ return localSocketAddress;
+ }
+
+ private void updateSocketAddresses() {
+ Socket socket = ((SocketChannel) sockKey.channel()).socket();
+ localSocketAddress = socket.getLocalSocketAddress();
+ remoteSocketAddress = socket.getRemoteSocketAddress();
+ }
+
+ @Override
+ void packetAdded() {
+ wakeupCnxn();
+ }
+
+ @Override
+ void onClosing() {
+ wakeupCnxn();
+ }
+
+ private synchronized void wakeupCnxn() {
+ selector.wakeup();
+ }
+
+ @Override
+ void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ selector.select(waitTimeOut);
+ Set<SelectionKey> selected;
+ synchronized (this) {
+ selected = selector.selectedKeys();
+ }
+ // Everything below and until we get back to the select is
+ // non blocking, so time is effectively a constant. That is
+ // Why we just have to do this once, here
+ updateNow();
+ for (SelectionKey k : selected) {
+ SocketChannel sc = ((SocketChannel) k.channel());
+ if ((k.readyOps() & SelectionKey.OP_CONNECT) != 0) {
+ if (sc.finishConnect()) {
+ updateLastSendAndHeard();
+ updateSocketAddresses();
+ sendThread.primeConnection();
+ }
+ } else if ((k.readyOps() & (SelectionKey.OP_READ | SelectionKey.OP_WRITE)) != 0) {
+ doIO(pendingQueue, cnxn);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doTransport(...)` indirectly writes to field `this.incomingBuffer` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
+ // On initial connection, write the complete connect request
+ // packet, but then disable further writes until after
+ // receiving a successful connection response. If the
+ // session is expired, then the server sends the expiration
+ // response and immediately closes its end of the socket. If
+ // the client is simultaneously writing on its end, then the
+ // TCP stack may choose to abort with RST, in which case the
+ // client would never receive the session expired event. See
+ // http://docs.oracle.com/javase/6/docs/technotes/guides/net/articles/connection_release.html
+ disableWrite();
+ } else {
+ // Just in case
+ enableWrite();
+ }
+ }
+ }
+
+ private Packet findSendablePacket(LinkedBlockingDeque<Packet> outgoingQueue,
+ boolean tunneledAuthInProgres) {
+ if (outgoingQueue.isEmpty()) {
+ return null;
+ }
+ // If we've already starting sending the first packet, we better finish
+ if (outgoingQueue.getFirst().bb != null || !tunneledAuthInProgres) {
+ return outgoingQueue.getFirst();
+ }
+ // Since client's authentication with server is in progress,
+ // send only the null-header packet queued by primeConnection().
+ // This packet must be sent so that the SASL authentication process
+ // can proceed, but all other packets should wait until
+ // SASL authentication completes.
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p.requestHeader == null) {
+ // We've found the priming-packet. Move it to the beginning of the queue.
+ iter.remove();
+ outgoingQueue.addFirst(p);
+ return p;
+ } else {
+ // Non-priming packet: defer it until later, leaving it in the queue
+ // until authentication completes.
+ LOG.debug("deferring non-priming packet {} until SASL authentation completes.", p);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ void cleanup() {
+ if (sockKey != null) {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ sockKey.cancel();
+ try {
+ sock.socket().shutdownInput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown input", e);
+ }
+ }
+ try {
+ sock.socket().shutdownOutput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown output",
+ e);
+ }
+ }
+ try {
+ sock.socket().close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during socket close", e);
+ }
+ }
+ try {
+ sock.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during channel close", e);
+ }
+ }
+ }
+ try {
+ Thread.sleep(100);
+ } catch (InterruptedException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SendThread interrupted during sleep, ignoring");
+ }
+ }
+ sockKey = null;
+ }
+
+ @Override
+ void close() {
+ try {
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Doing client selector close");
+ }
+ selector.close();
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closed client selector");
+ }
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during selector close", e);
+ }
+ }
+
+ /**
+ * create a socket channel.
+ * @return the created socket channel
+ * @throws IOException
+ */
+ SocketChannel createSock() throws IOException {
+ SocketChannel sock;
+ sock = SocketChannel.open();
+ sock.configureBlocking(false);
+ sock.socket().setSoLinger(false, -1);
+ sock.socket().setTcpNoDelay(true);
+ return sock;
+ }
+
+ /**
+ * register with the selection and connect
+ * @param sock the {@link SocketChannel}
+ * @param addr the address of remote host
+ * @throws IOException
+ */
+ void registerAndConnect(SocketChannel sock, InetSocketAddress addr)
+ throws IOException {
+ sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
+ boolean immediateConnect = sock.connect(addr);
+ if (immediateConnect) {
+ sendThread.primeConnection();
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ SocketChannel sock = createSock();
+ try {
+ registerAndConnect(sock, addr);
+ } catch (IOException e) {
+ LOG.error("Unable to open socket to " + addr);
+ sock.close();
+ throw e;
+ }
+ initialized = false;
+
+ /*
+ * Reset incomingBuffer
+ */
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ }
+
+ /**
+ * Returns the address to which the socket is connected.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getRemoteSocketAddress() {
+ return remoteSocketAddress;
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getLocalSocketAddress() {
+ return localSocketAddress;
+ }
+
+ private void updateSocketAddresses() {
+ Socket socket = ((SocketChannel) sockKey.channel()).socket();
+ localSocketAddress = socket.getLocalSocketAddress();
+ remoteSocketAddress = socket.getRemoteSocketAddress();
+ }
+
+ @Override
+ void packetAdded() {
+ wakeupCnxn();
+ }
+
+ @Override
+ void onClosing() {
+ wakeupCnxn();
+ }
+
+ private synchronized void wakeupCnxn() {
+ selector.wakeup();
+ }
+
+ @Override
+ void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ selector.select(waitTimeOut);
+ Set<SelectionKey> selected;
+ synchronized (this) {
+ selected = selector.selectedKeys();
+ }
+ // Everything below and until we get back to the select is
+ // non blocking, so time is effectively a constant. That is
+ // Why we just have to do this once, here
+ updateNow();
+ for (SelectionKey k : selected) {
+ SocketChannel sc = ((SocketChannel) k.channel());
+ if ((k.readyOps() & SelectionKey.OP_CONNECT) != 0) {
+ if (sc.finishConnect()) {
+ updateLastSendAndHeard();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doTransport(...)` indirectly writes to field `this.lastHeard` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
+ // On initial connection, write the complete connect request
+ // packet, but then disable further writes until after
+ // receiving a successful connection response. If the
+ // session is expired, then the server sends the expiration
+ // response and immediately closes its end of the socket. If
+ // the client is simultaneously writing on its end, then the
+ // TCP stack may choose to abort with RST, in which case the
+ // client would never receive the session expired event. See
+ // http://docs.oracle.com/javase/6/docs/technotes/guides/net/articles/connection_release.html
+ disableWrite();
+ } else {
+ // Just in case
+ enableWrite();
+ }
+ }
+ }
+
+ private Packet findSendablePacket(LinkedBlockingDeque<Packet> outgoingQueue,
+ boolean tunneledAuthInProgres) {
+ if (outgoingQueue.isEmpty()) {
+ return null;
+ }
+ // If we've already starting sending the first packet, we better finish
+ if (outgoingQueue.getFirst().bb != null || !tunneledAuthInProgres) {
+ return outgoingQueue.getFirst();
+ }
+ // Since client's authentication with server is in progress,
+ // send only the null-header packet queued by primeConnection().
+ // This packet must be sent so that the SASL authentication process
+ // can proceed, but all other packets should wait until
+ // SASL authentication completes.
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p.requestHeader == null) {
+ // We've found the priming-packet. Move it to the beginning of the queue.
+ iter.remove();
+ outgoingQueue.addFirst(p);
+ return p;
+ } else {
+ // Non-priming packet: defer it until later, leaving it in the queue
+ // until authentication completes.
+ LOG.debug("deferring non-priming packet {} until SASL authentation completes.", p);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ void cleanup() {
+ if (sockKey != null) {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ sockKey.cancel();
+ try {
+ sock.socket().shutdownInput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown input", e);
+ }
+ }
+ try {
+ sock.socket().shutdownOutput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown output",
+ e);
+ }
+ }
+ try {
+ sock.socket().close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during socket close", e);
+ }
+ }
+ try {
+ sock.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during channel close", e);
+ }
+ }
+ }
+ try {
+ Thread.sleep(100);
+ } catch (InterruptedException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SendThread interrupted during sleep, ignoring");
+ }
+ }
+ sockKey = null;
+ }
+
+ @Override
+ void close() {
+ try {
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Doing client selector close");
+ }
+ selector.close();
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closed client selector");
+ }
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during selector close", e);
+ }
+ }
+
+ /**
+ * create a socket channel.
+ * @return the created socket channel
+ * @throws IOException
+ */
+ SocketChannel createSock() throws IOException {
+ SocketChannel sock;
+ sock = SocketChannel.open();
+ sock.configureBlocking(false);
+ sock.socket().setSoLinger(false, -1);
+ sock.socket().setTcpNoDelay(true);
+ return sock;
+ }
+
+ /**
+ * register with the selection and connect
+ * @param sock the {@link SocketChannel}
+ * @param addr the address of remote host
+ * @throws IOException
+ */
+ void registerAndConnect(SocketChannel sock, InetSocketAddress addr)
+ throws IOException {
+ sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
+ boolean immediateConnect = sock.connect(addr);
+ if (immediateConnect) {
+ sendThread.primeConnection();
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ SocketChannel sock = createSock();
+ try {
+ registerAndConnect(sock, addr);
+ } catch (IOException e) {
+ LOG.error("Unable to open socket to " + addr);
+ sock.close();
+ throw e;
+ }
+ initialized = false;
+
+ /*
+ * Reset incomingBuffer
+ */
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ }
+
+ /**
+ * Returns the address to which the socket is connected.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getRemoteSocketAddress() {
+ return remoteSocketAddress;
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getLocalSocketAddress() {
+ return localSocketAddress;
+ }
+
+ private void updateSocketAddresses() {
+ Socket socket = ((SocketChannel) sockKey.channel()).socket();
+ localSocketAddress = socket.getLocalSocketAddress();
+ remoteSocketAddress = socket.getRemoteSocketAddress();
+ }
+
+ @Override
+ void packetAdded() {
+ wakeupCnxn();
+ }
+
+ @Override
+ void onClosing() {
+ wakeupCnxn();
+ }
+
+ private synchronized void wakeupCnxn() {
+ selector.wakeup();
+ }
+
+ @Override
+ void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ selector.select(waitTimeOut);
+ Set<SelectionKey> selected;
+ synchronized (this) {
+ selected = selector.selectedKeys();
+ }
+ // Everything below and until we get back to the select is
+ // non blocking, so time is effectively a constant. That is
+ // Why we just have to do this once, here
+ updateNow();
+ for (SelectionKey k : selected) {
+ SocketChannel sc = ((SocketChannel) k.channel());
+ if ((k.readyOps() & SelectionKey.OP_CONNECT) != 0) {
+ if (sc.finishConnect()) {
+ updateLastSendAndHeard();
+ updateSocketAddresses();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doTransport(...)` indirectly writes to field `this.localSocketAddress` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
+ // On initial connection, write the complete connect request
+ // packet, but then disable further writes until after
+ // receiving a successful connection response. If the
+ // session is expired, then the server sends the expiration
+ // response and immediately closes its end of the socket. If
+ // the client is simultaneously writing on its end, then the
+ // TCP stack may choose to abort with RST, in which case the
+ // client would never receive the session expired event. See
+ // http://docs.oracle.com/javase/6/docs/technotes/guides/net/articles/connection_release.html
+ disableWrite();
+ } else {
+ // Just in case
+ enableWrite();
+ }
+ }
+ }
+
+ private Packet findSendablePacket(LinkedBlockingDeque<Packet> outgoingQueue,
+ boolean tunneledAuthInProgres) {
+ if (outgoingQueue.isEmpty()) {
+ return null;
+ }
+ // If we've already starting sending the first packet, we better finish
+ if (outgoingQueue.getFirst().bb != null || !tunneledAuthInProgres) {
+ return outgoingQueue.getFirst();
+ }
+ // Since client's authentication with server is in progress,
+ // send only the null-header packet queued by primeConnection().
+ // This packet must be sent so that the SASL authentication process
+ // can proceed, but all other packets should wait until
+ // SASL authentication completes.
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p.requestHeader == null) {
+ // We've found the priming-packet. Move it to the beginning of the queue.
+ iter.remove();
+ outgoingQueue.addFirst(p);
+ return p;
+ } else {
+ // Non-priming packet: defer it until later, leaving it in the queue
+ // until authentication completes.
+ LOG.debug("deferring non-priming packet {} until SASL authentation completes.", p);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ void cleanup() {
+ if (sockKey != null) {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ sockKey.cancel();
+ try {
+ sock.socket().shutdownInput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown input", e);
+ }
+ }
+ try {
+ sock.socket().shutdownOutput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown output",
+ e);
+ }
+ }
+ try {
+ sock.socket().close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during socket close", e);
+ }
+ }
+ try {
+ sock.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during channel close", e);
+ }
+ }
+ }
+ try {
+ Thread.sleep(100);
+ } catch (InterruptedException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SendThread interrupted during sleep, ignoring");
+ }
+ }
+ sockKey = null;
+ }
+
+ @Override
+ void close() {
+ try {
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Doing client selector close");
+ }
+ selector.close();
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closed client selector");
+ }
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during selector close", e);
+ }
+ }
+
+ /**
+ * create a socket channel.
+ * @return the created socket channel
+ * @throws IOException
+ */
+ SocketChannel createSock() throws IOException {
+ SocketChannel sock;
+ sock = SocketChannel.open();
+ sock.configureBlocking(false);
+ sock.socket().setSoLinger(false, -1);
+ sock.socket().setTcpNoDelay(true);
+ return sock;
+ }
+
+ /**
+ * register with the selection and connect
+ * @param sock the {@link SocketChannel}
+ * @param addr the address of remote host
+ * @throws IOException
+ */
+ void registerAndConnect(SocketChannel sock, InetSocketAddress addr)
+ throws IOException {
+ sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
+ boolean immediateConnect = sock.connect(addr);
+ if (immediateConnect) {
+ sendThread.primeConnection();
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ SocketChannel sock = createSock();
+ try {
+ registerAndConnect(sock, addr);
+ } catch (IOException e) {
+ LOG.error("Unable to open socket to " + addr);
+ sock.close();
+ throw e;
+ }
+ initialized = false;
+
+ /*
+ * Reset incomingBuffer
+ */
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ }
+
+ /**
+ * Returns the address to which the socket is connected.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getRemoteSocketAddress() {
+ return remoteSocketAddress;
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getLocalSocketAddress() {
+ return localSocketAddress;
+ }
+
+ private void updateSocketAddresses() {
+ Socket socket = ((SocketChannel) sockKey.channel()).socket();
+ localSocketAddress = socket.getLocalSocketAddress();
+ remoteSocketAddress = socket.getRemoteSocketAddress();
+ }
+
+ @Override
+ void packetAdded() {
+ wakeupCnxn();
+ }
+
+ @Override
+ void onClosing() {
+ wakeupCnxn();
+ }
+
+ private synchronized void wakeupCnxn() {
+ selector.wakeup();
+ }
+
+ @Override
+ void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ selector.select(waitTimeOut);
+ Set<SelectionKey> selected;
+ synchronized (this) {
+ selected = selector.selectedKeys();
+ }
+ // Everything below and until we get back to the select is
+ // non blocking, so time is effectively a constant. That is
+ // Why we just have to do this once, here
+ updateNow();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doTransport(...)` indirectly writes to field `this.now` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNIO.java
##########
@@ -0,0 +1,432 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.nio.ByteBuffer;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.Selector;
+import java.nio.channels.SocketChannel;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingDeque;
+
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ClientCnxnSocketNIO extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(ClientCnxnSocketNIO.class);
+
+ private final Selector selector = Selector.open();
+
+ private SelectionKey sockKey;
+
+ private SocketAddress localSocketAddress;
+
+ private SocketAddress remoteSocketAddress;
+
+ ClientCnxnSocketNIO(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ initProperties();
+ }
+
+ @Override
+ boolean isConnected() {
+ return sockKey != null;
+ }
+
+ /**
+ * @return true if a packet was received
+ * @throws InterruptedException
+ * @throws IOException
+ */
+ void doIO(List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws InterruptedException, IOException {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ if (sock == null) {
+ throw new IOException("Socket is null!");
+ }
+ if (sockKey.isReadable()) {
+ int rc = sock.read(incomingBuffer);
+ if (rc < 0) {
+ throw new EndOfStreamException(
+ "Unable to read additional data from server sessionid 0x"
+ + Long.toHexString(sessionId)
+ + ", likely server has closed socket");
+ }
+ if (!incomingBuffer.hasRemaining()) {
+ incomingBuffer.flip();
+ if (incomingBuffer == lenBuffer) {
+ recvCount.getAndIncrement();
+ readLength();
+ } else if (!initialized) {
+ readConnectResult();
+ enableRead();
+ if (findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress()) != null) {
+ // Since SASL authentication has completed (if client is configured to do so),
+ // outgoing packets waiting in the outgoingQueue can now be sent.
+ enableWrite();
+ }
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ initialized = true;
+ } else {
+ sendThread.readResponse(incomingBuffer);
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ updateLastHeard();
+ }
+ }
+ }
+ if (sockKey.isWritable()) {
+ Packet p = findSendablePacket(outgoingQueue,
+ sendThread.tunnelAuthInProgress());
+
+ if (p != null) {
+ updateLastSend();
+ // If we already started writing p, p.bb will already exist
+ if (p.bb == null) {
+ if ((p.requestHeader != null) &&
+ (p.requestHeader.getType() != OpCode.ping) &&
+ (p.requestHeader.getType() != OpCode.auth)) {
+ p.requestHeader.setXid(cnxn.getXid());
+ }
+ p.createBB();
+ }
+ sock.write(p.bb);
+ if (!p.bb.hasRemaining()) {
+ sentCount.getAndIncrement();
+ outgoingQueue.removeFirstOccurrence(p);
+ if (p.requestHeader != null
+ && p.requestHeader.getType() != OpCode.ping
+ && p.requestHeader.getType() != OpCode.auth) {
+ synchronized (pendingQueue) {
+ pendingQueue.add(p);
+ }
+ }
+ }
+ }
+ if (outgoingQueue.isEmpty()) {
+ // No more packets to send: turn off write interest flag.
+ // Will be turned on later by a later call to enableWrite(),
+ // from within ZooKeeperSaslClient (if client is configured
+ // to attempt SASL authentication), or in either doIO() or
+ // in doTransport() if not.
+ disableWrite();
+ } else if (!initialized && p != null && !p.bb.hasRemaining()) {
+ // On initial connection, write the complete connect request
+ // packet, but then disable further writes until after
+ // receiving a successful connection response. If the
+ // session is expired, then the server sends the expiration
+ // response and immediately closes its end of the socket. If
+ // the client is simultaneously writing on its end, then the
+ // TCP stack may choose to abort with RST, in which case the
+ // client would never receive the session expired event. See
+ // http://docs.oracle.com/javase/6/docs/technotes/guides/net/articles/connection_release.html
+ disableWrite();
+ } else {
+ // Just in case
+ enableWrite();
+ }
+ }
+ }
+
+ private Packet findSendablePacket(LinkedBlockingDeque<Packet> outgoingQueue,
+ boolean tunneledAuthInProgres) {
+ if (outgoingQueue.isEmpty()) {
+ return null;
+ }
+ // If we've already starting sending the first packet, we better finish
+ if (outgoingQueue.getFirst().bb != null || !tunneledAuthInProgres) {
+ return outgoingQueue.getFirst();
+ }
+ // Since client's authentication with server is in progress,
+ // send only the null-header packet queued by primeConnection().
+ // This packet must be sent so that the SASL authentication process
+ // can proceed, but all other packets should wait until
+ // SASL authentication completes.
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p.requestHeader == null) {
+ // We've found the priming-packet. Move it to the beginning of the queue.
+ iter.remove();
+ outgoingQueue.addFirst(p);
+ return p;
+ } else {
+ // Non-priming packet: defer it until later, leaving it in the queue
+ // until authentication completes.
+ LOG.debug("deferring non-priming packet {} until SASL authentation completes.", p);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ void cleanup() {
+ if (sockKey != null) {
+ SocketChannel sock = (SocketChannel) sockKey.channel();
+ sockKey.cancel();
+ try {
+ sock.socket().shutdownInput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown input", e);
+ }
+ }
+ try {
+ sock.socket().shutdownOutput();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during shutdown output",
+ e);
+ }
+ }
+ try {
+ sock.socket().close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during socket close", e);
+ }
+ }
+ try {
+ sock.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring exception during channel close", e);
+ }
+ }
+ }
+ try {
+ Thread.sleep(100);
+ } catch (InterruptedException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("SendThread interrupted during sleep, ignoring");
+ }
+ }
+ sockKey = null;
+ }
+
+ @Override
+ void close() {
+ try {
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Doing client selector close");
+ }
+ selector.close();
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Closed client selector");
+ }
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during selector close", e);
+ }
+ }
+
+ /**
+ * create a socket channel.
+ * @return the created socket channel
+ * @throws IOException
+ */
+ SocketChannel createSock() throws IOException {
+ SocketChannel sock;
+ sock = SocketChannel.open();
+ sock.configureBlocking(false);
+ sock.socket().setSoLinger(false, -1);
+ sock.socket().setTcpNoDelay(true);
+ return sock;
+ }
+
+ /**
+ * register with the selection and connect
+ * @param sock the {@link SocketChannel}
+ * @param addr the address of remote host
+ * @throws IOException
+ */
+ void registerAndConnect(SocketChannel sock, InetSocketAddress addr)
+ throws IOException {
+ sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
+ boolean immediateConnect = sock.connect(addr);
+ if (immediateConnect) {
+ sendThread.primeConnection();
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ SocketChannel sock = createSock();
+ try {
+ registerAndConnect(sock, addr);
+ } catch (IOException e) {
+ LOG.error("Unable to open socket to " + addr);
+ sock.close();
+ throw e;
+ }
+ initialized = false;
+
+ /*
+ * Reset incomingBuffer
+ */
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+ }
+
+ /**
+ * Returns the address to which the socket is connected.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getRemoteSocketAddress() {
+ return remoteSocketAddress;
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ *
+ * @return ip address of the remote side of the connection or null if not
+ * connected
+ */
+ @Override
+ SocketAddress getLocalSocketAddress() {
+ return localSocketAddress;
+ }
+
+ private void updateSocketAddresses() {
+ Socket socket = ((SocketChannel) sockKey.channel()).socket();
+ localSocketAddress = socket.getLocalSocketAddress();
+ remoteSocketAddress = socket.getRemoteSocketAddress();
+ }
+
+ @Override
+ void packetAdded() {
+ wakeupCnxn();
+ }
+
+ @Override
+ void onClosing() {
+ wakeupCnxn();
+ }
+
+ private synchronized void wakeupCnxn() {
+ selector.wakeup();
+ }
+
+ @Override
+ void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ selector.select(waitTimeOut);
+ Set<SelectionKey> selected;
+ synchronized (this) {
+ selected = selector.selectedKeys();
+ }
+ // Everything below and until we get back to the select is
+ // non blocking, so time is effectively a constant. That is
+ // Why we just have to do this once, here
+ updateNow();
+ for (SelectionKey k : selected) {
+ SocketChannel sc = ((SocketChannel) k.channel());
+ if ((k.readyOps() & SelectionKey.OP_CONNECT) != 0) {
+ if (sc.finishConnect()) {
+ updateLastSendAndHeard();
+ updateSocketAddresses();
+ sendThread.primeConnection();
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ClientCnxnSocketNIO.doTransport(...)` indirectly writes to field `this.sendThread.isFirstConnect` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNetty.java
##########
@@ -0,0 +1,551 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.Iterator;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLEngine;
+
+import io.netty.bootstrap.Bootstrap;
+import io.netty.buffer.ByteBuf;
+import io.netty.buffer.ByteBufAllocator;
+import io.netty.buffer.Unpooled;
+import io.netty.channel.Channel;
+import io.netty.channel.ChannelFuture;
+import io.netty.channel.ChannelFutureListener;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.channel.ChannelInitializer;
+import io.netty.channel.ChannelOption;
+import io.netty.channel.ChannelPipeline;
+import io.netty.channel.EventLoopGroup;
+import io.netty.channel.SimpleChannelInboundHandler;
+import io.netty.channel.socket.SocketChannel;
+import io.netty.handler.ssl.SslHandler;
+import io.netty.util.concurrent.Future;
+import io.netty.util.concurrent.GenericFutureListener;
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.apache.zookeeper.common.NettyUtils;
+import org.apache.zookeeper.common.X509Util;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.apache.zookeeper.common.X509Exception.SSLContextException;
+
+/**
+ * ClientCnxnSocketNetty implements ClientCnxnSocket abstract methods.
+ * It's responsible for connecting to server, reading/writing network traffic and
+ * being a layer between network data and higher level packets.
+ */
+public class ClientCnxnSocketNetty extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory.getLogger(ClientCnxnSocketNetty.class);
+
+ private final EventLoopGroup eventLoopGroup;
+ private Channel channel;
+ private CountDownLatch firstConnect;
+ private ChannelFuture connectFuture;
+ private final Lock connectLock = new ReentrantLock();
+ private final AtomicBoolean disconnected = new AtomicBoolean();
+ private final AtomicBoolean needSasl = new AtomicBoolean();
+ private final Semaphore waitSasl = new Semaphore(0);
+
+ private static final AtomicReference<ByteBufAllocator> TEST_ALLOCATOR =
+ new AtomicReference<>(null);
+
+ ClientCnxnSocketNetty(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ // Client only has 1 outgoing socket, so the event loop group only needs
+ // a single thread.
+ eventLoopGroup = NettyUtils.newNioOrEpollEventLoopGroup(1 /* nThreads */);
+ initProperties();
+ }
+
+ /**
+ * lifecycles diagram:
+ * <p/>
+ * loop:
+ * - try:
+ * - - !isConnected()
+ * - - - connect()
+ * - - doTransport()
+ * - catch:
+ * - - cleanup()
+ * close()
+ * <p/>
+ * Other non-lifecycle methods are in jeopardy getting a null channel
+ * when calling in concurrency. We must handle it.
+ */
+
+ @Override
+ boolean isConnected() {
+ // Assuming that isConnected() is only used to initiate connection,
+ // not used by some other connection status judgement.
+ connectLock.lock();
+ try {
+ return channel != null || connectFuture != null;
+ } finally {
+ connectLock.unlock();
+ }
+ }
+
+ private Bootstrap configureBootstrapAllocator(Bootstrap bootstrap) {
+ ByteBufAllocator testAllocator = TEST_ALLOCATOR.get();
+ if (testAllocator != null) {
+ return bootstrap.option(ChannelOption.ALLOCATOR, testAllocator);
+ } else {
+ return bootstrap;
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ firstConnect = new CountDownLatch(1);
+
+ Bootstrap bootstrap = new Bootstrap()
+ .group(eventLoopGroup)
+ .channel(NettyUtils.nioOrEpollSocketChannel())
+ .option(ChannelOption.SO_LINGER, -1)
+ .option(ChannelOption.TCP_NODELAY, true)
+ .handler(new ZKClientPipelineFactory(addr.getHostString(), addr.getPort()));
+ bootstrap = configureBootstrapAllocator(bootstrap);
+ bootstrap.validate();
+
+ connectLock.lock();
+ try {
+ connectFuture = bootstrap.connect(addr);
+ connectFuture.addListener(new ChannelFutureListener() {
+ @Override
+ public void operationComplete(ChannelFuture channelFuture) throws Exception {
+ // this lock guarantees that channel won't be assigned after cleanup().
+ boolean connected = false;
+ connectLock.lock();
+ try {
+ if (!channelFuture.isSuccess()) {
+ LOG.info("future isn't success, cause:", channelFuture.cause());
+ return;
+ } else if (connectFuture == null) {
+ LOG.info("connect attempt cancelled");
+ // If the connect attempt was cancelled but succeeded
+ // anyway, make sure to close the channel, otherwise
+ // we may leak a file descriptor.
+ channelFuture.channel().close();
+ return;
+ }
+ // setup channel, variables, connection, etc.
+ channel = channelFuture.channel();
+
+ disconnected.set(false);
+ initialized = false;
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+
+ sendThread.primeConnection();
+ updateNow();
+ updateLastSendAndHeard();
+
+ if (sendThread.tunnelAuthInProgress()) {
+ waitSasl.drainPermits();
+ needSasl.set(true);
+ sendPrimePacket();
+ } else {
+ needSasl.set(false);
+ }
+ connected = true;
+ } finally {
+ connectFuture = null;
+ connectLock.unlock();
+ if (connected) {
+ LOG.info("channel is connected: {}", channelFuture.channel());
+ }
+ // need to wake on connect success or failure to avoid
+ // timing out ClientCnxn.SendThread which may be
+ // blocked waiting for first connect in doTransport().
+ wakeupCnxn();
+ firstConnect.countDown();
+ }
+ }
+ });
+ } finally {
+ connectLock.unlock();
+ }
+ }
+
+ @Override
+ void cleanup() {
+ connectLock.lock();
+ try {
+ if (connectFuture != null) {
+ connectFuture.cancel(false);
+ connectFuture = null;
+ }
+ if (channel != null) {
+ channel.close().syncUninterruptibly();
+ channel = null;
+ }
+ } finally {
+ connectLock.unlock();
+ }
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p == WakeupPacket.getInstance()) {
+ iter.remove();
+ }
+ }
+ }
+
+ @Override
+ void close() {
+ eventLoopGroup.shutdownGracefully();
+ }
+
+ @Override
+ void saslCompleted() {
+ needSasl.set(false);
+ waitSasl.release();
+ }
+
+ @Override
+ void connectionPrimed() {
+ }
+
+ @Override
+ void packetAdded() {
+ // NO-OP. Adding a packet will already wake up a netty connection
+ // so we don't need to add a dummy packet to the queue to trigger
+ // a wake-up.
+ }
+
+ @Override
+ void onClosing() {
+ firstConnect.countDown();
+ wakeupCnxn();
+ LOG.info("channel is told closing");
+ }
+
+ private void wakeupCnxn() {
+ if (needSasl.get()) {
+ waitSasl.release();
+ }
+ outgoingQueue.add(WakeupPacket.getInstance());
+ }
+
+ @Override
+ void doTransport(int waitTimeOut,
+ List<Packet> pendingQueue,
+ ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ try {
+ if (!firstConnect.await(waitTimeOut, TimeUnit.MILLISECONDS)) {
+ return;
+ }
+ Packet head = null;
+ if (needSasl.get()) {
+ if (!waitSasl.tryAcquire(waitTimeOut, TimeUnit.MILLISECONDS)) {
+ return;
+ }
+ } else {
+ head = outgoingQueue.poll(waitTimeOut, TimeUnit.MILLISECONDS);
+ }
+ // check if being waken up on closing.
+ if (!sendThread.getZkState().isAlive()) {
+ // adding back the packet to notify of failure in conLossPacket().
+ addBack(head);
+ return;
+ }
+ // channel disconnection happened
+ if (disconnected.get()) {
+ addBack(head);
+ throw new EndOfStreamException("channel for sessionid 0x"
+ + Long.toHexString(sessionId)
+ + " is lost");
+ }
+ if (head != null) {
+ doWrite(pendingQueue, head, cnxn);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNetty.doTransport(...)` indirectly reads without synchronization from `this.channel`. Potentially races with write in method `ClientCnxnSocketNetty.cleanup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNetty.java
##########
@@ -0,0 +1,551 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.Iterator;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Semaphore;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLEngine;
+
+import io.netty.bootstrap.Bootstrap;
+import io.netty.buffer.ByteBuf;
+import io.netty.buffer.ByteBufAllocator;
+import io.netty.buffer.Unpooled;
+import io.netty.channel.Channel;
+import io.netty.channel.ChannelFuture;
+import io.netty.channel.ChannelFutureListener;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.channel.ChannelInitializer;
+import io.netty.channel.ChannelOption;
+import io.netty.channel.ChannelPipeline;
+import io.netty.channel.EventLoopGroup;
+import io.netty.channel.SimpleChannelInboundHandler;
+import io.netty.channel.socket.SocketChannel;
+import io.netty.handler.ssl.SslHandler;
+import io.netty.util.concurrent.Future;
+import io.netty.util.concurrent.GenericFutureListener;
+import org.apache.zookeeper.ClientCnxn.EndOfStreamException;
+import org.apache.zookeeper.ClientCnxn.Packet;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.apache.zookeeper.common.NettyUtils;
+import org.apache.zookeeper.common.X509Util;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.apache.zookeeper.common.X509Exception.SSLContextException;
+
+/**
+ * ClientCnxnSocketNetty implements ClientCnxnSocket abstract methods.
+ * It's responsible for connecting to server, reading/writing network traffic and
+ * being a layer between network data and higher level packets.
+ */
+public class ClientCnxnSocketNetty extends ClientCnxnSocket {
+ private static final Logger LOG = LoggerFactory.getLogger(ClientCnxnSocketNetty.class);
+
+ private final EventLoopGroup eventLoopGroup;
+ private Channel channel;
+ private CountDownLatch firstConnect;
+ private ChannelFuture connectFuture;
+ private final Lock connectLock = new ReentrantLock();
+ private final AtomicBoolean disconnected = new AtomicBoolean();
+ private final AtomicBoolean needSasl = new AtomicBoolean();
+ private final Semaphore waitSasl = new Semaphore(0);
+
+ private static final AtomicReference<ByteBufAllocator> TEST_ALLOCATOR =
+ new AtomicReference<>(null);
+
+ ClientCnxnSocketNetty(ZKClientConfig clientConfig) throws IOException {
+ this.clientConfig = clientConfig;
+ // Client only has 1 outgoing socket, so the event loop group only needs
+ // a single thread.
+ eventLoopGroup = NettyUtils.newNioOrEpollEventLoopGroup(1 /* nThreads */);
+ initProperties();
+ }
+
+ /**
+ * lifecycles diagram:
+ * <p/>
+ * loop:
+ * - try:
+ * - - !isConnected()
+ * - - - connect()
+ * - - doTransport()
+ * - catch:
+ * - - cleanup()
+ * close()
+ * <p/>
+ * Other non-lifecycle methods are in jeopardy getting a null channel
+ * when calling in concurrency. We must handle it.
+ */
+
+ @Override
+ boolean isConnected() {
+ // Assuming that isConnected() is only used to initiate connection,
+ // not used by some other connection status judgement.
+ connectLock.lock();
+ try {
+ return channel != null || connectFuture != null;
+ } finally {
+ connectLock.unlock();
+ }
+ }
+
+ private Bootstrap configureBootstrapAllocator(Bootstrap bootstrap) {
+ ByteBufAllocator testAllocator = TEST_ALLOCATOR.get();
+ if (testAllocator != null) {
+ return bootstrap.option(ChannelOption.ALLOCATOR, testAllocator);
+ } else {
+ return bootstrap;
+ }
+ }
+
+ @Override
+ void connect(InetSocketAddress addr) throws IOException {
+ firstConnect = new CountDownLatch(1);
+
+ Bootstrap bootstrap = new Bootstrap()
+ .group(eventLoopGroup)
+ .channel(NettyUtils.nioOrEpollSocketChannel())
+ .option(ChannelOption.SO_LINGER, -1)
+ .option(ChannelOption.TCP_NODELAY, true)
+ .handler(new ZKClientPipelineFactory(addr.getHostString(), addr.getPort()));
+ bootstrap = configureBootstrapAllocator(bootstrap);
+ bootstrap.validate();
+
+ connectLock.lock();
+ try {
+ connectFuture = bootstrap.connect(addr);
+ connectFuture.addListener(new ChannelFutureListener() {
+ @Override
+ public void operationComplete(ChannelFuture channelFuture) throws Exception {
+ // this lock guarantees that channel won't be assigned after cleanup().
+ boolean connected = false;
+ connectLock.lock();
+ try {
+ if (!channelFuture.isSuccess()) {
+ LOG.info("future isn't success, cause:", channelFuture.cause());
+ return;
+ } else if (connectFuture == null) {
+ LOG.info("connect attempt cancelled");
+ // If the connect attempt was cancelled but succeeded
+ // anyway, make sure to close the channel, otherwise
+ // we may leak a file descriptor.
+ channelFuture.channel().close();
+ return;
+ }
+ // setup channel, variables, connection, etc.
+ channel = channelFuture.channel();
+
+ disconnected.set(false);
+ initialized = false;
+ lenBuffer.clear();
+ incomingBuffer = lenBuffer;
+
+ sendThread.primeConnection();
+ updateNow();
+ updateLastSendAndHeard();
+
+ if (sendThread.tunnelAuthInProgress()) {
+ waitSasl.drainPermits();
+ needSasl.set(true);
+ sendPrimePacket();
+ } else {
+ needSasl.set(false);
+ }
+ connected = true;
+ } finally {
+ connectFuture = null;
+ connectLock.unlock();
+ if (connected) {
+ LOG.info("channel is connected: {}", channelFuture.channel());
+ }
+ // need to wake on connect success or failure to avoid
+ // timing out ClientCnxn.SendThread which may be
+ // blocked waiting for first connect in doTransport().
+ wakeupCnxn();
+ firstConnect.countDown();
+ }
+ }
+ });
+ } finally {
+ connectLock.unlock();
+ }
+ }
+
+ @Override
+ void cleanup() {
+ connectLock.lock();
+ try {
+ if (connectFuture != null) {
+ connectFuture.cancel(false);
+ connectFuture = null;
+ }
+ if (channel != null) {
+ channel.close().syncUninterruptibly();
+ channel = null;
+ }
+ } finally {
+ connectLock.unlock();
+ }
+ Iterator<Packet> iter = outgoingQueue.iterator();
+ while (iter.hasNext()) {
+ Packet p = iter.next();
+ if (p == WakeupPacket.getInstance()) {
+ iter.remove();
+ }
+ }
+ }
+
+ @Override
+ void close() {
+ eventLoopGroup.shutdownGracefully();
+ }
+
+ @Override
+ void saslCompleted() {
+ needSasl.set(false);
+ waitSasl.release();
+ }
+
+ @Override
+ void connectionPrimed() {
+ }
+
+ @Override
+ void packetAdded() {
+ // NO-OP. Adding a packet will already wake up a netty connection
+ // so we don't need to add a dummy packet to the queue to trigger
+ // a wake-up.
+ }
+
+ @Override
+ void onClosing() {
+ firstConnect.countDown();
+ wakeupCnxn();
+ LOG.info("channel is told closing");
+ }
+
+ private void wakeupCnxn() {
+ if (needSasl.get()) {
+ waitSasl.release();
+ }
+ outgoingQueue.add(WakeupPacket.getInstance());
+ }
+
+ @Override
+ void doTransport(int waitTimeOut,
+ List<Packet> pendingQueue,
+ ClientCnxn cnxn)
+ throws IOException, InterruptedException {
+ try {
+ if (!firstConnect.await(waitTimeOut, TimeUnit.MILLISECONDS)) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ClientCnxnSocketNetty.doTransport(...)` reads without synchronization from `this.firstConnect`. Potentially races with write in method `ClientCnxnSocketNetty.connect(...)`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()));
+ }
+
+ List<OpResult> results = response.getResultList();
+
+ ErrorResult fatalError = null;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult && ((ErrorResult)result).getErr() != KeeperException.Code.OK.intValue()) {
+ fatalError = (ErrorResult) result;
+ break;
+ }
+ }
+
+ if (fatalError != null) {
+ KeeperException ex = KeeperException.create(KeeperException.Code.get(fatalError.getErr()));
+ ex.setMultiResults(results);
+ throw ex;
+ }
+
+ return results;
+ }
+
+ /**
+ * A Transaction is a thin wrapper on the {@link #multi} method
+ * which provides a builder object that can be used to construct
+ * and commit an atomic set of operations.
+ *
+ * @since 3.4.0
+ *
+ * @return a Transaction builder object
+ */
+ public Transaction transaction() {
+ return new Transaction(this);
+ }
+
+ /**
+ * The asynchronous version of delete.
+ *
+ * @see #delete(String, int)
+ */
+ public void delete(final String path, int version, VoidCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path the node path
+ * @param watcher explicit watcher
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat exists(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ if (r.getErr() == KeeperException.Code.NONODE.intValue()) {
+ return null;
+ }
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+
+ return response.getStat().getCzxid() == -1 ? null : response.getStat();
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path
+ * the node path
+ * @param watch
+ * whether need to watch this node
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public Stat exists(String path, boolean watch) throws KeeperException,
+ InterruptedException
+ {
+ return exists(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, Watcher)
+ */
+ public void exists(final String path, Watcher watcher,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, boolean)
+ */
+ public void exists(String path, boolean watch, StatCallback cb, Object ctx) {
+ exists(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watcher explicit watcher
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public byte[] getData(final String path, Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watch whether need to watch this node
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getData(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getData(path, watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, Watcher, Stat)
+ */
+ public void getData(final String path, Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getData(String path, boolean watch, DataCallback cb, Object ctx) {
+ getData(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the configuration node doesn't exists.
+ *
+ * @param watcher explicit watcher
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ configZnode);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * The asynchronous version of getConfig.
+ *
+ * @see #getConfig(Watcher, Stat)
+ */
+ public void getConfig(Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ configZnode, configZnode, ctx, wcb);
+ }
+
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param watch whether need to watch this node
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getConfig(watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The Asynchronous version of getConfig.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getConfig(boolean watch, DataCallback cb, Object ctx) {
+ getConfig(watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(List<String> joiningServers, List<String> leavingServers, List<String> newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(String joiningServers, String leavingServers,
+ String newMembers, long fromConfig, DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(List<String> joiningServers,
+ List<String> leavingServers, List<String> newMembers, long fromConfig,
+ DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * Set the data for the node of the given path if such a node exists and the
+ * given version matches the version of the node (if the given version is
+ * -1, it matches any node's versions). Return the stat of the node.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by getData calls.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperException to be thrown.
+ *
+ * @param path
+ * the path of the node
+ * @param data
+ * the data to set
+ * @param version
+ * the expected matching version
+ * @return the state of the node
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setData(final String path, byte data[], int version)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setData.
+ *
+ * @see #setData(String, byte[], int)
+ */
+ public void setData(final String path, byte data[], int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the ACL and stat of the node of the given path.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * the given path for the node
+ * @param stat
+ * the stat of the node will be copied to this parameter if
+ * not null.
+ * @return the ACL array of the given node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<ACL> getACL(final String path, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getAcl();
+ }
+
+ /**
+ * The asynchronous version of getACL.
+ *
+ * @see #getACL(String, Stat)
+ */
+ public void getACL(final String path, Stat stat, ACLCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Set the ACL for the node of the given path if such a node exists and the
+ * given aclVersion matches the acl version of the node. Return the stat of the
+ * node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given aclVersion does not match the node's aclVersion.
+ *
+ * @param path the given path for the node
+ * @param acl the given acl for the node
+ * @param aclVersion the given acl version of the node
+ * @return the stat of the node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws org.apache.zookeeper.KeeperException.InvalidACLException If the acl is invalide.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setACL(final String path, List<ACL> acl, int aclVersion)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException(clientPath);
+ }
+ request.setAcl(acl);
+ request.setVersion(aclVersion);
+ SetACLResponse response = new SetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setACL.
+ *
+ * @see #setACL(String, List, int)
+ */
+ public void setACL(final String path, List<ACL> acl, int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setVersion(version);
+ SetACLResponse response = new SetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watch
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ */
+ public List<String> getChildren(String path, boolean watch)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, Watcher)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ ChildrenCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, boolean)
+ */
+ public void getChildren(String path, boolean watch, ChildrenCallback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher,
+ Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watch
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero
+ * error code.
+ */
+ public List<String> getChildren(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null,
+ stat);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, Watcher, Stat)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ Children2Callback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, boolean, Stat)
+ */
+ public void getChildren(String path, boolean watch, Children2Callback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Asynchronous sync. Flushes channel between process and leader.
+ * @param path
+ * @param cb a handler for the callback
+ * @param ctx context to be provided to the callback
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void sync(final String path, VoidCallback cb, Object ctx){
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.sync);
+ SyncRequest request = new SyncRequest();
+ SyncResponse response = new SyncResponse();
+ request.setPath(serverPath);
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * For the given znode path, removes the specified watcher of given
+ * watcherType.
+ *
+ * <p>
+ * Watcher shouldn't be null. A successful call guarantees that, the
+ * removed watcher won't be triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcher
+ * - a concrete watcher
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether the watcher can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if any of the following is true:
+ * <ul>
+ * <li> {@code path} is invalid
+ * <li> {@code watcher} is null
+ * </ul>
+ *
+ * @since 3.5.0
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local);
+ }
+
+ /**
+ * The asynchronous version of removeWatches.
+ *
+ * @see #removeWatches
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local, cb, ctx);
+ }
+
+ /**
+ * For the given znode path, removes all the registered watchers of given
+ * watcherType.
+ *
+ * <p>
+ * A successful call guarantees that, the removed watchers won't be
+ * triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether watches can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if an invalid {@code path} is specified
+ *
+ * @since 3.5.0
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local) throws InterruptedException, KeeperException {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null, watcherType,
+ local);
+ }
+
+ /**
+ * The asynchronous version of removeAllWatches.
+ *
+ * @see #removeAllWatches
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local, VoidCallback cb, Object ctx) {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null,
+ watcherType, local, cb, ctx);
+ }
+
+ private void validateWatcher(Watcher watcher) {
+ if (watcher == null) {
+ throw new IllegalArgumentException(
+ "Invalid Watcher, shouldn't be null!");
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null, wcb);
+ }
+
+ private Record getRemoveWatchesRequest(int opCode, WatcherType watcherType,
+ final String serverPath) {
+ Record request = null;
+ switch (opCode) {
+ case ZooDefs.OpCode.checkWatches:
+ CheckWatchesRequest chkReq = new CheckWatchesRequest();
+ chkReq.setPath(serverPath);
+ chkReq.setType(watcherType.getIntValue());
+ request = chkReq;
+ break;
+ case ZooDefs.OpCode.removeWatches:
+ RemoveWatchesRequest rmReq = new RemoveWatchesRequest();
+ rmReq.setPath(serverPath);
+ rmReq.setType(watcherType.getIntValue());
+ request = rmReq;
+ break;
+ default:
+ LOG.warn("unknown type " + opCode);
+ break;
+ }
+ return request;
+ }
+
+ public States getState() {
+ return cnxn.getState();
+ }
+
+ /**
+ * String representation of this ZooKeeper client. Suitable for things
+ * like logging.
+ *
+ * Do NOT count on the format of this string, it may change without
+ * warning.
+ *
+ * @since 3.3.0
+ */
+ @Override
+ public String toString() {
+ States state = getState();
+ return ("State:" + state.toString()
+ + (state.isConnected() ?
+ " Timeout:" + getSessionTimeout() + " " :
+ " ")
+ + cnxn);
+ }
+
+ /*
+ * Methods to aid in testing follow.
+ *
+ * THESE METHODS ARE EXPECTED TO BE USED FOR TESTING ONLY!!!
+ */
+
+ /**
+ * Wait up to wait milliseconds for the underlying threads to shutdown.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @param wait max wait in milliseconds
+ * @return true iff all threads are shutdown, otw false
+ */
+ protected boolean testableWaitForShutdown(int wait)
+ throws InterruptedException
+ {
+ cnxn.sendThread.join(wait);
+ if (cnxn.sendThread.isAlive()) return false;
+ cnxn.eventThread.join(wait);
+ if (cnxn.eventThread.isAlive()) return false;
+ return true;
+ }
+
+ /**
+ * Returns the address to which the socket is connected. Useful for testing
+ * against an ensemble - test client may need to know which server
+ * to shutdown if interested in verifying that the code handles
+ * disconnection/reconnection correctly.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableRemoteSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getRemoteSocketAddress();
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableLocalSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getLocalSocketAddress();
+ }
+
+ private ClientCnxnSocket getClientCnxnSocket() throws IOException {
+ String clientCnxnSocketName = getClientConfig().getProperty(
+ ZKClientConfig.ZOOKEEPER_CLIENT_CNXN_SOCKET);
+ if (clientCnxnSocketName == null) {
+ clientCnxnSocketName = ClientCnxnSocketNIO.class.getName();
+ }
+ try {
+ Constructor<?> clientCxnConstructor = Class.forName(clientCnxnSocketName).getDeclaredConstructor(ZKClientConfig.class);
+ ClientCnxnSocket clientCxnSocket = (ClientCnxnSocket) clientCxnConstructor.newInstance(getClientConfig());
+ return clientCxnSocket;
+ } catch (Exception e) {
+ IOException ioe = new IOException("Couldn't instantiate "
+ + clientCnxnSocketName);
+ ioe.initCause(e);
+ throw ioe;
+ }
+ }
+
+ protected byte[] internalReconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.reconfig);
+ ReconfigRequest request = new ReconfigRequest(joiningServers, leavingServers, newMembers, fromConfig);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeper.internalReconfig(...)` indirectly writes to field `this.cnxn.sendThread.clientCnxnSocket.lastHeard` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()));
+ }
+
+ List<OpResult> results = response.getResultList();
+
+ ErrorResult fatalError = null;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult && ((ErrorResult)result).getErr() != KeeperException.Code.OK.intValue()) {
+ fatalError = (ErrorResult) result;
+ break;
+ }
+ }
+
+ if (fatalError != null) {
+ KeeperException ex = KeeperException.create(KeeperException.Code.get(fatalError.getErr()));
+ ex.setMultiResults(results);
+ throw ex;
+ }
+
+ return results;
+ }
+
+ /**
+ * A Transaction is a thin wrapper on the {@link #multi} method
+ * which provides a builder object that can be used to construct
+ * and commit an atomic set of operations.
+ *
+ * @since 3.4.0
+ *
+ * @return a Transaction builder object
+ */
+ public Transaction transaction() {
+ return new Transaction(this);
+ }
+
+ /**
+ * The asynchronous version of delete.
+ *
+ * @see #delete(String, int)
+ */
+ public void delete(final String path, int version, VoidCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path the node path
+ * @param watcher explicit watcher
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat exists(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ if (r.getErr() == KeeperException.Code.NONODE.intValue()) {
+ return null;
+ }
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+
+ return response.getStat().getCzxid() == -1 ? null : response.getStat();
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path
+ * the node path
+ * @param watch
+ * whether need to watch this node
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public Stat exists(String path, boolean watch) throws KeeperException,
+ InterruptedException
+ {
+ return exists(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, Watcher)
+ */
+ public void exists(final String path, Watcher watcher,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, boolean)
+ */
+ public void exists(String path, boolean watch, StatCallback cb, Object ctx) {
+ exists(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watcher explicit watcher
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public byte[] getData(final String path, Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watch whether need to watch this node
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getData(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getData(path, watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, Watcher, Stat)
+ */
+ public void getData(final String path, Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getData(String path, boolean watch, DataCallback cb, Object ctx) {
+ getData(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the configuration node doesn't exists.
+ *
+ * @param watcher explicit watcher
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ configZnode);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * The asynchronous version of getConfig.
+ *
+ * @see #getConfig(Watcher, Stat)
+ */
+ public void getConfig(Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ configZnode, configZnode, ctx, wcb);
+ }
+
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param watch whether need to watch this node
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getConfig(watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The Asynchronous version of getConfig.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getConfig(boolean watch, DataCallback cb, Object ctx) {
+ getConfig(watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(List<String> joiningServers, List<String> leavingServers, List<String> newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(String joiningServers, String leavingServers,
+ String newMembers, long fromConfig, DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(List<String> joiningServers,
+ List<String> leavingServers, List<String> newMembers, long fromConfig,
+ DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * Set the data for the node of the given path if such a node exists and the
+ * given version matches the version of the node (if the given version is
+ * -1, it matches any node's versions). Return the stat of the node.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by getData calls.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperException to be thrown.
+ *
+ * @param path
+ * the path of the node
+ * @param data
+ * the data to set
+ * @param version
+ * the expected matching version
+ * @return the state of the node
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setData(final String path, byte data[], int version)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setData.
+ *
+ * @see #setData(String, byte[], int)
+ */
+ public void setData(final String path, byte data[], int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the ACL and stat of the node of the given path.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * the given path for the node
+ * @param stat
+ * the stat of the node will be copied to this parameter if
+ * not null.
+ * @return the ACL array of the given node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<ACL> getACL(final String path, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getAcl();
+ }
+
+ /**
+ * The asynchronous version of getACL.
+ *
+ * @see #getACL(String, Stat)
+ */
+ public void getACL(final String path, Stat stat, ACLCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Set the ACL for the node of the given path if such a node exists and the
+ * given aclVersion matches the acl version of the node. Return the stat of the
+ * node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given aclVersion does not match the node's aclVersion.
+ *
+ * @param path the given path for the node
+ * @param acl the given acl for the node
+ * @param aclVersion the given acl version of the node
+ * @return the stat of the node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws org.apache.zookeeper.KeeperException.InvalidACLException If the acl is invalide.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setACL(final String path, List<ACL> acl, int aclVersion)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException(clientPath);
+ }
+ request.setAcl(acl);
+ request.setVersion(aclVersion);
+ SetACLResponse response = new SetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setACL.
+ *
+ * @see #setACL(String, List, int)
+ */
+ public void setACL(final String path, List<ACL> acl, int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setVersion(version);
+ SetACLResponse response = new SetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watch
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ */
+ public List<String> getChildren(String path, boolean watch)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, Watcher)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ ChildrenCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, boolean)
+ */
+ public void getChildren(String path, boolean watch, ChildrenCallback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher,
+ Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watch
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero
+ * error code.
+ */
+ public List<String> getChildren(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null,
+ stat);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, Watcher, Stat)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ Children2Callback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, boolean, Stat)
+ */
+ public void getChildren(String path, boolean watch, Children2Callback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Asynchronous sync. Flushes channel between process and leader.
+ * @param path
+ * @param cb a handler for the callback
+ * @param ctx context to be provided to the callback
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void sync(final String path, VoidCallback cb, Object ctx){
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.sync);
+ SyncRequest request = new SyncRequest();
+ SyncResponse response = new SyncResponse();
+ request.setPath(serverPath);
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * For the given znode path, removes the specified watcher of given
+ * watcherType.
+ *
+ * <p>
+ * Watcher shouldn't be null. A successful call guarantees that, the
+ * removed watcher won't be triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcher
+ * - a concrete watcher
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether the watcher can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if any of the following is true:
+ * <ul>
+ * <li> {@code path} is invalid
+ * <li> {@code watcher} is null
+ * </ul>
+ *
+ * @since 3.5.0
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local);
+ }
+
+ /**
+ * The asynchronous version of removeWatches.
+ *
+ * @see #removeWatches
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local, cb, ctx);
+ }
+
+ /**
+ * For the given znode path, removes all the registered watchers of given
+ * watcherType.
+ *
+ * <p>
+ * A successful call guarantees that, the removed watchers won't be
+ * triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether watches can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if an invalid {@code path} is specified
+ *
+ * @since 3.5.0
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local) throws InterruptedException, KeeperException {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null, watcherType,
+ local);
+ }
+
+ /**
+ * The asynchronous version of removeAllWatches.
+ *
+ * @see #removeAllWatches
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local, VoidCallback cb, Object ctx) {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null,
+ watcherType, local, cb, ctx);
+ }
+
+ private void validateWatcher(Watcher watcher) {
+ if (watcher == null) {
+ throw new IllegalArgumentException(
+ "Invalid Watcher, shouldn't be null!");
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null, wcb);
+ }
+
+ private Record getRemoveWatchesRequest(int opCode, WatcherType watcherType,
+ final String serverPath) {
+ Record request = null;
+ switch (opCode) {
+ case ZooDefs.OpCode.checkWatches:
+ CheckWatchesRequest chkReq = new CheckWatchesRequest();
+ chkReq.setPath(serverPath);
+ chkReq.setType(watcherType.getIntValue());
+ request = chkReq;
+ break;
+ case ZooDefs.OpCode.removeWatches:
+ RemoveWatchesRequest rmReq = new RemoveWatchesRequest();
+ rmReq.setPath(serverPath);
+ rmReq.setType(watcherType.getIntValue());
+ request = rmReq;
+ break;
+ default:
+ LOG.warn("unknown type " + opCode);
+ break;
+ }
+ return request;
+ }
+
+ public States getState() {
+ return cnxn.getState();
+ }
+
+ /**
+ * String representation of this ZooKeeper client. Suitable for things
+ * like logging.
+ *
+ * Do NOT count on the format of this string, it may change without
+ * warning.
+ *
+ * @since 3.3.0
+ */
+ @Override
+ public String toString() {
+ States state = getState();
+ return ("State:" + state.toString()
+ + (state.isConnected() ?
+ " Timeout:" + getSessionTimeout() + " " :
+ " ")
+ + cnxn);
+ }
+
+ /*
+ * Methods to aid in testing follow.
+ *
+ * THESE METHODS ARE EXPECTED TO BE USED FOR TESTING ONLY!!!
+ */
+
+ /**
+ * Wait up to wait milliseconds for the underlying threads to shutdown.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @param wait max wait in milliseconds
+ * @return true iff all threads are shutdown, otw false
+ */
+ protected boolean testableWaitForShutdown(int wait)
+ throws InterruptedException
+ {
+ cnxn.sendThread.join(wait);
+ if (cnxn.sendThread.isAlive()) return false;
+ cnxn.eventThread.join(wait);
+ if (cnxn.eventThread.isAlive()) return false;
+ return true;
+ }
+
+ /**
+ * Returns the address to which the socket is connected. Useful for testing
+ * against an ensemble - test client may need to know which server
+ * to shutdown if interested in verifying that the code handles
+ * disconnection/reconnection correctly.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableRemoteSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getRemoteSocketAddress();
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableLocalSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getLocalSocketAddress();
+ }
+
+ private ClientCnxnSocket getClientCnxnSocket() throws IOException {
+ String clientCnxnSocketName = getClientConfig().getProperty(
+ ZKClientConfig.ZOOKEEPER_CLIENT_CNXN_SOCKET);
+ if (clientCnxnSocketName == null) {
+ clientCnxnSocketName = ClientCnxnSocketNIO.class.getName();
+ }
+ try {
+ Constructor<?> clientCxnConstructor = Class.forName(clientCnxnSocketName).getDeclaredConstructor(ZKClientConfig.class);
+ ClientCnxnSocket clientCxnSocket = (ClientCnxnSocket) clientCxnConstructor.newInstance(getClientConfig());
+ return clientCxnSocket;
+ } catch (Exception e) {
+ IOException ioe = new IOException("Couldn't instantiate "
+ + clientCnxnSocketName);
+ ioe.initCause(e);
+ throw ioe;
+ }
+ }
+
+ protected byte[] internalReconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.reconfig);
+ ReconfigRequest request = new ReconfigRequest(joiningServers, leavingServers, newMembers, fromConfig);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()), "");
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ protected byte[] internalReconfig(List<String> joiningServers, List<String> leavingServers, List<String> newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(StringUtils.joinStrings(joiningServers, ","),
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeper.internalReconfig(...)` indirectly writes to field `this.cnxn.sendThread.clientCnxnSocket.lastHeard` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeper.multiInternal(...)` indirectly writes to field `this.cnxn.sendThread.clientCnxnSocket.lastHeard` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()));
+ }
+
+ List<OpResult> results = response.getResultList();
+
+ ErrorResult fatalError = null;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult && ((ErrorResult)result).getErr() != KeeperException.Code.OK.intValue()) {
+ fatalError = (ErrorResult) result;
+ break;
+ }
+ }
+
+ if (fatalError != null) {
+ KeeperException ex = KeeperException.create(KeeperException.Code.get(fatalError.getErr()));
+ ex.setMultiResults(results);
+ throw ex;
+ }
+
+ return results;
+ }
+
+ /**
+ * A Transaction is a thin wrapper on the {@link #multi} method
+ * which provides a builder object that can be used to construct
+ * and commit an atomic set of operations.
+ *
+ * @since 3.4.0
+ *
+ * @return a Transaction builder object
+ */
+ public Transaction transaction() {
+ return new Transaction(this);
+ }
+
+ /**
+ * The asynchronous version of delete.
+ *
+ * @see #delete(String, int)
+ */
+ public void delete(final String path, int version, VoidCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path the node path
+ * @param watcher explicit watcher
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat exists(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ if (r.getErr() == KeeperException.Code.NONODE.intValue()) {
+ return null;
+ }
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+
+ return response.getStat().getCzxid() == -1 ? null : response.getStat();
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path
+ * the node path
+ * @param watch
+ * whether need to watch this node
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public Stat exists(String path, boolean watch) throws KeeperException,
+ InterruptedException
+ {
+ return exists(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, Watcher)
+ */
+ public void exists(final String path, Watcher watcher,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, boolean)
+ */
+ public void exists(String path, boolean watch, StatCallback cb, Object ctx) {
+ exists(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watcher explicit watcher
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public byte[] getData(final String path, Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watch whether need to watch this node
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getData(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getData(path, watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, Watcher, Stat)
+ */
+ public void getData(final String path, Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getData(String path, boolean watch, DataCallback cb, Object ctx) {
+ getData(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the configuration node doesn't exists.
+ *
+ * @param watcher explicit watcher
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ configZnode);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * The asynchronous version of getConfig.
+ *
+ * @see #getConfig(Watcher, Stat)
+ */
+ public void getConfig(Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ configZnode, configZnode, ctx, wcb);
+ }
+
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param watch whether need to watch this node
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getConfig(watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The Asynchronous version of getConfig.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getConfig(boolean watch, DataCallback cb, Object ctx) {
+ getConfig(watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeper.reconfig(...)` indirectly writes to field `this.cnxn.sendThread.clientCnxnSocket.lastHeard` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()));
+ }
+
+ List<OpResult> results = response.getResultList();
+
+ ErrorResult fatalError = null;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult && ((ErrorResult)result).getErr() != KeeperException.Code.OK.intValue()) {
+ fatalError = (ErrorResult) result;
+ break;
+ }
+ }
+
+ if (fatalError != null) {
+ KeeperException ex = KeeperException.create(KeeperException.Code.get(fatalError.getErr()));
+ ex.setMultiResults(results);
+ throw ex;
+ }
+
+ return results;
+ }
+
+ /**
+ * A Transaction is a thin wrapper on the {@link #multi} method
+ * which provides a builder object that can be used to construct
+ * and commit an atomic set of operations.
+ *
+ * @since 3.4.0
+ *
+ * @return a Transaction builder object
+ */
+ public Transaction transaction() {
+ return new Transaction(this);
+ }
+
+ /**
+ * The asynchronous version of delete.
+ *
+ * @see #delete(String, int)
+ */
+ public void delete(final String path, int version, VoidCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path the node path
+ * @param watcher explicit watcher
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat exists(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ if (r.getErr() == KeeperException.Code.NONODE.intValue()) {
+ return null;
+ }
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+
+ return response.getStat().getCzxid() == -1 ? null : response.getStat();
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path
+ * the node path
+ * @param watch
+ * whether need to watch this node
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public Stat exists(String path, boolean watch) throws KeeperException,
+ InterruptedException
+ {
+ return exists(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, Watcher)
+ */
+ public void exists(final String path, Watcher watcher,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, boolean)
+ */
+ public void exists(String path, boolean watch, StatCallback cb, Object ctx) {
+ exists(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watcher explicit watcher
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public byte[] getData(final String path, Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watch whether need to watch this node
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getData(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getData(path, watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, Watcher, Stat)
+ */
+ public void getData(final String path, Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getData(String path, boolean watch, DataCallback cb, Object ctx) {
+ getData(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the configuration node doesn't exists.
+ *
+ * @param watcher explicit watcher
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ configZnode);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * The asynchronous version of getConfig.
+ *
+ * @see #getConfig(Watcher, Stat)
+ */
+ public void getConfig(Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ configZnode, configZnode, ctx, wcb);
+ }
+
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param watch whether need to watch this node
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getConfig(watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The Asynchronous version of getConfig.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getConfig(boolean watch, DataCallback cb, Object ctx) {
+ getConfig(watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(List<String> joiningServers, List<String> leavingServers, List<String> newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeper.reconfig(...)` indirectly writes to field `this.cnxn.sendThread.clientCnxnSocket.lastHeard` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java
##########
@@ -0,0 +1,1486 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoNodeException;
+import org.apache.zookeeper.KeeperException.NodeExistsException;
+import org.apache.zookeeper.Quotas;
+import org.apache.zookeeper.StatsTrack;
+import org.apache.zookeeper.WatchedEvent;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.Watcher.Event;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.Event.KeeperState;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.PathTrie;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.txn.CheckVersionTxn;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.Txn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * This class maintains the tree data structure. It doesn't have any networking
+ * or client connection code in it so that it can be tested in a stand alone
+ * way.
+ * <p>
+ * The tree maintains two parallel data structures: a hashtable that maps from
+ * full paths to DataNodes and a tree of DataNodes. All accesses to a path is
+ * through the hashtable. The tree is traversed only when serializing to disk.
+ */
+public class DataTree {
+ private static final Logger LOG = LoggerFactory.getLogger(DataTree.class);
+
+ /**
+ * This hashtable provides a fast lookup to the datanodes. The tree is the
+ * source of truth and is where all the locking occurs
+ */
+ private final ConcurrentHashMap<String, DataNode> nodes =
+ new ConcurrentHashMap<String, DataNode>();
+
+ private final WatchManager dataWatches = new WatchManager();
+
+ private final WatchManager childWatches = new WatchManager();
+
+ /** the root of zookeeper tree */
+ private static final String rootZookeeper = "/";
+
+ /** the zookeeper nodes that acts as the management and status node **/
+ private static final String procZookeeper = Quotas.procZookeeper;
+
+ /** this will be the string thats stored as a child of root */
+ private static final String procChildZookeeper = procZookeeper.substring(1);
+
+ /**
+ * the zookeeper quota node that acts as the quota management node for
+ * zookeeper
+ */
+ private static final String quotaZookeeper = Quotas.quotaZookeeper;
+
+ /** this will be the string thats stored as a child of /zookeeper */
+ private static final String quotaChildZookeeper = quotaZookeeper
+ .substring(procZookeeper.length() + 1);
+
+ /**
+ * the zookeeper config node that acts as the config management node for
+ * zookeeper
+ */
+ private static final String configZookeeper = ZooDefs.CONFIG_NODE;
+
+ /** this will be the string thats stored as a child of /zookeeper */
+ private static final String configChildZookeeper = configZookeeper
+ .substring(procZookeeper.length() + 1);
+
+ /**
+ * the path trie that keeps track fo the quota nodes in this datatree
+ */
+ private final PathTrie pTrie = new PathTrie();
+
+ /**
+ * This hashtable lists the paths of the ephemeral nodes of a session.
+ */
+ private final Map<Long, HashSet<String>> ephemerals =
+ new ConcurrentHashMap<Long, HashSet<String>>();
+
+ /**
+ * This set contains the paths of all container nodes
+ */
+ private final Set<String> containers =
+ Collections.newSetFromMap(new ConcurrentHashMap<String, Boolean>());
+
+ /**
+ * This set contains the paths of all ttl nodes
+ */
+ private final Set<String> ttls =
+ Collections.newSetFromMap(new ConcurrentHashMap<String, Boolean>());
+
+ private final ReferenceCountedACLCache aclCache = new ReferenceCountedACLCache();
+
+ @SuppressWarnings("unchecked")
+ public Set<String> getEphemerals(long sessionId) {
+ HashSet<String> retv = ephemerals.get(sessionId);
+ if (retv == null) {
+ return new HashSet<String>();
+ }
+ HashSet<String> cloned = null;
+ synchronized (retv) {
+ cloned = (HashSet<String>) retv.clone();
+ }
+ return cloned;
+ }
+
+ public Set<String> getContainers() {
+ return new HashSet<String>(containers);
+ }
+
+ public Set<String> getTtls() {
+ return new HashSet<String>(ttls);
+ }
+
+ public Collection<Long> getSessions() {
+ return ephemerals.keySet();
+ }
+
+ public DataNode getNode(String path) {
+ return nodes.get(path);
+ }
+
+ public int getNodeCount() {
+ return nodes.size();
+ }
+
+ public int getWatchCount() {
+ return dataWatches.size() + childWatches.size();
+ }
+
+ public int getEphemeralsCount() {
+ int result = 0;
+ for (HashSet<String> set : ephemerals.values()) {
+ result += set.size();
+ }
+ return result;
+ }
+
+ /**
+ * Get the size of the nodes based on path and data length.
+ *
+ * @return size of the data
+ */
+ public long approximateDataSize() {
+ long result = 0;
+ for (Map.Entry<String, DataNode> entry : nodes.entrySet()) {
+ DataNode value = entry.getValue();
+ synchronized (value) {
+ result += entry.getKey().length();
+ result += value.getApproximateDataSize();
+ }
+ }
+ return result;
+ }
+
+ /**
+ * This is a pointer to the root of the DataTree. It is the source of truth,
+ * but we usually use the nodes hashmap to find nodes in the tree.
+ */
+ private DataNode root = new DataNode(new byte[0], -1L, new StatPersisted());
+
+ /**
+ * create a /zookeeper filesystem that is the proc filesystem of zookeeper
+ */
+ private final DataNode procDataNode = new DataNode(new byte[0], -1L, new StatPersisted());
+
+ /**
+ * create a /zookeeper/quota node for maintaining quota properties for
+ * zookeeper
+ */
+ private final DataNode quotaDataNode = new DataNode(new byte[0], -1L, new StatPersisted());
+
+ public DataTree() {
+ /* Rather than fight it, let root have an alias */
+ nodes.put("", root);
+ nodes.put(rootZookeeper, root);
+
+ /** add the proc node and quota node */
+ root.addChild(procChildZookeeper);
+ nodes.put(procZookeeper, procDataNode);
+
+ procDataNode.addChild(quotaChildZookeeper);
+ nodes.put(quotaZookeeper, quotaDataNode);
+
+ addConfigNode();
+ }
+
+ /**
+ * create a /zookeeper/config node for maintaining the configuration (membership and quorum system) info for
+ * zookeeper
+ */
+ public void addConfigNode() {
+ DataNode zookeeperZnode = nodes.get(procZookeeper);
+ if (zookeeperZnode != null) { // should always be the case
+ zookeeperZnode.addChild(configChildZookeeper);
+ } else {
+ assert false : "There's no /zookeeper znode - this should never happen.";
+ }
+
+ nodes.put(configZookeeper, new DataNode(new byte[0], -1L, new StatPersisted()));
+ try {
+ // Reconfig node is access controlled by default (ZOOKEEPER-2014).
+ setACL(configZookeeper, ZooDefs.Ids.READ_ACL_UNSAFE, -1);
+ } catch (KeeperException.NoNodeException e) {
+ assert false : "There's no " + configZookeeper +
+ " znode - this should never happen.";
+ }
+ }
+
+ /**
+ * is the path one of the special paths owned by zookeeper.
+ *
+ * @param path
+ * the path to be checked
+ * @return true if a special path. false if not.
+ */
+ boolean isSpecialPath(String path) {
+ if (rootZookeeper.equals(path) || procZookeeper.equals(path)
+ || quotaZookeeper.equals(path) || configZookeeper.equals(path)) {
+ return true;
+ }
+ return false;
+ }
+
+ static public void copyStatPersisted(StatPersisted from, StatPersisted to) {
+ to.setAversion(from.getAversion());
+ to.setCtime(from.getCtime());
+ to.setCversion(from.getCversion());
+ to.setCzxid(from.getCzxid());
+ to.setMtime(from.getMtime());
+ to.setMzxid(from.getMzxid());
+ to.setPzxid(from.getPzxid());
+ to.setVersion(from.getVersion());
+ to.setEphemeralOwner(from.getEphemeralOwner());
+ }
+
+ static public void copyStat(Stat from, Stat to) {
+ to.setAversion(from.getAversion());
+ to.setCtime(from.getCtime());
+ to.setCversion(from.getCversion());
+ to.setCzxid(from.getCzxid());
+ to.setMtime(from.getMtime());
+ to.setMzxid(from.getMzxid());
+ to.setPzxid(from.getPzxid());
+ to.setVersion(from.getVersion());
+ to.setEphemeralOwner(from.getEphemeralOwner());
+ to.setDataLength(from.getDataLength());
+ to.setNumChildren(from.getNumChildren());
+ }
+
+ /**
+ * update the count of this stat datanode
+ *
+ * @param lastPrefix
+ * the path of the node that is quotaed.
+ * @param diff
+ * the diff to be added to the count
+ */
+ public void updateCount(String lastPrefix, int diff) {
+ String statNode = Quotas.statPath(lastPrefix);
+ DataNode node = nodes.get(statNode);
+ StatsTrack updatedStat = null;
+ if (node == null) {
+ // should not happen
+ LOG.error("Missing count node for stat " + statNode);
+ return;
+ }
+ synchronized (node) {
+ updatedStat = new StatsTrack(new String(node.data));
+ updatedStat.setCount(updatedStat.getCount() + diff);
+ node.data = updatedStat.toString().getBytes();
+ }
+ // now check if the counts match the quota
+ String quotaNode = Quotas.quotaPath(lastPrefix);
+ node = nodes.get(quotaNode);
+ StatsTrack thisStats = null;
+ if (node == null) {
+ // should not happen
+ LOG.error("Missing count node for quota " + quotaNode);
+ return;
+ }
+ synchronized (node) {
+ thisStats = new StatsTrack(new String(node.data));
+ }
+ if (thisStats.getCount() > -1 && (thisStats.getCount() < updatedStat.getCount())) {
+ LOG
+ .warn("Quota exceeded: " + lastPrefix + " count="
+ + updatedStat.getCount() + " limit="
+ + thisStats.getCount());
+ }
+ }
+
+ /**
+ * update the count of bytes of this stat datanode
+ *
+ * @param lastPrefix
+ * the path of the node that is quotaed
+ * @param diff
+ * the diff to added to number of bytes
+ * @throws IOException
+ * if path is not found
+ */
+ public void updateBytes(String lastPrefix, long diff) {
+ String statNode = Quotas.statPath(lastPrefix);
+ DataNode node = nodes.get(statNode);
+ if (node == null) {
+ // should never be null but just to make
+ // findbugs happy
+ LOG.error("Missing stat node for bytes " + statNode);
+ return;
+ }
+ StatsTrack updatedStat = null;
+ synchronized (node) {
+ updatedStat = new StatsTrack(new String(node.data));
+ updatedStat.setBytes(updatedStat.getBytes() + diff);
+ node.data = updatedStat.toString().getBytes();
+ }
+ // now check if the bytes match the quota
+ String quotaNode = Quotas.quotaPath(lastPrefix);
+ node = nodes.get(quotaNode);
+ if (node == null) {
+ // should never be null but just to make
+ // findbugs happy
+ LOG.error("Missing quota node for bytes " + quotaNode);
+ return;
+ }
+ StatsTrack thisStats = null;
+ synchronized (node) {
+ thisStats = new StatsTrack(new String(node.data));
+ }
+ if (thisStats.getBytes() > -1 && (thisStats.getBytes() < updatedStat.getBytes())) {
+ LOG
+ .warn("Quota exceeded: " + lastPrefix + " bytes="
+ + updatedStat.getBytes() + " limit="
+ + thisStats.getBytes());
+ }
+ }
+
+ /**
+ * Add a new node to the DataTree.
+ * @param path
+ * Path for the new node.
+ * @param data
+ * Data to store in the node.
+ * @param acl
+ * Node acls
+ * @param ephemeralOwner
+ * the session id that owns this node. -1 indicates this is not
+ * an ephemeral node.
+ * @param zxid
+ * Transaction ID
+ * @param time
+ * @throws NodeExistsException
+ * @throws NoNodeException
+ * @throws KeeperException
+ */
+ public void createNode(final String path, byte data[], List<ACL> acl,
+ long ephemeralOwner, int parentCVersion, long zxid, long time)
+ throws NoNodeException, NodeExistsException {
+ createNode(path, data, acl, ephemeralOwner, parentCVersion, zxid, time, null);
+ }
+
+ /**
+ * Add a new node to the DataTree.
+ * @param path
+ * Path for the new node.
+ * @param data
+ * Data to store in the node.
+ * @param acl
+ * Node acls
+ * @param ephemeralOwner
+ * the session id that owns this node. -1 indicates this is not
+ * an ephemeral node.
+ * @param zxid
+ * Transaction ID
+ * @param time
+ * @param outputStat
+ * A Stat object to store Stat output results into.
+ * @throws NodeExistsException
+ * @throws NoNodeException
+ * @throws KeeperException
+ */
+ public void createNode(final String path, byte data[], List<ACL> acl,
+ long ephemeralOwner, int parentCVersion, long zxid, long time, Stat outputStat)
+ throws KeeperException.NoNodeException,
+ KeeperException.NodeExistsException {
+ int lastSlash = path.lastIndexOf('/');
+ String parentName = path.substring(0, lastSlash);
+ String childName = path.substring(lastSlash + 1);
+ StatPersisted stat = new StatPersisted();
+ stat.setCtime(time);
+ stat.setMtime(time);
+ stat.setCzxid(zxid);
+ stat.setMzxid(zxid);
+ stat.setPzxid(zxid);
+ stat.setVersion(0);
+ stat.setAversion(0);
+ stat.setEphemeralOwner(ephemeralOwner);
+ DataNode parent = nodes.get(parentName);
+ if (parent == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (parent) {
+ Set<String> children = parent.getChildren();
+ if (children.contains(childName)) {
+ throw new KeeperException.NodeExistsException();
+ }
+
+ if (parentCVersion == -1) {
+ parentCVersion = parent.stat.getCversion();
+ parentCVersion++;
+ }
+ parent.stat.setCversion(parentCVersion);
+ parent.stat.setPzxid(zxid);
+ Long longval = aclCache.convertAcls(acl);
+ DataNode child = new DataNode(data, longval, stat);
+ parent.addChild(childName);
+ nodes.put(path, child);
+ EphemeralType ephemeralType = EphemeralType.get(ephemeralOwner);
+ if (ephemeralType == EphemeralType.CONTAINER) {
+ containers.add(path);
+ } else if (ephemeralType == EphemeralType.TTL) {
+ ttls.add(path);
+ } else if (ephemeralOwner != 0) {
+ HashSet<String> list = ephemerals.get(ephemeralOwner);
+ if (list == null) {
+ list = new HashSet<String>();
+ ephemerals.put(ephemeralOwner, list);
+ }
+ synchronized (list) {
+ list.add(path);
+ }
+ }
+ if (outputStat != null) {
+ child.copyStat(outputStat);
+ }
+ }
+ // now check if its one of the zookeeper node child
+ if (parentName.startsWith(quotaZookeeper)) {
+ // now check if its the limit node
+ if (Quotas.limitNode.equals(childName)) {
+ // this is the limit node
+ // get the parent and add it to the trie
+ pTrie.addPath(parentName.substring(quotaZookeeper.length()));
+ }
+ if (Quotas.statNode.equals(childName)) {
+ updateQuotaForPath(parentName
+ .substring(quotaZookeeper.length()));
+ }
+ }
+ // also check to update the quotas for this node
+ String lastPrefix = getMaxPrefixWithQuota(path);
+ if(lastPrefix != null) {
+ // ok we have some match and need to update
+ updateCount(lastPrefix, 1);
+ updateBytes(lastPrefix, data == null ? 0 : data.length);
+ }
+ dataWatches.triggerWatch(path, Event.EventType.NodeCreated);
+ childWatches.triggerWatch(parentName.equals("") ? "/" : parentName,
+ Event.EventType.NodeChildrenChanged);
+ }
+
+ /**
+ * remove the path from the datatree
+ *
+ * @param path
+ * the path to of the node to be deleted
+ * @param zxid
+ * the current zxid
+ * @throws KeeperException.NoNodeException
+ */
+ public void deleteNode(String path, long zxid)
+ throws KeeperException.NoNodeException {
+ int lastSlash = path.lastIndexOf('/');
+ String parentName = path.substring(0, lastSlash);
+ String childName = path.substring(lastSlash + 1);
+
+ // The child might already be deleted during taking fuzzy snapshot,
+ // but we still need to update the pzxid here before throw exception
+ // for no such child
+ DataNode parent = nodes.get(parentName);
+ if (parent == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (parent) {
+ parent.removeChild(childName);
+ // Only update pzxid when the zxid is larger than the current pzxid,
+ // otherwise we might override higher pzxid set by a following create
+ // Txn, which could cause the cversion and pzxid inconsistent
+ if (zxid > parent.stat.getPzxid()) {
+ parent.stat.setPzxid(zxid);
+ }
+ }
+
+ DataNode node = nodes.get(path);
+ if (node == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ nodes.remove(path);
+ synchronized (node) {
+ aclCache.removeUsage(node.acl);
+ }
+
+ // Synchronized to sync the containers and ttls change, probably
+ // only need to sync on containers and ttls, will update it in a
+ // separate patch.
+ synchronized (parent) {
+ long eowner = node.stat.getEphemeralOwner();
+ EphemeralType ephemeralType = EphemeralType.get(eowner);
+ if (ephemeralType == EphemeralType.CONTAINER) {
+ containers.remove(path);
+ } else if (ephemeralType == EphemeralType.TTL) {
+ ttls.remove(path);
+ } else if (eowner != 0) {
+ HashSet<String> nodes = ephemerals.get(eowner);
+ if (nodes != null) {
+ synchronized (nodes) {
+ nodes.remove(path);
+ }
+ }
+ }
+ }
+
+ if (parentName.startsWith(procZookeeper) && Quotas.limitNode.equals(childName)) {
+ // delete the node in the trie.
+ // we need to update the trie as well
+ pTrie.deletePath(parentName.substring(quotaZookeeper.length()));
+ }
+
+ // also check to update the quotas for this node
+ String lastPrefix = getMaxPrefixWithQuota(path);
+ if(lastPrefix != null) {
+ // ok we have some match and need to update
+ updateCount(lastPrefix, -1);
+ int bytes = 0;
+ synchronized (node) {
+ bytes = (node.data == null ? 0 : -(node.data.length));
+ }
+ updateBytes(lastPrefix, bytes);
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.EVENT_DELIVERY_TRACE_MASK,
+ "dataWatches.triggerWatch " + path);
+ ZooTrace.logTraceMessage(LOG, ZooTrace.EVENT_DELIVERY_TRACE_MASK,
+ "childWatches.triggerWatch " + parentName);
+ }
+ Set<Watcher> processed = dataWatches.triggerWatch(path,
+ EventType.NodeDeleted);
+ childWatches.triggerWatch(path, EventType.NodeDeleted, processed);
+ childWatches.triggerWatch("".equals(parentName) ? "/" : parentName,
+ EventType.NodeChildrenChanged);
+ }
+
+ public Stat setData(String path, byte data[], int version, long zxid,
+ long time) throws KeeperException.NoNodeException {
+ Stat s = new Stat();
+ DataNode n = nodes.get(path);
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ byte lastdata[] = null;
+ synchronized (n) {
+ lastdata = n.data;
+ n.data = data;
+ n.stat.setMtime(time);
+ n.stat.setMzxid(zxid);
+ n.stat.setVersion(version);
+ n.copyStat(s);
+ }
+ // now update if the path is in a quota subtree.
+ String lastPrefix = getMaxPrefixWithQuota(path);
+ if(lastPrefix != null) {
+ this.updateBytes(lastPrefix, (data == null ? 0 : data.length)
+ - (lastdata == null ? 0 : lastdata.length));
+ }
+ dataWatches.triggerWatch(path, EventType.NodeDataChanged);
+ return s;
+ }
+
+ /**
+ * If there is a quota set, return the appropriate prefix for that quota
+ * Else return null
+ * @param path The ZK path to check for quota
+ * @return Max quota prefix, or null if none
+ */
+ public String getMaxPrefixWithQuota(String path) {
+ // do nothing for the root.
+ // we are not keeping a quota on the zookeeper
+ // root node for now.
+ String lastPrefix = pTrie.findMaxPrefix(path);
+
+ if (rootZookeeper.equals(lastPrefix) || "".equals(lastPrefix)) {
+ return null;
+ }
+ else {
+ return lastPrefix;
+ }
+ }
+
+ public byte[] getData(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ DataNode n = nodes.get(path);
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (n) {
+ n.copyStat(stat);
+ if (watcher != null) {
+ dataWatches.addWatch(path, watcher);
+ }
+ return n.data;
+ }
+ }
+
+ public Stat statNode(String path, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ Stat stat = new Stat();
+ DataNode n = nodes.get(path);
+ if (watcher != null) {
+ dataWatches.addWatch(path, watcher);
+ }
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (n) {
+ n.copyStat(stat);
+ return stat;
+ }
+ }
+
+ public List<String> getChildren(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ DataNode n = nodes.get(path);
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (n) {
+ if (stat != null) {
+ n.copyStat(stat);
+ }
+ List<String> children=new ArrayList<String>(n.getChildren());
+
+ if (watcher != null) {
+ childWatches.addWatch(path, watcher);
+ }
+ return children;
+ }
+ }
+
+ public Stat setACL(String path, List<ACL> acl, int version)
+ throws KeeperException.NoNodeException {
+ Stat stat = new Stat();
+ DataNode n = nodes.get(path);
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (n) {
+ aclCache.removeUsage(n.acl);
+ n.stat.setAversion(version);
+ n.acl = aclCache.convertAcls(acl);
+ n.copyStat(stat);
+ return stat;
+ }
+ }
+
+ public List<ACL> getACL(String path, Stat stat)
+ throws KeeperException.NoNodeException {
+ DataNode n = nodes.get(path);
+ if (n == null) {
+ throw new KeeperException.NoNodeException();
+ }
+ synchronized (n) {
+ n.copyStat(stat);
+ return new ArrayList<ACL>(aclCache.convertLong(n.acl));
+ }
+ }
+
+ public List<ACL> getACL(DataNode node) {
+ synchronized (node) {
+ return aclCache.convertLong(node.acl);
+ }
+ }
+
+ public int aclCacheSize() {
+ return aclCache.size();
+ }
+
+ static public class ProcessTxnResult {
+ public long clientId;
+
+ public int cxid;
+
+ public long zxid;
+
+ public int err;
+
+ public int type;
+
+ public String path;
+
+ public Stat stat;
+
+ public List<ProcessTxnResult> multiResult;
+
+ /**
+ * Equality is defined as the clientId and the cxid being the same. This
+ * allows us to use hash tables to track completion of transactions.
+ *
+ * @see java.lang.Object#equals(java.lang.Object)
+ */
+ @Override
+ public boolean equals(Object o) {
+ if (o instanceof ProcessTxnResult) {
+ ProcessTxnResult other = (ProcessTxnResult) o;
+ return other.clientId == clientId && other.cxid == cxid;
+ }
+ return false;
+ }
+
+ /**
+ * See equals() to find the rational for how this hashcode is generated.
+ *
+ * @see ProcessTxnResult#equals(Object)
+ * @see java.lang.Object#hashCode()
+ */
+ @Override
+ public int hashCode() {
+ return (int) ((clientId ^ cxid) % Integer.MAX_VALUE);
+ }
+
+ }
+
+ public volatile long lastProcessedZxid = 0;
+
+ public ProcessTxnResult processTxn(TxnHeader header, Record txn) {
+ return this.processTxn(header, txn, false);
+ }
+
+ public ProcessTxnResult processTxn(TxnHeader header, Record txn, boolean isSubTxn)
+ {
+ ProcessTxnResult rc = new ProcessTxnResult();
+
+ try {
+ rc.clientId = header.getClientId();
+ rc.cxid = header.getCxid();
+ rc.zxid = header.getZxid();
+ rc.type = header.getType();
+ rc.err = 0;
+ rc.multiResult = null;
+ switch (header.getType()) {
+ case OpCode.create:
+ CreateTxn createTxn = (CreateTxn) txn;
+ rc.path = createTxn.getPath();
+ createNode(
+ createTxn.getPath(),
+ createTxn.getData(),
+ createTxn.getAcl(),
+ createTxn.getEphemeral() ? header.getClientId() : 0,
+ createTxn.getParentCVersion(),
+ header.getZxid(), header.getTime(), null);
+ break;
+ case OpCode.create2:
+ CreateTxn create2Txn = (CreateTxn) txn;
+ rc.path = create2Txn.getPath();
+ Stat stat = new Stat();
+ createNode(
+ create2Txn.getPath(),
+ create2Txn.getData(),
+ create2Txn.getAcl(),
+ create2Txn.getEphemeral() ? header.getClientId() : 0,
+ create2Txn.getParentCVersion(),
+ header.getZxid(), header.getTime(), stat);
+ rc.stat = stat;
+ break;
+ case OpCode.createTTL:
+ CreateTTLTxn createTtlTxn = (CreateTTLTxn) txn;
+ rc.path = createTtlTxn.getPath();
+ stat = new Stat();
+ createNode(
+ createTtlTxn.getPath(),
+ createTtlTxn.getData(),
+ createTtlTxn.getAcl(),
+ EphemeralType.TTL.toEphemeralOwner(createTtlTxn.getTtl()),
+ createTtlTxn.getParentCVersion(),
+ header.getZxid(), header.getTime(), stat);
+ rc.stat = stat;
+ break;
+ case OpCode.createContainer:
+ CreateContainerTxn createContainerTxn = (CreateContainerTxn) txn;
+ rc.path = createContainerTxn.getPath();
+ stat = new Stat();
+ createNode(
+ createContainerTxn.getPath(),
+ createContainerTxn.getData(),
+ createContainerTxn.getAcl(),
+ EphemeralType.CONTAINER_EPHEMERAL_OWNER,
+ createContainerTxn.getParentCVersion(),
+ header.getZxid(), header.getTime(), stat);
+ rc.stat = stat;
+ break;
+ case OpCode.delete:
+ case OpCode.deleteContainer:
+ DeleteTxn deleteTxn = (DeleteTxn) txn;
+ rc.path = deleteTxn.getPath();
+ deleteNode(deleteTxn.getPath(), header.getZxid());
+ break;
+ case OpCode.reconfig:
+ case OpCode.setData:
+ SetDataTxn setDataTxn = (SetDataTxn) txn;
+ rc.path = setDataTxn.getPath();
+ rc.stat = setData(setDataTxn.getPath(), setDataTxn
+ .getData(), setDataTxn.getVersion(), header
+ .getZxid(), header.getTime());
+ break;
+ case OpCode.setACL:
+ SetACLTxn setACLTxn = (SetACLTxn) txn;
+ rc.path = setACLTxn.getPath();
+ rc.stat = setACL(setACLTxn.getPath(), setACLTxn.getAcl(),
+ setACLTxn.getVersion());
+ break;
+ case OpCode.closeSession:
+ killSession(header.getClientId(), header.getZxid());
+ break;
+ case OpCode.error:
+ ErrorTxn errTxn = (ErrorTxn) txn;
+ rc.err = errTxn.getErr();
+ break;
+ case OpCode.check:
+ CheckVersionTxn checkTxn = (CheckVersionTxn) txn;
+ rc.path = checkTxn.getPath();
+ break;
+ case OpCode.multi:
+ MultiTxn multiTxn = (MultiTxn) txn ;
+ List<Txn> txns = multiTxn.getTxns();
+ rc.multiResult = new ArrayList<ProcessTxnResult>();
+ boolean failed = false;
+ for (Txn subtxn : txns) {
+ if (subtxn.getType() == OpCode.error) {
+ failed = true;
+ break;
+ }
+ }
+
+ boolean post_failed = false;
+ for (Txn subtxn : txns) {
+ ByteBuffer bb = ByteBuffer.wrap(subtxn.getData());
+ Record record = null;
+ switch (subtxn.getType()) {
+ case OpCode.create:
+ record = new CreateTxn();
+ break;
+ case OpCode.createTTL:
+ record = new CreateTTLTxn();
+ break;
+ case OpCode.createContainer:
+ record = new CreateContainerTxn();
+ break;
+ case OpCode.delete:
+ case OpCode.deleteContainer:
+ record = new DeleteTxn();
+ break;
+ case OpCode.setData:
+ record = new SetDataTxn();
+ break;
+ case OpCode.error:
+ record = new ErrorTxn();
+ post_failed = true;
+ break;
+ case OpCode.check:
+ record = new CheckVersionTxn();
+ break;
+ default:
+ throw new IOException("Invalid type of op: " + subtxn.getType());
+ }
+ assert(record != null);
+
+ ByteBufferInputStream.byteBuffer2Record(bb, record);
+
+ if (failed && subtxn.getType() != OpCode.error){
+ int ec = post_failed ? Code.RUNTIMEINCONSISTENCY.intValue()
+ : Code.OK.intValue();
+
+ subtxn.setType(OpCode.error);
+ record = new ErrorTxn(ec);
+ }
+
+ if (failed) {
+ assert(subtxn.getType() == OpCode.error) ;
+ }
+
+ TxnHeader subHdr = new TxnHeader(header.getClientId(), header.getCxid(),
+ header.getZxid(), header.getTime(),
+ subtxn.getType());
+ ProcessTxnResult subRc = processTxn(subHdr, record, true);
+ rc.multiResult.add(subRc);
+ if (subRc.err != 0 && rc.err == 0) {
+ rc.err = subRc.err ;
+ }
+ }
+ break;
+ }
+ } catch (KeeperException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Failed: " + header + ":" + txn, e);
+ }
+ rc.err = e.code().intValue();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Failed: " + header + ":" + txn, e);
+ }
+ }
+
+
+ /*
+ * Things we can only update after the whole txn is applied to data
+ * tree.
+ *
+ * If we update the lastProcessedZxid with the first sub txn in multi
+ * and there is a snapshot in progress, it's possible that the zxid
+ * associated with the snapshot only include partial of the multi op.
+ *
+ * When loading snapshot, it will only load the txns after the zxid
+ * associated with snapshot file, which could cause data inconsistency
+ * due to missing sub txns.
+ *
+ * To avoid this, we only update the lastProcessedZxid when the whole
+ * multi-op txn is applied to DataTree.
+ */
+ if (!isSubTxn) {
+ /*
+ * A snapshot might be in progress while we are modifying the data
+ * tree. If we set lastProcessedZxid prior to making corresponding
+ * change to the tree, then the zxid associated with the snapshot
+ * file will be ahead of its contents. Thus, while restoring from
+ * the snapshot, the restore method will not apply the transaction
+ * for zxid associated with the snapshot file, since the restore
+ * method assumes that transaction to be present in the snapshot.
+ *
+ * To avoid this, we first apply the transaction and then modify
+ * lastProcessedZxid. During restore, we correctly handle the
+ * case where the snapshot contains data ahead of the zxid associated
+ * with the file.
+ */
+ if (rc.zxid > lastProcessedZxid) {
+ lastProcessedZxid = rc.zxid;
+ }
+ }
+
+ /*
+ * Snapshots are taken lazily. It can happen that the child
+ * znodes of a parent are created after the parent
+ * is serialized. Therefore, while replaying logs during restore, a
+ * create might fail because the node was already
+ * created.
+ *
+ * After seeing this failure, we should increment
+ * the cversion of the parent znode since the parent was serialized
+ * before its children.
+ *
+ * Note, such failures on DT should be seen only during
+ * restore.
+ */
+ if (header.getType() == OpCode.create &&
+ rc.err == Code.NODEEXISTS.intValue()) {
+ LOG.debug("Adjusting parent cversion for Txn: " + header.getType() +
+ " path:" + rc.path + " err: " + rc.err);
+ int lastSlash = rc.path.lastIndexOf('/');
+ String parentName = rc.path.substring(0, lastSlash);
+ CreateTxn cTxn = (CreateTxn)txn;
+ try {
+ setCversionPzxid(parentName, cTxn.getParentCVersion(),
+ header.getZxid());
+ } catch (KeeperException.NoNodeException e) {
+ LOG.error("Failed to set parent cversion for: " +
+ parentName, e);
+ rc.err = e.code().intValue();
+ }
+ } else if (rc.err != Code.OK.intValue()) {
+ LOG.debug("Ignoring processTxn failure hdr: " + header.getType() +
+ " : error: " + rc.err);
+ }
+ return rc;
+ }
+
+ void killSession(long session, long zxid) {
+ // the list is already removed from the ephemerals
+ // so we do not have to worry about synchronizing on
+ // the list. This is only called from FinalRequestProcessor
+ // so there is no need for synchronization. The list is not
+ // changed here. Only create and delete change the list which
+ // are again called from FinalRequestProcessor in sequence.
+ HashSet<String> list = ephemerals.remove(session);
+ if (list != null) {
+ for (String path : list) {
+ try {
+ deleteNode(path, zxid);
+ if (LOG.isDebugEnabled()) {
+ LOG
+ .debug("Deleting ephemeral node " + path
+ + " for session 0x"
+ + Long.toHexString(session));
+ }
+ } catch (NoNodeException e) {
+ LOG.warn("Ignoring NoNodeException for path " + path
+ + " while removing ephemeral for dead session 0x"
+ + Long.toHexString(session));
+ }
+ }
+ }
+ }
+
+ /**
+ * a encapsultaing class for return value
+ */
+ private static class Counts {
+ long bytes;
+ int count;
+ }
+
+ /**
+ * this method gets the count of nodes and the bytes under a subtree
+ *
+ * @param path
+ * the path to be used
+ * @param counts
+ * the int count
+ */
+ private void getCounts(String path, Counts counts) {
+ DataNode node = getNode(path);
+ if (node == null) {
+ return;
+ }
+ String[] children = null;
+ int len = 0;
+ synchronized (node) {
+ Set<String> childs = node.getChildren();
+ children = childs.toArray(new String[childs.size()]);
+ len = (node.data == null ? 0 : node.data.length);
+ }
+ // add itself
+ counts.count += 1;
+ counts.bytes += len;
+ for (String child : children) {
+ getCounts(path + "/" + child, counts);
+ }
+ }
+
+ /**
+ * update the quota for the given path
+ *
+ * @param path
+ * the path to be used
+ */
+ private void updateQuotaForPath(String path) {
+ Counts c = new Counts();
+ getCounts(path, c);
+ StatsTrack strack = new StatsTrack();
+ strack.setBytes(c.bytes);
+ strack.setCount(c.count);
+ String statPath = Quotas.quotaZookeeper + path + "/" + Quotas.statNode;
+ DataNode node = getNode(statPath);
+ // it should exist
+ if (node == null) {
+ LOG.warn("Missing quota stat node " + statPath);
+ return;
+ }
+ synchronized (node) {
+ node.data = strack.toString().getBytes();
+ }
+ }
+
+ /**
+ * this method traverses the quota path and update the path trie and sets
+ *
+ * @param path
+ */
+ private void traverseNode(String path) {
+ DataNode node = getNode(path);
+ String children[] = null;
+ synchronized (node) {
+ Set<String> childs = node.getChildren();
+ children = childs.toArray(new String[childs.size()]);
+ }
+ if (children.length == 0) {
+ // this node does not have a child
+ // is the leaf node
+ // check if its the leaf node
+ String endString = "/" + Quotas.limitNode;
+ if (path.endsWith(endString)) {
+ // ok this is the limit node
+ // get the real node and update
+ // the count and the bytes
+ String realPath = path.substring(Quotas.quotaZookeeper
+ .length(), path.indexOf(endString));
+ updateQuotaForPath(realPath);
+ this.pTrie.addPath(realPath);
+ }
+ return;
+ }
+ for (String child : children) {
+ traverseNode(path + "/" + child);
+ }
+ }
+
+ /**
+ * this method sets up the path trie and sets up stats for quota nodes
+ */
+ private void setupQuota() {
+ String quotaPath = Quotas.quotaZookeeper;
+ DataNode node = getNode(quotaPath);
+ if (node == null) {
+ return;
+ }
+ traverseNode(quotaPath);
+ }
+
+ /**
+ * this method uses a stringbuilder to create a new path for children. This
+ * is faster than string appends ( str1 + str2).
+ *
+ * @param oa
+ * OutputArchive to write to.
+ * @param path
+ * a string builder.
+ * @throws IOException
+ * @throws InterruptedException
+ */
+ void serializeNode(OutputArchive oa, StringBuilder path) throws IOException {
+ String pathString = path.toString();
+ DataNode node = getNode(pathString);
+ if (node == null) {
+ return;
+ }
+ String children[] = null;
+ DataNode nodeCopy;
+ synchronized (node) {
+ StatPersisted statCopy = new StatPersisted();
+ copyStatPersisted(node.stat, statCopy);
+ //we do not need to make a copy of node.data because the contents
+ //are never changed
+ nodeCopy = new DataNode(node.data, node.acl, statCopy);
+ Set<String> childs = node.getChildren();
+ children = childs.toArray(new String[childs.size()]);
+ }
+ serializeNodeData(oa, pathString, nodeCopy);
+ path.append('/');
+ int off = path.length();
+ for (String child : children) {
+ // since this is single buffer being resused
+ // we need
+ // to truncate the previous bytes of string.
+ path.delete(off, Integer.MAX_VALUE);
+ path.append(child);
+ serializeNode(oa, path);
+ }
+ }
+
+ // visiable for test
+ public void serializeNodeData(OutputArchive oa, String path, DataNode node) throws IOException {
+ oa.writeString(path, "path");
+ oa.writeRecord(node, "node");
+ }
+
+ public void serialize(OutputArchive oa, String tag) throws IOException {
+ aclCache.serialize(oa);
+ serializeNode(oa, new StringBuilder(""));
+ // / marks end of stream
+ // we need to check if clear had been called in between the snapshot.
+ if (root != null) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `DataTree.serialize(...)` reads without synchronization from `this.root`. Potentially races with write in method `DataTree.deserialize(...)`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java
##########
@@ -0,0 +1,1012 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.BadArgumentsException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.MultiTransactionRecord;
+import org.apache.zookeeper.Op;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.proto.CheckVersionRequest;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.server.ZooKeeperServer.ChangeRecord;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.quorum.Leader.XidRolloverException;
+import org.apache.zookeeper.server.quorum.LeaderZooKeeperServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.txn.CheckVersionTxn;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.Txn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.StringReader;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingQueue;
+
+/**
+ * This request processor is generally at the start of a RequestProcessor
+ * change. It sets up any transactions associated with requests that change the
+ * state of the system. It counts on ZooKeeperServer to update
+ * outstandingRequests, so that it can take into account transactions that are
+ * in the queue to be applied when generating a transaction.
+ */
+public class PrepRequestProcessor extends ZooKeeperCriticalThread implements
+ RequestProcessor {
+ private static final Logger LOG = LoggerFactory.getLogger(PrepRequestProcessor.class);
+
+ static boolean skipACL;
+ static {
+ skipACL = System.getProperty("zookeeper.skipACL", "no").equals("yes");
+ if (skipACL) {
+ LOG.info("zookeeper.skipACL==\"yes\", ACL checks will be skipped");
+ }
+ }
+
+ /**
+ * this is only for testing purposes.
+ * should never be used otherwise
+ */
+ private static boolean failCreate = false;
+
+ LinkedBlockingQueue<Request> submittedRequests = new LinkedBlockingQueue<Request>();
+
+ private final RequestProcessor nextProcessor;
+
+ ZooKeeperServer zks;
+
+ public PrepRequestProcessor(ZooKeeperServer zks,
+ RequestProcessor nextProcessor) {
+ super("ProcessThread(sid:" + zks.getServerId() + " cport:"
+ + zks.getClientPort() + "):", zks.getZooKeeperServerListener());
+ this.nextProcessor = nextProcessor;
+ this.zks = zks;
+ }
+
+ /**
+ * method for tests to set failCreate
+ * @param b
+ */
+ public static void setFailCreate(boolean b) {
+ failCreate = b;
+ }
+ @Override
+ public void run() {
+ try {
+ while (true) {
+ Request request = submittedRequests.take();
+ long traceMask = ZooTrace.CLIENT_REQUEST_TRACE_MASK;
+ if (request.type == OpCode.ping) {
+ traceMask = ZooTrace.CLIENT_PING_TRACE_MASK;
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logRequest(LOG, traceMask, 'P', request, "");
+ }
+ if (Request.requestOfDeath == request) {
+ break;
+ }
+ pRequest(request);
+ }
+ } catch (RequestProcessorException e) {
+ if (e.getCause() instanceof XidRolloverException) {
+ LOG.info(e.getCause().getMessage());
+ }
+ handleException(this.getName(), e);
+ } catch (Exception e) {
+ handleException(this.getName(), e);
+ }
+ LOG.info("PrepRequestProcessor exited loop!");
+ }
+
+ private ChangeRecord getRecordForPath(String path) throws KeeperException.NoNodeException {
+ ChangeRecord lastChange = null;
+ synchronized (zks.outstandingChanges) {
+ lastChange = zks.outstandingChangesForPath.get(path);
+ if (lastChange == null) {
+ DataNode n = zks.getZKDatabase().getNode(path);
+ if (n != null) {
+ Set<String> children;
+ synchronized(n) {
+ children = n.getChildren();
+ }
+ lastChange = new ChangeRecord(-1, path, n.stat, children.size(),
+ zks.getZKDatabase().aclForNode(n));
+ }
+ }
+ }
+ if (lastChange == null || lastChange.stat == null) {
+ throw new KeeperException.NoNodeException(path);
+ }
+ return lastChange;
+ }
+
+ private ChangeRecord getOutstandingChange(String path) {
+ synchronized (zks.outstandingChanges) {
+ return zks.outstandingChangesForPath.get(path);
+ }
+ }
+
+ private void addChangeRecord(ChangeRecord c) {
+ synchronized (zks.outstandingChanges) {
+ zks.outstandingChanges.add(c);
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+
+ /**
+ * Grab current pending change records for each op in a multi-op.
+ *
+ * This is used inside MultiOp error code path to rollback in the event
+ * of a failed multi-op.
+ *
+ * @param multiRequest
+ * @return a map that contains previously existed records that probably need to be
+ * rolled back in any failure.
+ */
+ private Map<String, ChangeRecord> getPendingChanges(MultiTransactionRecord multiRequest) {
+ HashMap<String, ChangeRecord> pendingChangeRecords = new HashMap<String, ChangeRecord>();
+
+ for (Op op : multiRequest) {
+ String path = op.getPath();
+ ChangeRecord cr = getOutstandingChange(path);
+ // only previously existing records need to be rolled back.
+ if (cr != null) {
+ pendingChangeRecords.put(path, cr);
+ }
+
+ /*
+ * ZOOKEEPER-1624 - We need to store for parent's ChangeRecord
+ * of the parent node of a request. So that if this is a
+ * sequential node creation request, rollbackPendingChanges()
+ * can restore previous parent's ChangeRecord correctly.
+ *
+ * Otherwise, sequential node name generation will be incorrect
+ * for a subsequent request.
+ */
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1) {
+ continue;
+ }
+ String parentPath = path.substring(0, lastSlash);
+ ChangeRecord parentCr = getOutstandingChange(parentPath);
+ if (parentCr != null) {
+ pendingChangeRecords.put(parentPath, parentCr);
+ }
+ }
+
+ return pendingChangeRecords;
+ }
+
+ /**
+ * Rollback pending changes records from a failed multi-op.
+ *
+ * If a multi-op fails, we can't leave any invalid change records we created
+ * around. We also need to restore their prior value (if any) if their prior
+ * value is still valid.
+ *
+ * @param zxid
+ * @param pendingChangeRecords
+ */
+ void rollbackPendingChanges(long zxid, Map<String, ChangeRecord>pendingChangeRecords) {
+ synchronized (zks.outstandingChanges) {
+ // Grab a list iterator starting at the END of the list so we can iterate in reverse
+ Iterator<ChangeRecord> iter = zks.outstandingChanges.descendingIterator();
+ while (iter.hasNext()) {
+ ChangeRecord c = iter.next();
+ if (c.zxid == zxid) {
+ iter.remove();
+ // Remove all outstanding changes for paths of this multi.
+ // Previous records will be added back later.
+ zks.outstandingChangesForPath.remove(c.path);
+ } else {
+ break;
+ }
+ }
+
+ // we don't need to roll back any records because there is nothing left.
+ if (zks.outstandingChanges.isEmpty()) {
+ return;
+ }
+
+ long firstZxid = zks.outstandingChanges.peek().zxid;
+
+ for (ChangeRecord c : pendingChangeRecords.values()) {
+ // Don't apply any prior change records less than firstZxid.
+ // Note that previous outstanding requests might have been removed
+ // once they are completed.
+ if (c.zxid < firstZxid) {
+ continue;
+ }
+
+ // add previously existing records back.
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+ }
+
+ /**
+ * Grant or deny authorization to an operation on a node as a function of:
+ *
+ * @param zks: not used.
+ * @param acl: set of ACLs for the node
+ * @param perm: the permission that the client is requesting
+ * @param ids: the credentials supplied by the client
+ */
+ static void checkACL(ZooKeeperServer zks, List<ACL> acl, int perm,
+ List<Id> ids) throws KeeperException.NoAuthException {
+ if (skipACL) {
+ return;
+ }
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Permission requested: {} ", perm);
+ LOG.debug("ACLs for node: {}", acl);
+ LOG.debug("Client credentials: {}", ids);
+ }
+ if (acl == null || acl.size() == 0) {
+ return;
+ }
+ for (Id authId : ids) {
+ if (authId.getScheme().equals("super")) {
+ return;
+ }
+ }
+ for (ACL a : acl) {
+ Id id = a.getId();
+ if ((a.getPerms() & perm) != 0) {
+ if (id.getScheme().equals("world")
+ && id.getId().equals("anyone")) {
+ return;
+ }
+ AuthenticationProvider ap = ProviderRegistry.getProvider(id
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `PrepRequestProcessor.checkACL(...)` indirectly reads without synchronization from `auth.ProviderRegistry.initialized`. Potentially races with write in method `PrepRequestProcessor.pRequest2Txn(...)`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java
##########
@@ -0,0 +1,1012 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.BadArgumentsException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.MultiTransactionRecord;
+import org.apache.zookeeper.Op;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.proto.CheckVersionRequest;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.server.ZooKeeperServer.ChangeRecord;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.quorum.Leader.XidRolloverException;
+import org.apache.zookeeper.server.quorum.LeaderZooKeeperServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.txn.CheckVersionTxn;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.Txn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.StringReader;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingQueue;
+
+/**
+ * This request processor is generally at the start of a RequestProcessor
+ * change. It sets up any transactions associated with requests that change the
+ * state of the system. It counts on ZooKeeperServer to update
+ * outstandingRequests, so that it can take into account transactions that are
+ * in the queue to be applied when generating a transaction.
+ */
+public class PrepRequestProcessor extends ZooKeeperCriticalThread implements
+ RequestProcessor {
+ private static final Logger LOG = LoggerFactory.getLogger(PrepRequestProcessor.class);
+
+ static boolean skipACL;
+ static {
+ skipACL = System.getProperty("zookeeper.skipACL", "no").equals("yes");
+ if (skipACL) {
+ LOG.info("zookeeper.skipACL==\"yes\", ACL checks will be skipped");
+ }
+ }
+
+ /**
+ * this is only for testing purposes.
+ * should never be used otherwise
+ */
+ private static boolean failCreate = false;
+
+ LinkedBlockingQueue<Request> submittedRequests = new LinkedBlockingQueue<Request>();
+
+ private final RequestProcessor nextProcessor;
+
+ ZooKeeperServer zks;
+
+ public PrepRequestProcessor(ZooKeeperServer zks,
+ RequestProcessor nextProcessor) {
+ super("ProcessThread(sid:" + zks.getServerId() + " cport:"
+ + zks.getClientPort() + "):", zks.getZooKeeperServerListener());
+ this.nextProcessor = nextProcessor;
+ this.zks = zks;
+ }
+
+ /**
+ * method for tests to set failCreate
+ * @param b
+ */
+ public static void setFailCreate(boolean b) {
+ failCreate = b;
+ }
+ @Override
+ public void run() {
+ try {
+ while (true) {
+ Request request = submittedRequests.take();
+ long traceMask = ZooTrace.CLIENT_REQUEST_TRACE_MASK;
+ if (request.type == OpCode.ping) {
+ traceMask = ZooTrace.CLIENT_PING_TRACE_MASK;
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logRequest(LOG, traceMask, 'P', request, "");
+ }
+ if (Request.requestOfDeath == request) {
+ break;
+ }
+ pRequest(request);
+ }
+ } catch (RequestProcessorException e) {
+ if (e.getCause() instanceof XidRolloverException) {
+ LOG.info(e.getCause().getMessage());
+ }
+ handleException(this.getName(), e);
+ } catch (Exception e) {
+ handleException(this.getName(), e);
+ }
+ LOG.info("PrepRequestProcessor exited loop!");
+ }
+
+ private ChangeRecord getRecordForPath(String path) throws KeeperException.NoNodeException {
+ ChangeRecord lastChange = null;
+ synchronized (zks.outstandingChanges) {
+ lastChange = zks.outstandingChangesForPath.get(path);
+ if (lastChange == null) {
+ DataNode n = zks.getZKDatabase().getNode(path);
+ if (n != null) {
+ Set<String> children;
+ synchronized(n) {
+ children = n.getChildren();
+ }
+ lastChange = new ChangeRecord(-1, path, n.stat, children.size(),
+ zks.getZKDatabase().aclForNode(n));
+ }
+ }
+ }
+ if (lastChange == null || lastChange.stat == null) {
+ throw new KeeperException.NoNodeException(path);
+ }
+ return lastChange;
+ }
+
+ private ChangeRecord getOutstandingChange(String path) {
+ synchronized (zks.outstandingChanges) {
+ return zks.outstandingChangesForPath.get(path);
+ }
+ }
+
+ private void addChangeRecord(ChangeRecord c) {
+ synchronized (zks.outstandingChanges) {
+ zks.outstandingChanges.add(c);
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+
+ /**
+ * Grab current pending change records for each op in a multi-op.
+ *
+ * This is used inside MultiOp error code path to rollback in the event
+ * of a failed multi-op.
+ *
+ * @param multiRequest
+ * @return a map that contains previously existed records that probably need to be
+ * rolled back in any failure.
+ */
+ private Map<String, ChangeRecord> getPendingChanges(MultiTransactionRecord multiRequest) {
+ HashMap<String, ChangeRecord> pendingChangeRecords = new HashMap<String, ChangeRecord>();
+
+ for (Op op : multiRequest) {
+ String path = op.getPath();
+ ChangeRecord cr = getOutstandingChange(path);
+ // only previously existing records need to be rolled back.
+ if (cr != null) {
+ pendingChangeRecords.put(path, cr);
+ }
+
+ /*
+ * ZOOKEEPER-1624 - We need to store for parent's ChangeRecord
+ * of the parent node of a request. So that if this is a
+ * sequential node creation request, rollbackPendingChanges()
+ * can restore previous parent's ChangeRecord correctly.
+ *
+ * Otherwise, sequential node name generation will be incorrect
+ * for a subsequent request.
+ */
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1) {
+ continue;
+ }
+ String parentPath = path.substring(0, lastSlash);
+ ChangeRecord parentCr = getOutstandingChange(parentPath);
+ if (parentCr != null) {
+ pendingChangeRecords.put(parentPath, parentCr);
+ }
+ }
+
+ return pendingChangeRecords;
+ }
+
+ /**
+ * Rollback pending changes records from a failed multi-op.
+ *
+ * If a multi-op fails, we can't leave any invalid change records we created
+ * around. We also need to restore their prior value (if any) if their prior
+ * value is still valid.
+ *
+ * @param zxid
+ * @param pendingChangeRecords
+ */
+ void rollbackPendingChanges(long zxid, Map<String, ChangeRecord>pendingChangeRecords) {
+ synchronized (zks.outstandingChanges) {
+ // Grab a list iterator starting at the END of the list so we can iterate in reverse
+ Iterator<ChangeRecord> iter = zks.outstandingChanges.descendingIterator();
+ while (iter.hasNext()) {
+ ChangeRecord c = iter.next();
+ if (c.zxid == zxid) {
+ iter.remove();
+ // Remove all outstanding changes for paths of this multi.
+ // Previous records will be added back later.
+ zks.outstandingChangesForPath.remove(c.path);
+ } else {
+ break;
+ }
+ }
+
+ // we don't need to roll back any records because there is nothing left.
+ if (zks.outstandingChanges.isEmpty()) {
+ return;
+ }
+
+ long firstZxid = zks.outstandingChanges.peek().zxid;
+
+ for (ChangeRecord c : pendingChangeRecords.values()) {
+ // Don't apply any prior change records less than firstZxid.
+ // Note that previous outstanding requests might have been removed
+ // once they are completed.
+ if (c.zxid < firstZxid) {
+ continue;
+ }
+
+ // add previously existing records back.
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+ }
+
+ /**
+ * Grant or deny authorization to an operation on a node as a function of:
+ *
+ * @param zks: not used.
+ * @param acl: set of ACLs for the node
+ * @param perm: the permission that the client is requesting
+ * @param ids: the credentials supplied by the client
+ */
+ static void checkACL(ZooKeeperServer zks, List<ACL> acl, int perm,
+ List<Id> ids) throws KeeperException.NoAuthException {
+ if (skipACL) {
+ return;
+ }
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Permission requested: {} ", perm);
+ LOG.debug("ACLs for node: {}", acl);
+ LOG.debug("Client credentials: {}", ids);
+ }
+ if (acl == null || acl.size() == 0) {
+ return;
+ }
+ for (Id authId : ids) {
+ if (authId.getScheme().equals("super")) {
+ return;
+ }
+ }
+ for (ACL a : acl) {
+ Id id = a.getId();
+ if ((a.getPerms() & perm) != 0) {
+ if (id.getScheme().equals("world")
+ && id.getId().equals("anyone")) {
+ return;
+ }
+ AuthenticationProvider ap = ProviderRegistry.getProvider(id
+ .getScheme());
+ if (ap != null) {
+ for (Id authId : ids) {
+ if (authId.getScheme().equals(id.getScheme())
+ && ap.matches(authId.getId(), id.getId())) {
+ return;
+ }
+ }
+ }
+ }
+ }
+ throw new KeeperException.NoAuthException();
+ }
+
+ /**
+ * Performs basic validation of a path for a create request.
+ * Throws if the path is not valid and returns the parent path.
+ * @throws BadArgumentsException
+ */
+ private String validatePathForCreate(String path, long sessionId)
+ throws BadArgumentsException {
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1 || failCreate) {
+ LOG.info("Invalid path %s with session 0x%s",
+ path, Long.toHexString(sessionId));
+ throw new KeeperException.BadArgumentsException(path);
+ }
+ return path.substring(0, lastSlash);
+ }
+
+ /**
+ * This method will be called inside the ProcessRequestThread, which is a
+ * singleton, so there will be a single thread calling this code.
+ *
+ * @param type
+ * @param zxid
+ * @param request
+ * @param record
+ */
+ protected void pRequest2Txn(int type, long zxid, Request request,
+ Record record, boolean deserialize)
+ throws KeeperException, IOException, RequestProcessorException
+ {
+ request.setHdr(new TxnHeader(request.sessionId, request.cxid, zxid,
+ Time.currentWallTime(), type));
+
+ switch (type) {
+ case OpCode.create:
+ case OpCode.create2:
+ case OpCode.createTTL:
+ case OpCode.createContainer: {
+ pRequest2TxnCreate(type, request, record, deserialize);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `PrepRequestProcessor.pRequest2Txn(...)` indirectly reads without synchronization from `server.PrepRequestProcessor.failCreate`. Potentially races with write in method `PrepRequestProcessor.setFailCreate(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ReferenceCountedACLCache.java
##########
@@ -0,0 +1,226 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.Index;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.data.ACL;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+
+public class ReferenceCountedACLCache {
+ private static final Logger LOG = LoggerFactory.getLogger(ReferenceCountedACLCache.class);
+
+ final Map<Long, List<ACL>> longKeyMap =
+ new HashMap<Long, List<ACL>>();
+
+ final Map<List<ACL>, Long> aclKeyMap =
+ new HashMap<List<ACL>, Long>();
+
+ final Map<Long, AtomicLongWithEquals> referenceCounter =
+ new HashMap<Long, AtomicLongWithEquals>();
+ private static final long OPEN_UNSAFE_ACL_ID = -1L;
+
+ /**
+ * these are the number of acls that we have in the datatree
+ */
+ long aclIndex = 0;
+
+ /**
+ * converts the list of acls to a long.
+ * Increments the reference counter for this ACL.
+ * @param acls
+ * @return a long that map to the acls
+ */
+ public synchronized Long convertAcls(List<ACL> acls) {
+ if (acls == null)
+ return OPEN_UNSAFE_ACL_ID;
+
+ // get the value from the map
+ Long ret = aclKeyMap.get(acls);
+ if (ret == null) {
+ ret = incrementIndex();
+ longKeyMap.put(ret, acls);
+ aclKeyMap.put(acls, ret);
+ }
+
+ addUsage(ret);
+
+ return ret;
+ }
+
+ /**
+ * converts a long to a list of acls.
+ *
+ * @param longVal
+ * @return a list of ACLs that map to the long
+ */
+ public synchronized List<ACL> convertLong(Long longVal) {
+ if (longVal == null)
+ return null;
+ if (longVal == OPEN_UNSAFE_ACL_ID)
+ return ZooDefs.Ids.OPEN_ACL_UNSAFE;
+ List<ACL> acls = longKeyMap.get(longVal);
+ if (acls == null) {
+ LOG.error("ERROR: ACL not available for long " + longVal);
+ throw new RuntimeException("Failed to fetch acls for " + longVal);
+ }
+ return acls;
+ }
+
+ private long incrementIndex() {
+ return ++aclIndex;
+ }
+
+ public synchronized void deserialize(InputArchive ia) throws IOException {
+ clear();
+ int i = ia.readInt("map");
+ while (i > 0) {
+ Long val = ia.readLong("long");
+ if (aclIndex < val) {
+ aclIndex = val;
+ }
+ List<ACL> aclList = new ArrayList<ACL>();
+ Index j = ia.startVector("acls");
+ if (j == null) {
+ throw new RuntimeException("Incorrent format of InputArchive when deserialize DataTree - missing acls");
+ }
+ while (!j.done()) {
+ ACL acl = new ACL();
+ acl.deserialize(ia, "acl");
+ aclList.add(acl);
+ j.incr();
+ }
+ longKeyMap.put(val, aclList);
+ aclKeyMap.put(aclList, val);
+ referenceCounter.put(val, new AtomicLongWithEquals(0));
+ i--;
+ }
+ }
+
+ public synchronized void serialize(OutputArchive oa) throws IOException {
+ oa.writeInt(longKeyMap.size(), "map");
+ Set<Map.Entry<Long, List<ACL>>> set = longKeyMap.entrySet();
+ for (Map.Entry<Long, List<ACL>> val : set) {
+ oa.writeLong(val.getKey(), "long");
+ List<ACL> aclList = val.getValue();
+ oa.startVector(aclList, "acls");
+ for (ACL acl : aclList) {
+ acl.serialize(oa, "acl");
+ }
+ oa.endVector(aclList, "acls");
+ }
+ }
+
+ public int size() {
+ return aclKeyMap.size();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ReferenceCountedACLCache.size()` reads without synchronization from container `this.aclKeyMap` via call to `Map.size()`. Potentially races with write in method `ReferenceCountedACLCache.purgeUnused()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java
##########
@@ -0,0 +1,652 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock;
+
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.NoNodeException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog.PlayBackListener;
+import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
+import org.apache.zookeeper.server.quorum.Leader;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPacket;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class maintains the in memory database of zookeeper
+ * server states that includes the sessions, datatree and the
+ * committed logs. It is booted up after reading the logs
+ * and snapshots from the disk.
+ */
+public class ZKDatabase {
+
+ private static final Logger LOG = LoggerFactory.getLogger(ZKDatabase.class);
+
+ /**
+ * make sure on a clear you take care of
+ * all these members.
+ */
+ protected DataTree dataTree;
+ protected ConcurrentHashMap<Long, Integer> sessionsWithTimeouts;
+ protected FileTxnSnapLog snapLog;
+ protected long minCommittedLog, maxCommittedLog;
+
+ /**
+ * Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot
+ */
+ public static final String SNAPSHOT_SIZE_FACTOR = "zookeeper.snapshotSizeFactor";
+ public static final double DEFAULT_SNAPSHOT_SIZE_FACTOR = 0.33;
+ private double snapshotSizeFactor;
+
+ public static final int commitLogCount = 500;
+ protected static int commitLogBuffer = 700;
+ protected LinkedList<Proposal> committedLog = new LinkedList<Proposal>();
+ protected ReentrantReadWriteLock logLock = new ReentrantReadWriteLock();
+ volatile private boolean initialized = false;
+
+ /**
+ * the filetxnsnaplog that this zk database
+ * maps to. There is a one to one relationship
+ * between a filetxnsnaplog and zkdatabase.
+ * @param snapLog the FileTxnSnapLog mapping this zkdatabase
+ */
+ public ZKDatabase(FileTxnSnapLog snapLog) {
+ dataTree = createDataTree();
+ sessionsWithTimeouts = new ConcurrentHashMap<Long, Integer>();
+ this.snapLog = snapLog;
+
+ try {
+ snapshotSizeFactor = Double.parseDouble(
+ System.getProperty(SNAPSHOT_SIZE_FACTOR,
+ Double.toString(DEFAULT_SNAPSHOT_SIZE_FACTOR)));
+ if (snapshotSizeFactor > 1) {
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ LOG.warn("The configured {} is invalid, going to use " +
+ "the default {}", SNAPSHOT_SIZE_FACTOR,
+ DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ }
+ } catch (NumberFormatException e) {
+ LOG.error("Error parsing {}, using default value {}",
+ SNAPSHOT_SIZE_FACTOR, DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ }
+ LOG.info("{} = {}", SNAPSHOT_SIZE_FACTOR, snapshotSizeFactor);
+ }
+
+ /**
+ * checks to see if the zk database has been
+ * initialized or not.
+ * @return true if zk database is initialized and false if not
+ */
+ public boolean isInitialized() {
+ return initialized;
+ }
+
+ /**
+ * clear the zkdatabase.
+ * Note to developers - be careful to see that
+ * the clear method does clear out all the
+ * data structures in zkdatabase.
+ */
+ public void clear() {
+ minCommittedLog = 0;
+ maxCommittedLog = 0;
+ /* to be safe we just create a new
+ * datatree.
+ */
+ dataTree = createDataTree();
+ sessionsWithTimeouts.clear();
+ WriteLock lock = logLock.writeLock();
+ try {
+ lock.lock();
+ committedLog.clear();
+ } finally {
+ lock.unlock();
+ }
+ initialized = false;
+ }
+
+ /**
+ * the datatree for this zkdatabase
+ * @return the datatree for this zkdatabase
+ */
+ public DataTree getDataTree() {
+ return this.dataTree;
+ }
+
+ /**
+ * the committed log for this zk database
+ * @return the committed log for this zkdatabase
+ */
+ public long getmaxCommittedLog() {
+ return maxCommittedLog;
+ }
+
+
+ /**
+ * the minimum committed transaction log
+ * available in memory
+ * @return the minimum committed transaction
+ * log available in memory
+ */
+ public long getminCommittedLog() {
+ return minCommittedLog;
+ }
+ /**
+ * Get the lock that controls the committedLog. If you want to get the pointer to the committedLog, you need
+ * to use this lock to acquire a read lock before calling getCommittedLog()
+ * @return the lock that controls the committed log
+ */
+ public ReentrantReadWriteLock getLogLock() {
+ return logLock;
+ }
+
+
+ public synchronized List<Proposal> getCommittedLog() {
+ ReadLock rl = logLock.readLock();
+ // only make a copy if this thread isn't already holding a lock
+ if(logLock.getReadHoldCount() <=0) {
+ try {
+ rl.lock();
+ return new LinkedList<Proposal>(this.committedLog);
+ } finally {
+ rl.unlock();
+ }
+ }
+ return this.committedLog;
+ }
+
+ /**
+ * get the last processed zxid from a datatree
+ * @return the last processed zxid of a datatree
+ */
+ public long getDataTreeLastProcessedZxid() {
+ return dataTree.lastProcessedZxid;
+ }
+
+ /**
+ * return the sessions in the datatree
+ * @return the data tree sessions
+ */
+ public Collection<Long> getSessions() {
+ return dataTree.getSessions();
+ }
+
+ /**
+ * get sessions with timeouts
+ * @return the hashmap of sessions with timeouts
+ */
+ public ConcurrentHashMap<Long, Integer> getSessionWithTimeOuts() {
+ return sessionsWithTimeouts;
+ }
+
+ private final PlayBackListener commitProposalPlaybackListener = new PlayBackListener() {
+ public void onTxnLoaded(TxnHeader hdr, Record txn){
+ addCommittedProposal(hdr, txn);
+ }
+ };
+
+ /**
+ * load the database from the disk onto memory and also add
+ * the transactions to the committedlog in memory.
+ * @return the last valid zxid on disk
+ * @throws IOException
+ */
+ public long loadDataBase() throws IOException {
+ long zxid = snapLog.restore(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ /**
+ * Fast forward the database adding transactions from the committed log into memory.
+ * @return the last valid zxid.
+ * @throws IOException
+ */
+ public long fastForwardDataBase() throws IOException {
+ long zxid = snapLog.fastForwardFromEdits(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ private void addCommittedProposal(TxnHeader hdr, Record txn) {
+ Request r = new Request(0, hdr.getCxid(), hdr.getType(), hdr, txn, hdr.getZxid());
+ addCommittedProposal(r);
+ }
+
+ /**
+ * maintains a list of last <i>committedLog</i>
+ * or so committed requests. This is used for
+ * fast follower synchronization.
+ * @param request committed request
+ */
+ public void addCommittedProposal(Request request) {
+ WriteLock wl = logLock.writeLock();
+ try {
+ wl.lock();
+ if (committedLog.size() > commitLogCount) {
+ committedLog.removeFirst();
+ minCommittedLog = committedLog.getFirst().packet.getZxid();
+ }
+ if (committedLog.isEmpty()) {
+ minCommittedLog = request.zxid;
+ maxCommittedLog = request.zxid;
+ }
+
+ byte[] data = SerializeUtils.serializeRequest(request);
+ QuorumPacket pp = new QuorumPacket(Leader.PROPOSAL, request.zxid, data, null);
+ Proposal p = new Proposal();
+ p.packet = pp;
+ p.request = request;
+ committedLog.add(p);
+ maxCommittedLog = p.packet.getZxid();
+ } finally {
+ wl.unlock();
+ }
+ }
+
+ public boolean isTxnLogSyncEnabled() {
+ boolean enabled = snapshotSizeFactor >= 0;
+ if (enabled) {
+ LOG.info("On disk txn sync enabled with snapshotSizeFactor "
+ + snapshotSizeFactor);
+ } else {
+ LOG.info("On disk txn sync disabled");
+ }
+ return enabled;
+ }
+
+ public long calculateTxnLogSizeLimit() {
+ long snapSize = 0;
+ try {
+ File snapFile = snapLog.findMostRecentSnapshot();
+ if (snapFile != null) {
+ snapSize = snapFile.length();
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to get size of most recent snapshot");
+ }
+ return (long) (snapSize * snapshotSizeFactor);
+ }
+
+ /**
+ * Get proposals from txnlog. Only packet part of proposal is populated.
+ *
+ * @param startZxid the starting zxid of the proposal
+ * @param sizeLimit maximum on-disk size of txnlog to fetch
+ * 0 is unlimited, negative value means disable.
+ * @return list of proposal (request part of each proposal is null)
+ */
+ public Iterator<Proposal> getProposalsFromTxnLog(long startZxid,
+ long sizeLimit) {
+ if (sizeLimit < 0) {
+ LOG.debug("Negative size limit - retrieving proposal via txnlog is disabled");
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ TxnIterator itr = null;
+ try {
+
+ itr = snapLog.readTxnLog(startZxid, false);
+
+ // If we cannot guarantee that this is strictly the starting txn
+ // after a given zxid, we should fail.
+ if ((itr.getHeader() != null)
+ && (itr.getHeader().getZxid() > startZxid)) {
+ LOG.warn("Unable to find proposals from txnlog for zxid: "
+ + startZxid);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ if (sizeLimit > 0) {
+ long txnSize = itr.getStorageSize();
+ if (txnSize > sizeLimit) {
+ LOG.info("Txnlog size: " + txnSize + " exceeds sizeLimit: "
+ + sizeLimit);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to read txnlog from disk", e);
+ try {
+ if (itr != null) {
+ itr.close();
+ }
+ } catch (IOException ioe) {
+ LOG.warn("Error closing file iterator", ioe);
+ }
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ return new TxnLogProposalIterator(itr);
+ }
+
+ public List<ACL> aclForNode(DataNode n) {
+ return dataTree.getACL(n);
+ }
+ /**
+ * remove a cnxn from the datatree
+ * @param cnxn the cnxn to remove from the datatree
+ */
+ public void removeCnxn(ServerCnxn cnxn) {
+ dataTree.removeCnxn(cnxn);
+ }
+
+ /**
+ * kill a given session in the datatree
+ * @param sessionId the session id to be killed
+ * @param zxid the zxid of kill session transaction
+ */
+ public void killSession(long sessionId, long zxid) {
+ dataTree.killSession(sessionId, zxid);
+ }
+
+ /**
+ * write a text dump of all the ephemerals in the datatree
+ * @param pwriter the output to write to
+ */
+ public void dumpEphemerals(PrintWriter pwriter) {
+ dataTree.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return dataTree.getEphemerals();
+ }
+
+ /**
+ * the node count of the datatree
+ * @return the node count of datatree
+ */
+ public int getNodeCount() {
+ return dataTree.getNodeCount();
+ }
+
+ /**
+ * the paths for ephemeral session id
+ * @param sessionId the session id for which paths match to
+ * @return the paths for a session id
+ */
+ public Set<String> getEphemerals(long sessionId) {
+ return dataTree.getEphemerals(sessionId);
+ }
+
+ /**
+ * the last processed zxid in the datatree
+ * @param zxid the last processed zxid in the datatree
+ */
+ public void setlastProcessedZxid(long zxid) {
+ dataTree.lastProcessedZxid = zxid;
+ }
+
+ /**
+ * the process txn on the data
+ * @param hdr the txnheader for the txn
+ * @param txn the transaction that needs to be processed
+ * @return the result of processing the transaction on this
+ * datatree/zkdatabase
+ */
+ public ProcessTxnResult processTxn(TxnHeader hdr, Record txn) {
+ return dataTree.processTxn(hdr, txn);
+ }
+
+ /**
+ * stat the path
+ * @param path the path for which stat is to be done
+ * @param serverCnxn the servercnxn attached to this request
+ * @return the stat of this node
+ * @throws KeeperException.NoNodeException
+ */
+ public Stat statNode(String path, ServerCnxn serverCnxn) throws KeeperException.NoNodeException {
+ return dataTree.statNode(path, serverCnxn);
+ }
+
+ /**
+ * get the datanode for this path
+ * @param path the path to lookup
+ * @return the datanode for getting the path
+ */
+ public DataNode getNode(String path) {
+ return dataTree.getNode(path);
+ }
+
+ /**
+ * get data and stat for a path
+ * @param path the path being queried
+ * @param stat the stat for this path
+ * @param watcher the watcher function
+ * @return
+ * @throws KeeperException.NoNodeException
+ */
+ public byte[] getData(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ return dataTree.getData(path, stat, watcher);
+ }
+
+ /**
+ * set watches on the datatree
+ * @param relativeZxid the relative zxid that client has seen
+ * @param dataWatches the data watches the client wants to reset
+ * @param existWatches the exists watches the client wants to reset
+ * @param childWatches the child watches the client wants to reset
+ * @param watcher the watcher function
+ */
+ public void setWatches(long relativeZxid, List<String> dataWatches,
+ List<String> existWatches, List<String> childWatches, Watcher watcher) {
+ dataTree.setWatches(relativeZxid, dataWatches, existWatches, childWatches, watcher);
+ }
+
+ /**
+ * get acl for a path
+ * @param path the path to query for acl
+ * @param stat the stat for the node
+ * @return the acl list for this path
+ * @throws NoNodeException
+ */
+ public List<ACL> getACL(String path, Stat stat) throws NoNodeException {
+ return dataTree.getACL(path, stat);
+ }
+
+ /**
+ * get children list for this path
+ * @param path the path of the node
+ * @param stat the stat of the node
+ * @param watcher the watcher function for this path
+ * @return the list of children for this path
+ * @throws KeeperException.NoNodeException
+ */
+ public List<String> getChildren(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ return dataTree.getChildren(path, stat, watcher);
+ }
+
+ /**
+ * check if the path is special or not
+ * @param path the input path
+ * @return true if path is special and false if not
+ */
+ public boolean isSpecialPath(String path) {
+ return dataTree.isSpecialPath(path);
+ }
+
+ /**
+ * get the acl size of the datatree
+ * @return the acl size of the datatree
+ */
+ public int getAclSize() {
+ return dataTree.aclCacheSize();
+ }
+
+ /**
+ * Truncate the ZKDatabase to the specified zxid
+ * @param zxid the zxid to truncate zk database to
+ * @return true if the truncate is successful and false if not
+ * @throws IOException
+ */
+ public boolean truncateLog(long zxid) throws IOException {
+ clear();
+
+ // truncate the log
+ boolean truncated = snapLog.truncateLog(zxid);
+
+ if (!truncated) {
+ return false;
+ }
+
+ loadDataBase();
+ return true;
+ }
+
+ /**
+ * deserialize a snapshot from an input archive
+ * @param ia the input archive you want to deserialize from
+ * @throws IOException
+ */
+ public void deserializeSnapshot(InputArchive ia) throws IOException {
+ clear();
+ SerializeUtils.deserializeSnapshot(getDataTree(),ia,getSessionWithTimeOuts());
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZKDatabase.deserializeSnapshot(...)` indirectly reads without synchronization from `this.dataTree`. Potentially races with write in method `ZKDatabase.clear()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java
##########
@@ -0,0 +1,652 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock;
+
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.NoNodeException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog.PlayBackListener;
+import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
+import org.apache.zookeeper.server.quorum.Leader;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPacket;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class maintains the in memory database of zookeeper
+ * server states that includes the sessions, datatree and the
+ * committed logs. It is booted up after reading the logs
+ * and snapshots from the disk.
+ */
+public class ZKDatabase {
+
+ private static final Logger LOG = LoggerFactory.getLogger(ZKDatabase.class);
+
+ /**
+ * make sure on a clear you take care of
+ * all these members.
+ */
+ protected DataTree dataTree;
+ protected ConcurrentHashMap<Long, Integer> sessionsWithTimeouts;
+ protected FileTxnSnapLog snapLog;
+ protected long minCommittedLog, maxCommittedLog;
+
+ /**
+ * Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot
+ */
+ public static final String SNAPSHOT_SIZE_FACTOR = "zookeeper.snapshotSizeFactor";
+ public static final double DEFAULT_SNAPSHOT_SIZE_FACTOR = 0.33;
+ private double snapshotSizeFactor;
+
+ public static final int commitLogCount = 500;
+ protected static int commitLogBuffer = 700;
+ protected LinkedList<Proposal> committedLog = new LinkedList<Proposal>();
+ protected ReentrantReadWriteLock logLock = new ReentrantReadWriteLock();
+ volatile private boolean initialized = false;
+
+ /**
+ * the filetxnsnaplog that this zk database
+ * maps to. There is a one to one relationship
+ * between a filetxnsnaplog and zkdatabase.
+ * @param snapLog the FileTxnSnapLog mapping this zkdatabase
+ */
+ public ZKDatabase(FileTxnSnapLog snapLog) {
+ dataTree = createDataTree();
+ sessionsWithTimeouts = new ConcurrentHashMap<Long, Integer>();
+ this.snapLog = snapLog;
+
+ try {
+ snapshotSizeFactor = Double.parseDouble(
+ System.getProperty(SNAPSHOT_SIZE_FACTOR,
+ Double.toString(DEFAULT_SNAPSHOT_SIZE_FACTOR)));
+ if (snapshotSizeFactor > 1) {
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ LOG.warn("The configured {} is invalid, going to use " +
+ "the default {}", SNAPSHOT_SIZE_FACTOR,
+ DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ }
+ } catch (NumberFormatException e) {
+ LOG.error("Error parsing {}, using default value {}",
+ SNAPSHOT_SIZE_FACTOR, DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ }
+ LOG.info("{} = {}", SNAPSHOT_SIZE_FACTOR, snapshotSizeFactor);
+ }
+
+ /**
+ * checks to see if the zk database has been
+ * initialized or not.
+ * @return true if zk database is initialized and false if not
+ */
+ public boolean isInitialized() {
+ return initialized;
+ }
+
+ /**
+ * clear the zkdatabase.
+ * Note to developers - be careful to see that
+ * the clear method does clear out all the
+ * data structures in zkdatabase.
+ */
+ public void clear() {
+ minCommittedLog = 0;
+ maxCommittedLog = 0;
+ /* to be safe we just create a new
+ * datatree.
+ */
+ dataTree = createDataTree();
+ sessionsWithTimeouts.clear();
+ WriteLock lock = logLock.writeLock();
+ try {
+ lock.lock();
+ committedLog.clear();
+ } finally {
+ lock.unlock();
+ }
+ initialized = false;
+ }
+
+ /**
+ * the datatree for this zkdatabase
+ * @return the datatree for this zkdatabase
+ */
+ public DataTree getDataTree() {
+ return this.dataTree;
+ }
+
+ /**
+ * the committed log for this zk database
+ * @return the committed log for this zkdatabase
+ */
+ public long getmaxCommittedLog() {
+ return maxCommittedLog;
+ }
+
+
+ /**
+ * the minimum committed transaction log
+ * available in memory
+ * @return the minimum committed transaction
+ * log available in memory
+ */
+ public long getminCommittedLog() {
+ return minCommittedLog;
+ }
+ /**
+ * Get the lock that controls the committedLog. If you want to get the pointer to the committedLog, you need
+ * to use this lock to acquire a read lock before calling getCommittedLog()
+ * @return the lock that controls the committed log
+ */
+ public ReentrantReadWriteLock getLogLock() {
+ return logLock;
+ }
+
+
+ public synchronized List<Proposal> getCommittedLog() {
+ ReadLock rl = logLock.readLock();
+ // only make a copy if this thread isn't already holding a lock
+ if(logLock.getReadHoldCount() <=0) {
+ try {
+ rl.lock();
+ return new LinkedList<Proposal>(this.committedLog);
+ } finally {
+ rl.unlock();
+ }
+ }
+ return this.committedLog;
+ }
+
+ /**
+ * get the last processed zxid from a datatree
+ * @return the last processed zxid of a datatree
+ */
+ public long getDataTreeLastProcessedZxid() {
+ return dataTree.lastProcessedZxid;
+ }
+
+ /**
+ * return the sessions in the datatree
+ * @return the data tree sessions
+ */
+ public Collection<Long> getSessions() {
+ return dataTree.getSessions();
+ }
+
+ /**
+ * get sessions with timeouts
+ * @return the hashmap of sessions with timeouts
+ */
+ public ConcurrentHashMap<Long, Integer> getSessionWithTimeOuts() {
+ return sessionsWithTimeouts;
+ }
+
+ private final PlayBackListener commitProposalPlaybackListener = new PlayBackListener() {
+ public void onTxnLoaded(TxnHeader hdr, Record txn){
+ addCommittedProposal(hdr, txn);
+ }
+ };
+
+ /**
+ * load the database from the disk onto memory and also add
+ * the transactions to the committedlog in memory.
+ * @return the last valid zxid on disk
+ * @throws IOException
+ */
+ public long loadDataBase() throws IOException {
+ long zxid = snapLog.restore(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ /**
+ * Fast forward the database adding transactions from the committed log into memory.
+ * @return the last valid zxid.
+ * @throws IOException
+ */
+ public long fastForwardDataBase() throws IOException {
+ long zxid = snapLog.fastForwardFromEdits(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ private void addCommittedProposal(TxnHeader hdr, Record txn) {
+ Request r = new Request(0, hdr.getCxid(), hdr.getType(), hdr, txn, hdr.getZxid());
+ addCommittedProposal(r);
+ }
+
+ /**
+ * maintains a list of last <i>committedLog</i>
+ * or so committed requests. This is used for
+ * fast follower synchronization.
+ * @param request committed request
+ */
+ public void addCommittedProposal(Request request) {
+ WriteLock wl = logLock.writeLock();
+ try {
+ wl.lock();
+ if (committedLog.size() > commitLogCount) {
+ committedLog.removeFirst();
+ minCommittedLog = committedLog.getFirst().packet.getZxid();
+ }
+ if (committedLog.isEmpty()) {
+ minCommittedLog = request.zxid;
+ maxCommittedLog = request.zxid;
+ }
+
+ byte[] data = SerializeUtils.serializeRequest(request);
+ QuorumPacket pp = new QuorumPacket(Leader.PROPOSAL, request.zxid, data, null);
+ Proposal p = new Proposal();
+ p.packet = pp;
+ p.request = request;
+ committedLog.add(p);
+ maxCommittedLog = p.packet.getZxid();
+ } finally {
+ wl.unlock();
+ }
+ }
+
+ public boolean isTxnLogSyncEnabled() {
+ boolean enabled = snapshotSizeFactor >= 0;
+ if (enabled) {
+ LOG.info("On disk txn sync enabled with snapshotSizeFactor "
+ + snapshotSizeFactor);
+ } else {
+ LOG.info("On disk txn sync disabled");
+ }
+ return enabled;
+ }
+
+ public long calculateTxnLogSizeLimit() {
+ long snapSize = 0;
+ try {
+ File snapFile = snapLog.findMostRecentSnapshot();
+ if (snapFile != null) {
+ snapSize = snapFile.length();
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to get size of most recent snapshot");
+ }
+ return (long) (snapSize * snapshotSizeFactor);
+ }
+
+ /**
+ * Get proposals from txnlog. Only packet part of proposal is populated.
+ *
+ * @param startZxid the starting zxid of the proposal
+ * @param sizeLimit maximum on-disk size of txnlog to fetch
+ * 0 is unlimited, negative value means disable.
+ * @return list of proposal (request part of each proposal is null)
+ */
+ public Iterator<Proposal> getProposalsFromTxnLog(long startZxid,
+ long sizeLimit) {
+ if (sizeLimit < 0) {
+ LOG.debug("Negative size limit - retrieving proposal via txnlog is disabled");
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ TxnIterator itr = null;
+ try {
+
+ itr = snapLog.readTxnLog(startZxid, false);
+
+ // If we cannot guarantee that this is strictly the starting txn
+ // after a given zxid, we should fail.
+ if ((itr.getHeader() != null)
+ && (itr.getHeader().getZxid() > startZxid)) {
+ LOG.warn("Unable to find proposals from txnlog for zxid: "
+ + startZxid);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ if (sizeLimit > 0) {
+ long txnSize = itr.getStorageSize();
+ if (txnSize > sizeLimit) {
+ LOG.info("Txnlog size: " + txnSize + " exceeds sizeLimit: "
+ + sizeLimit);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to read txnlog from disk", e);
+ try {
+ if (itr != null) {
+ itr.close();
+ }
+ } catch (IOException ioe) {
+ LOG.warn("Error closing file iterator", ioe);
+ }
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ return new TxnLogProposalIterator(itr);
+ }
+
+ public List<ACL> aclForNode(DataNode n) {
+ return dataTree.getACL(n);
+ }
+ /**
+ * remove a cnxn from the datatree
+ * @param cnxn the cnxn to remove from the datatree
+ */
+ public void removeCnxn(ServerCnxn cnxn) {
+ dataTree.removeCnxn(cnxn);
+ }
+
+ /**
+ * kill a given session in the datatree
+ * @param sessionId the session id to be killed
+ * @param zxid the zxid of kill session transaction
+ */
+ public void killSession(long sessionId, long zxid) {
+ dataTree.killSession(sessionId, zxid);
+ }
+
+ /**
+ * write a text dump of all the ephemerals in the datatree
+ * @param pwriter the output to write to
+ */
+ public void dumpEphemerals(PrintWriter pwriter) {
+ dataTree.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return dataTree.getEphemerals();
+ }
+
+ /**
+ * the node count of the datatree
+ * @return the node count of datatree
+ */
+ public int getNodeCount() {
+ return dataTree.getNodeCount();
+ }
+
+ /**
+ * the paths for ephemeral session id
+ * @param sessionId the session id for which paths match to
+ * @return the paths for a session id
+ */
+ public Set<String> getEphemerals(long sessionId) {
+ return dataTree.getEphemerals(sessionId);
+ }
+
+ /**
+ * the last processed zxid in the datatree
+ * @param zxid the last processed zxid in the datatree
+ */
+ public void setlastProcessedZxid(long zxid) {
+ dataTree.lastProcessedZxid = zxid;
+ }
+
+ /**
+ * the process txn on the data
+ * @param hdr the txnheader for the txn
+ * @param txn the transaction that needs to be processed
+ * @return the result of processing the transaction on this
+ * datatree/zkdatabase
+ */
+ public ProcessTxnResult processTxn(TxnHeader hdr, Record txn) {
+ return dataTree.processTxn(hdr, txn);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZKDatabase.processTxn(...)` reads without synchronization from `this.dataTree`. Potentially races with write in method `ZKDatabase.clear()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java
##########
@@ -0,0 +1,652 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock;
+
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.NoNodeException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog.PlayBackListener;
+import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
+import org.apache.zookeeper.server.quorum.Leader;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPacket;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class maintains the in memory database of zookeeper
+ * server states that includes the sessions, datatree and the
+ * committed logs. It is booted up after reading the logs
+ * and snapshots from the disk.
+ */
+public class ZKDatabase {
+
+ private static final Logger LOG = LoggerFactory.getLogger(ZKDatabase.class);
+
+ /**
+ * make sure on a clear you take care of
+ * all these members.
+ */
+ protected DataTree dataTree;
+ protected ConcurrentHashMap<Long, Integer> sessionsWithTimeouts;
+ protected FileTxnSnapLog snapLog;
+ protected long minCommittedLog, maxCommittedLog;
+
+ /**
+ * Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot
+ */
+ public static final String SNAPSHOT_SIZE_FACTOR = "zookeeper.snapshotSizeFactor";
+ public static final double DEFAULT_SNAPSHOT_SIZE_FACTOR = 0.33;
+ private double snapshotSizeFactor;
+
+ public static final int commitLogCount = 500;
+ protected static int commitLogBuffer = 700;
+ protected LinkedList<Proposal> committedLog = new LinkedList<Proposal>();
+ protected ReentrantReadWriteLock logLock = new ReentrantReadWriteLock();
+ volatile private boolean initialized = false;
+
+ /**
+ * the filetxnsnaplog that this zk database
+ * maps to. There is a one to one relationship
+ * between a filetxnsnaplog and zkdatabase.
+ * @param snapLog the FileTxnSnapLog mapping this zkdatabase
+ */
+ public ZKDatabase(FileTxnSnapLog snapLog) {
+ dataTree = createDataTree();
+ sessionsWithTimeouts = new ConcurrentHashMap<Long, Integer>();
+ this.snapLog = snapLog;
+
+ try {
+ snapshotSizeFactor = Double.parseDouble(
+ System.getProperty(SNAPSHOT_SIZE_FACTOR,
+ Double.toString(DEFAULT_SNAPSHOT_SIZE_FACTOR)));
+ if (snapshotSizeFactor > 1) {
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ LOG.warn("The configured {} is invalid, going to use " +
+ "the default {}", SNAPSHOT_SIZE_FACTOR,
+ DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ }
+ } catch (NumberFormatException e) {
+ LOG.error("Error parsing {}, using default value {}",
+ SNAPSHOT_SIZE_FACTOR, DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ }
+ LOG.info("{} = {}", SNAPSHOT_SIZE_FACTOR, snapshotSizeFactor);
+ }
+
+ /**
+ * checks to see if the zk database has been
+ * initialized or not.
+ * @return true if zk database is initialized and false if not
+ */
+ public boolean isInitialized() {
+ return initialized;
+ }
+
+ /**
+ * clear the zkdatabase.
+ * Note to developers - be careful to see that
+ * the clear method does clear out all the
+ * data structures in zkdatabase.
+ */
+ public void clear() {
+ minCommittedLog = 0;
+ maxCommittedLog = 0;
+ /* to be safe we just create a new
+ * datatree.
+ */
+ dataTree = createDataTree();
+ sessionsWithTimeouts.clear();
+ WriteLock lock = logLock.writeLock();
+ try {
+ lock.lock();
+ committedLog.clear();
+ } finally {
+ lock.unlock();
+ }
+ initialized = false;
+ }
+
+ /**
+ * the datatree for this zkdatabase
+ * @return the datatree for this zkdatabase
+ */
+ public DataTree getDataTree() {
+ return this.dataTree;
+ }
+
+ /**
+ * the committed log for this zk database
+ * @return the committed log for this zkdatabase
+ */
+ public long getmaxCommittedLog() {
+ return maxCommittedLog;
+ }
+
+
+ /**
+ * the minimum committed transaction log
+ * available in memory
+ * @return the minimum committed transaction
+ * log available in memory
+ */
+ public long getminCommittedLog() {
+ return minCommittedLog;
+ }
+ /**
+ * Get the lock that controls the committedLog. If you want to get the pointer to the committedLog, you need
+ * to use this lock to acquire a read lock before calling getCommittedLog()
+ * @return the lock that controls the committed log
+ */
+ public ReentrantReadWriteLock getLogLock() {
+ return logLock;
+ }
+
+
+ public synchronized List<Proposal> getCommittedLog() {
+ ReadLock rl = logLock.readLock();
+ // only make a copy if this thread isn't already holding a lock
+ if(logLock.getReadHoldCount() <=0) {
+ try {
+ rl.lock();
+ return new LinkedList<Proposal>(this.committedLog);
+ } finally {
+ rl.unlock();
+ }
+ }
+ return this.committedLog;
+ }
+
+ /**
+ * get the last processed zxid from a datatree
+ * @return the last processed zxid of a datatree
+ */
+ public long getDataTreeLastProcessedZxid() {
+ return dataTree.lastProcessedZxid;
+ }
+
+ /**
+ * return the sessions in the datatree
+ * @return the data tree sessions
+ */
+ public Collection<Long> getSessions() {
+ return dataTree.getSessions();
+ }
+
+ /**
+ * get sessions with timeouts
+ * @return the hashmap of sessions with timeouts
+ */
+ public ConcurrentHashMap<Long, Integer> getSessionWithTimeOuts() {
+ return sessionsWithTimeouts;
+ }
+
+ private final PlayBackListener commitProposalPlaybackListener = new PlayBackListener() {
+ public void onTxnLoaded(TxnHeader hdr, Record txn){
+ addCommittedProposal(hdr, txn);
+ }
+ };
+
+ /**
+ * load the database from the disk onto memory and also add
+ * the transactions to the committedlog in memory.
+ * @return the last valid zxid on disk
+ * @throws IOException
+ */
+ public long loadDataBase() throws IOException {
+ long zxid = snapLog.restore(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ /**
+ * Fast forward the database adding transactions from the committed log into memory.
+ * @return the last valid zxid.
+ * @throws IOException
+ */
+ public long fastForwardDataBase() throws IOException {
+ long zxid = snapLog.fastForwardFromEdits(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ private void addCommittedProposal(TxnHeader hdr, Record txn) {
+ Request r = new Request(0, hdr.getCxid(), hdr.getType(), hdr, txn, hdr.getZxid());
+ addCommittedProposal(r);
+ }
+
+ /**
+ * maintains a list of last <i>committedLog</i>
+ * or so committed requests. This is used for
+ * fast follower synchronization.
+ * @param request committed request
+ */
+ public void addCommittedProposal(Request request) {
+ WriteLock wl = logLock.writeLock();
+ try {
+ wl.lock();
+ if (committedLog.size() > commitLogCount) {
+ committedLog.removeFirst();
+ minCommittedLog = committedLog.getFirst().packet.getZxid();
+ }
+ if (committedLog.isEmpty()) {
+ minCommittedLog = request.zxid;
+ maxCommittedLog = request.zxid;
+ }
+
+ byte[] data = SerializeUtils.serializeRequest(request);
+ QuorumPacket pp = new QuorumPacket(Leader.PROPOSAL, request.zxid, data, null);
+ Proposal p = new Proposal();
+ p.packet = pp;
+ p.request = request;
+ committedLog.add(p);
+ maxCommittedLog = p.packet.getZxid();
+ } finally {
+ wl.unlock();
+ }
+ }
+
+ public boolean isTxnLogSyncEnabled() {
+ boolean enabled = snapshotSizeFactor >= 0;
+ if (enabled) {
+ LOG.info("On disk txn sync enabled with snapshotSizeFactor "
+ + snapshotSizeFactor);
+ } else {
+ LOG.info("On disk txn sync disabled");
+ }
+ return enabled;
+ }
+
+ public long calculateTxnLogSizeLimit() {
+ long snapSize = 0;
+ try {
+ File snapFile = snapLog.findMostRecentSnapshot();
+ if (snapFile != null) {
+ snapSize = snapFile.length();
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to get size of most recent snapshot");
+ }
+ return (long) (snapSize * snapshotSizeFactor);
+ }
+
+ /**
+ * Get proposals from txnlog. Only packet part of proposal is populated.
+ *
+ * @param startZxid the starting zxid of the proposal
+ * @param sizeLimit maximum on-disk size of txnlog to fetch
+ * 0 is unlimited, negative value means disable.
+ * @return list of proposal (request part of each proposal is null)
+ */
+ public Iterator<Proposal> getProposalsFromTxnLog(long startZxid,
+ long sizeLimit) {
+ if (sizeLimit < 0) {
+ LOG.debug("Negative size limit - retrieving proposal via txnlog is disabled");
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ TxnIterator itr = null;
+ try {
+
+ itr = snapLog.readTxnLog(startZxid, false);
+
+ // If we cannot guarantee that this is strictly the starting txn
+ // after a given zxid, we should fail.
+ if ((itr.getHeader() != null)
+ && (itr.getHeader().getZxid() > startZxid)) {
+ LOG.warn("Unable to find proposals from txnlog for zxid: "
+ + startZxid);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ if (sizeLimit > 0) {
+ long txnSize = itr.getStorageSize();
+ if (txnSize > sizeLimit) {
+ LOG.info("Txnlog size: " + txnSize + " exceeds sizeLimit: "
+ + sizeLimit);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to read txnlog from disk", e);
+ try {
+ if (itr != null) {
+ itr.close();
+ }
+ } catch (IOException ioe) {
+ LOG.warn("Error closing file iterator", ioe);
+ }
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ return new TxnLogProposalIterator(itr);
+ }
+
+ public List<ACL> aclForNode(DataNode n) {
+ return dataTree.getACL(n);
+ }
+ /**
+ * remove a cnxn from the datatree
+ * @param cnxn the cnxn to remove from the datatree
+ */
+ public void removeCnxn(ServerCnxn cnxn) {
+ dataTree.removeCnxn(cnxn);
+ }
+
+ /**
+ * kill a given session in the datatree
+ * @param sessionId the session id to be killed
+ * @param zxid the zxid of kill session transaction
+ */
+ public void killSession(long sessionId, long zxid) {
+ dataTree.killSession(sessionId, zxid);
+ }
+
+ /**
+ * write a text dump of all the ephemerals in the datatree
+ * @param pwriter the output to write to
+ */
+ public void dumpEphemerals(PrintWriter pwriter) {
+ dataTree.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return dataTree.getEphemerals();
+ }
+
+ /**
+ * the node count of the datatree
+ * @return the node count of datatree
+ */
+ public int getNodeCount() {
+ return dataTree.getNodeCount();
+ }
+
+ /**
+ * the paths for ephemeral session id
+ * @param sessionId the session id for which paths match to
+ * @return the paths for a session id
+ */
+ public Set<String> getEphemerals(long sessionId) {
+ return dataTree.getEphemerals(sessionId);
+ }
+
+ /**
+ * the last processed zxid in the datatree
+ * @param zxid the last processed zxid in the datatree
+ */
+ public void setlastProcessedZxid(long zxid) {
+ dataTree.lastProcessedZxid = zxid;
+ }
+
+ /**
+ * the process txn on the data
+ * @param hdr the txnheader for the txn
+ * @param txn the transaction that needs to be processed
+ * @return the result of processing the transaction on this
+ * datatree/zkdatabase
+ */
+ public ProcessTxnResult processTxn(TxnHeader hdr, Record txn) {
+ return dataTree.processTxn(hdr, txn);
+ }
+
+ /**
+ * stat the path
+ * @param path the path for which stat is to be done
+ * @param serverCnxn the servercnxn attached to this request
+ * @return the stat of this node
+ * @throws KeeperException.NoNodeException
+ */
+ public Stat statNode(String path, ServerCnxn serverCnxn) throws KeeperException.NoNodeException {
+ return dataTree.statNode(path, serverCnxn);
+ }
+
+ /**
+ * get the datanode for this path
+ * @param path the path to lookup
+ * @return the datanode for getting the path
+ */
+ public DataNode getNode(String path) {
+ return dataTree.getNode(path);
+ }
+
+ /**
+ * get data and stat for a path
+ * @param path the path being queried
+ * @param stat the stat for this path
+ * @param watcher the watcher function
+ * @return
+ * @throws KeeperException.NoNodeException
+ */
+ public byte[] getData(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ return dataTree.getData(path, stat, watcher);
+ }
+
+ /**
+ * set watches on the datatree
+ * @param relativeZxid the relative zxid that client has seen
+ * @param dataWatches the data watches the client wants to reset
+ * @param existWatches the exists watches the client wants to reset
+ * @param childWatches the child watches the client wants to reset
+ * @param watcher the watcher function
+ */
+ public void setWatches(long relativeZxid, List<String> dataWatches,
+ List<String> existWatches, List<String> childWatches, Watcher watcher) {
+ dataTree.setWatches(relativeZxid, dataWatches, existWatches, childWatches, watcher);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZKDatabase.setWatches(...)` reads without synchronization from `this.dataTree`. Potentially races with write in method `ZKDatabase.clear()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java
##########
@@ -0,0 +1,652 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock;
+
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.NoNodeException;
+import org.apache.zookeeper.Watcher;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog.PlayBackListener;
+import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
+import org.apache.zookeeper.server.quorum.Leader;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPacket;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class maintains the in memory database of zookeeper
+ * server states that includes the sessions, datatree and the
+ * committed logs. It is booted up after reading the logs
+ * and snapshots from the disk.
+ */
+public class ZKDatabase {
+
+ private static final Logger LOG = LoggerFactory.getLogger(ZKDatabase.class);
+
+ /**
+ * make sure on a clear you take care of
+ * all these members.
+ */
+ protected DataTree dataTree;
+ protected ConcurrentHashMap<Long, Integer> sessionsWithTimeouts;
+ protected FileTxnSnapLog snapLog;
+ protected long minCommittedLog, maxCommittedLog;
+
+ /**
+ * Default value is to use snapshot if txnlog size exceeds 1/3 the size of snapshot
+ */
+ public static final String SNAPSHOT_SIZE_FACTOR = "zookeeper.snapshotSizeFactor";
+ public static final double DEFAULT_SNAPSHOT_SIZE_FACTOR = 0.33;
+ private double snapshotSizeFactor;
+
+ public static final int commitLogCount = 500;
+ protected static int commitLogBuffer = 700;
+ protected LinkedList<Proposal> committedLog = new LinkedList<Proposal>();
+ protected ReentrantReadWriteLock logLock = new ReentrantReadWriteLock();
+ volatile private boolean initialized = false;
+
+ /**
+ * the filetxnsnaplog that this zk database
+ * maps to. There is a one to one relationship
+ * between a filetxnsnaplog and zkdatabase.
+ * @param snapLog the FileTxnSnapLog mapping this zkdatabase
+ */
+ public ZKDatabase(FileTxnSnapLog snapLog) {
+ dataTree = createDataTree();
+ sessionsWithTimeouts = new ConcurrentHashMap<Long, Integer>();
+ this.snapLog = snapLog;
+
+ try {
+ snapshotSizeFactor = Double.parseDouble(
+ System.getProperty(SNAPSHOT_SIZE_FACTOR,
+ Double.toString(DEFAULT_SNAPSHOT_SIZE_FACTOR)));
+ if (snapshotSizeFactor > 1) {
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ LOG.warn("The configured {} is invalid, going to use " +
+ "the default {}", SNAPSHOT_SIZE_FACTOR,
+ DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ }
+ } catch (NumberFormatException e) {
+ LOG.error("Error parsing {}, using default value {}",
+ SNAPSHOT_SIZE_FACTOR, DEFAULT_SNAPSHOT_SIZE_FACTOR);
+ snapshotSizeFactor = DEFAULT_SNAPSHOT_SIZE_FACTOR;
+ }
+ LOG.info("{} = {}", SNAPSHOT_SIZE_FACTOR, snapshotSizeFactor);
+ }
+
+ /**
+ * checks to see if the zk database has been
+ * initialized or not.
+ * @return true if zk database is initialized and false if not
+ */
+ public boolean isInitialized() {
+ return initialized;
+ }
+
+ /**
+ * clear the zkdatabase.
+ * Note to developers - be careful to see that
+ * the clear method does clear out all the
+ * data structures in zkdatabase.
+ */
+ public void clear() {
+ minCommittedLog = 0;
+ maxCommittedLog = 0;
+ /* to be safe we just create a new
+ * datatree.
+ */
+ dataTree = createDataTree();
+ sessionsWithTimeouts.clear();
+ WriteLock lock = logLock.writeLock();
+ try {
+ lock.lock();
+ committedLog.clear();
+ } finally {
+ lock.unlock();
+ }
+ initialized = false;
+ }
+
+ /**
+ * the datatree for this zkdatabase
+ * @return the datatree for this zkdatabase
+ */
+ public DataTree getDataTree() {
+ return this.dataTree;
+ }
+
+ /**
+ * the committed log for this zk database
+ * @return the committed log for this zkdatabase
+ */
+ public long getmaxCommittedLog() {
+ return maxCommittedLog;
+ }
+
+
+ /**
+ * the minimum committed transaction log
+ * available in memory
+ * @return the minimum committed transaction
+ * log available in memory
+ */
+ public long getminCommittedLog() {
+ return minCommittedLog;
+ }
+ /**
+ * Get the lock that controls the committedLog. If you want to get the pointer to the committedLog, you need
+ * to use this lock to acquire a read lock before calling getCommittedLog()
+ * @return the lock that controls the committed log
+ */
+ public ReentrantReadWriteLock getLogLock() {
+ return logLock;
+ }
+
+
+ public synchronized List<Proposal> getCommittedLog() {
+ ReadLock rl = logLock.readLock();
+ // only make a copy if this thread isn't already holding a lock
+ if(logLock.getReadHoldCount() <=0) {
+ try {
+ rl.lock();
+ return new LinkedList<Proposal>(this.committedLog);
+ } finally {
+ rl.unlock();
+ }
+ }
+ return this.committedLog;
+ }
+
+ /**
+ * get the last processed zxid from a datatree
+ * @return the last processed zxid of a datatree
+ */
+ public long getDataTreeLastProcessedZxid() {
+ return dataTree.lastProcessedZxid;
+ }
+
+ /**
+ * return the sessions in the datatree
+ * @return the data tree sessions
+ */
+ public Collection<Long> getSessions() {
+ return dataTree.getSessions();
+ }
+
+ /**
+ * get sessions with timeouts
+ * @return the hashmap of sessions with timeouts
+ */
+ public ConcurrentHashMap<Long, Integer> getSessionWithTimeOuts() {
+ return sessionsWithTimeouts;
+ }
+
+ private final PlayBackListener commitProposalPlaybackListener = new PlayBackListener() {
+ public void onTxnLoaded(TxnHeader hdr, Record txn){
+ addCommittedProposal(hdr, txn);
+ }
+ };
+
+ /**
+ * load the database from the disk onto memory and also add
+ * the transactions to the committedlog in memory.
+ * @return the last valid zxid on disk
+ * @throws IOException
+ */
+ public long loadDataBase() throws IOException {
+ long zxid = snapLog.restore(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ /**
+ * Fast forward the database adding transactions from the committed log into memory.
+ * @return the last valid zxid.
+ * @throws IOException
+ */
+ public long fastForwardDataBase() throws IOException {
+ long zxid = snapLog.fastForwardFromEdits(dataTree, sessionsWithTimeouts, commitProposalPlaybackListener);
+ initialized = true;
+ return zxid;
+ }
+
+ private void addCommittedProposal(TxnHeader hdr, Record txn) {
+ Request r = new Request(0, hdr.getCxid(), hdr.getType(), hdr, txn, hdr.getZxid());
+ addCommittedProposal(r);
+ }
+
+ /**
+ * maintains a list of last <i>committedLog</i>
+ * or so committed requests. This is used for
+ * fast follower synchronization.
+ * @param request committed request
+ */
+ public void addCommittedProposal(Request request) {
+ WriteLock wl = logLock.writeLock();
+ try {
+ wl.lock();
+ if (committedLog.size() > commitLogCount) {
+ committedLog.removeFirst();
+ minCommittedLog = committedLog.getFirst().packet.getZxid();
+ }
+ if (committedLog.isEmpty()) {
+ minCommittedLog = request.zxid;
+ maxCommittedLog = request.zxid;
+ }
+
+ byte[] data = SerializeUtils.serializeRequest(request);
+ QuorumPacket pp = new QuorumPacket(Leader.PROPOSAL, request.zxid, data, null);
+ Proposal p = new Proposal();
+ p.packet = pp;
+ p.request = request;
+ committedLog.add(p);
+ maxCommittedLog = p.packet.getZxid();
+ } finally {
+ wl.unlock();
+ }
+ }
+
+ public boolean isTxnLogSyncEnabled() {
+ boolean enabled = snapshotSizeFactor >= 0;
+ if (enabled) {
+ LOG.info("On disk txn sync enabled with snapshotSizeFactor "
+ + snapshotSizeFactor);
+ } else {
+ LOG.info("On disk txn sync disabled");
+ }
+ return enabled;
+ }
+
+ public long calculateTxnLogSizeLimit() {
+ long snapSize = 0;
+ try {
+ File snapFile = snapLog.findMostRecentSnapshot();
+ if (snapFile != null) {
+ snapSize = snapFile.length();
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to get size of most recent snapshot");
+ }
+ return (long) (snapSize * snapshotSizeFactor);
+ }
+
+ /**
+ * Get proposals from txnlog. Only packet part of proposal is populated.
+ *
+ * @param startZxid the starting zxid of the proposal
+ * @param sizeLimit maximum on-disk size of txnlog to fetch
+ * 0 is unlimited, negative value means disable.
+ * @return list of proposal (request part of each proposal is null)
+ */
+ public Iterator<Proposal> getProposalsFromTxnLog(long startZxid,
+ long sizeLimit) {
+ if (sizeLimit < 0) {
+ LOG.debug("Negative size limit - retrieving proposal via txnlog is disabled");
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ TxnIterator itr = null;
+ try {
+
+ itr = snapLog.readTxnLog(startZxid, false);
+
+ // If we cannot guarantee that this is strictly the starting txn
+ // after a given zxid, we should fail.
+ if ((itr.getHeader() != null)
+ && (itr.getHeader().getZxid() > startZxid)) {
+ LOG.warn("Unable to find proposals from txnlog for zxid: "
+ + startZxid);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+
+ if (sizeLimit > 0) {
+ long txnSize = itr.getStorageSize();
+ if (txnSize > sizeLimit) {
+ LOG.info("Txnlog size: " + txnSize + " exceeds sizeLimit: "
+ + sizeLimit);
+ itr.close();
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ }
+ } catch (IOException e) {
+ LOG.error("Unable to read txnlog from disk", e);
+ try {
+ if (itr != null) {
+ itr.close();
+ }
+ } catch (IOException ioe) {
+ LOG.warn("Error closing file iterator", ioe);
+ }
+ return TxnLogProposalIterator.EMPTY_ITERATOR;
+ }
+ return new TxnLogProposalIterator(itr);
+ }
+
+ public List<ACL> aclForNode(DataNode n) {
+ return dataTree.getACL(n);
+ }
+ /**
+ * remove a cnxn from the datatree
+ * @param cnxn the cnxn to remove from the datatree
+ */
+ public void removeCnxn(ServerCnxn cnxn) {
+ dataTree.removeCnxn(cnxn);
+ }
+
+ /**
+ * kill a given session in the datatree
+ * @param sessionId the session id to be killed
+ * @param zxid the zxid of kill session transaction
+ */
+ public void killSession(long sessionId, long zxid) {
+ dataTree.killSession(sessionId, zxid);
+ }
+
+ /**
+ * write a text dump of all the ephemerals in the datatree
+ * @param pwriter the output to write to
+ */
+ public void dumpEphemerals(PrintWriter pwriter) {
+ dataTree.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return dataTree.getEphemerals();
+ }
+
+ /**
+ * the node count of the datatree
+ * @return the node count of datatree
+ */
+ public int getNodeCount() {
+ return dataTree.getNodeCount();
+ }
+
+ /**
+ * the paths for ephemeral session id
+ * @param sessionId the session id for which paths match to
+ * @return the paths for a session id
+ */
+ public Set<String> getEphemerals(long sessionId) {
+ return dataTree.getEphemerals(sessionId);
+ }
+
+ /**
+ * the last processed zxid in the datatree
+ * @param zxid the last processed zxid in the datatree
+ */
+ public void setlastProcessedZxid(long zxid) {
+ dataTree.lastProcessedZxid = zxid;
+ }
+
+ /**
+ * the process txn on the data
+ * @param hdr the txnheader for the txn
+ * @param txn the transaction that needs to be processed
+ * @return the result of processing the transaction on this
+ * datatree/zkdatabase
+ */
+ public ProcessTxnResult processTxn(TxnHeader hdr, Record txn) {
+ return dataTree.processTxn(hdr, txn);
+ }
+
+ /**
+ * stat the path
+ * @param path the path for which stat is to be done
+ * @param serverCnxn the servercnxn attached to this request
+ * @return the stat of this node
+ * @throws KeeperException.NoNodeException
+ */
+ public Stat statNode(String path, ServerCnxn serverCnxn) throws KeeperException.NoNodeException {
+ return dataTree.statNode(path, serverCnxn);
+ }
+
+ /**
+ * get the datanode for this path
+ * @param path the path to lookup
+ * @return the datanode for getting the path
+ */
+ public DataNode getNode(String path) {
+ return dataTree.getNode(path);
+ }
+
+ /**
+ * get data and stat for a path
+ * @param path the path being queried
+ * @param stat the stat for this path
+ * @param watcher the watcher function
+ * @return
+ * @throws KeeperException.NoNodeException
+ */
+ public byte[] getData(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ return dataTree.getData(path, stat, watcher);
+ }
+
+ /**
+ * set watches on the datatree
+ * @param relativeZxid the relative zxid that client has seen
+ * @param dataWatches the data watches the client wants to reset
+ * @param existWatches the exists watches the client wants to reset
+ * @param childWatches the child watches the client wants to reset
+ * @param watcher the watcher function
+ */
+ public void setWatches(long relativeZxid, List<String> dataWatches,
+ List<String> existWatches, List<String> childWatches, Watcher watcher) {
+ dataTree.setWatches(relativeZxid, dataWatches, existWatches, childWatches, watcher);
+ }
+
+ /**
+ * get acl for a path
+ * @param path the path to query for acl
+ * @param stat the stat for the node
+ * @return the acl list for this path
+ * @throws NoNodeException
+ */
+ public List<ACL> getACL(String path, Stat stat) throws NoNodeException {
+ return dataTree.getACL(path, stat);
+ }
+
+ /**
+ * get children list for this path
+ * @param path the path of the node
+ * @param stat the stat of the node
+ * @param watcher the watcher function for this path
+ * @return the list of children for this path
+ * @throws KeeperException.NoNodeException
+ */
+ public List<String> getChildren(String path, Stat stat, Watcher watcher)
+ throws KeeperException.NoNodeException {
+ return dataTree.getChildren(path, stat, watcher);
+ }
+
+ /**
+ * check if the path is special or not
+ * @param path the input path
+ * @return true if path is special and false if not
+ */
+ public boolean isSpecialPath(String path) {
+ return dataTree.isSpecialPath(path);
+ }
+
+ /**
+ * get the acl size of the datatree
+ * @return the acl size of the datatree
+ */
+ public int getAclSize() {
+ return dataTree.aclCacheSize();
+ }
+
+ /**
+ * Truncate the ZKDatabase to the specified zxid
+ * @param zxid the zxid to truncate zk database to
+ * @return true if the truncate is successful and false if not
+ * @throws IOException
+ */
+ public boolean truncateLog(long zxid) throws IOException {
+ clear();
+
+ // truncate the log
+ boolean truncated = snapLog.truncateLog(zxid);
+
+ if (!truncated) {
+ return false;
+ }
+
+ loadDataBase();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZKDatabase.truncateLog(...)` indirectly reads without synchronization from `this.dataTree`. Potentially races with write in method `ZKDatabase.clear()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.closeSession(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.closeSession(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.createSession(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.expire(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.loadData()` indirectly reads without synchronization from `this.sessionTracker`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
+ boolean validpacket = Request.isValid(si.type);
+ if (validpacket) {
+ firstProcessor.processRequest(si);
+ if (si.cnxn != null) {
+ incInProcess();
+ }
+ } else {
+ LOG.warn("Received packet at server of unknown type " + si.type);
+ new UnimplementedRequestProcessor().processRequest(si);
+ }
+ } catch (MissingSessionException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Dropping request: " + e.getMessage());
+ }
+ } catch (RequestProcessorException e) {
+ LOG.error("Unable to process request:" + e.getMessage(), e);
+ }
+ }
+
+ public static int getSnapCount() {
+ String sc = System.getProperty("zookeeper.snapCount");
+ try {
+ int snapCount = Integer.parseInt(sc);
+
+ // snapCount must be 2 or more. See org.apache.zookeeper.server.SyncRequestProcessor
+ if( snapCount < 2 ) {
+ LOG.warn("SnapCount should be 2 or more. Now, snapCount is reset to 2");
+ snapCount = 2;
+ }
+ return snapCount;
+ } catch (Exception e) {
+ return 100000;
+ }
+ }
+
+ public int getGlobalOutstandingLimit() {
+ String sc = System.getProperty("zookeeper.globalOutstandingLimit");
+ int limit;
+ try {
+ limit = Integer.parseInt(sc);
+ } catch (Exception e) {
+ limit = 1000;
+ }
+ return limit;
+ }
+
+ public void setServerCnxnFactory(ServerCnxnFactory factory) {
+ serverCnxnFactory = factory;
+ }
+
+ public ServerCnxnFactory getServerCnxnFactory() {
+ return serverCnxnFactory;
+ }
+
+ public ServerCnxnFactory getSecureServerCnxnFactory() {
+ return secureServerCnxnFactory;
+ }
+
+ public void setSecureServerCnxnFactory(ServerCnxnFactory factory) {
+ secureServerCnxnFactory = factory;
+ }
+
+ /**
+ * return the last proceesed id from the
+ * datatree
+ */
+ public long getLastProcessedZxid() {
+ return zkDb.getDataTreeLastProcessedZxid();
+ }
+
+ /**
+ * return the outstanding requests
+ * in the queue, which havent been
+ * processed yet
+ */
+ public long getOutstandingRequests() {
+ return getInProcess();
+ }
+
+ /**
+ * return the total number of client connections that are alive
+ * to this server
+ */
+ public int getNumAliveConnections() {
+ int numAliveConnections = 0;
+
+ if (serverCnxnFactory != null) {
+ numAliveConnections += serverCnxnFactory.getNumAliveConnections();
+ }
+
+ if (secureServerCnxnFactory != null) {
+ numAliveConnections += secureServerCnxnFactory.getNumAliveConnections();
+ }
+
+ return numAliveConnections;
+ }
+
+ /**
+ * trunccate the log to get in sync with others
+ * if in a quorum
+ * @param zxid the zxid that it needs to get in sync
+ * with others
+ * @throws IOException
+ */
+ public void truncateLog(long zxid) throws IOException {
+ this.zkDb.truncateLog(zxid);
+ }
+
+ public int getTickTime() {
+ return tickTime;
+ }
+
+ public void setTickTime(int tickTime) {
+ LOG.info("tickTime set to " + tickTime);
+ this.tickTime = tickTime;
+ }
+
+ public int getMinSessionTimeout() {
+ return minSessionTimeout;
+ }
+
+ public void setMinSessionTimeout(int min) {
+ this.minSessionTimeout = min == -1 ? tickTime * 2 : min;
+ LOG.info("minSessionTimeout set to {}", this.minSessionTimeout);
+ }
+
+ public int getMaxSessionTimeout() {
+ return maxSessionTimeout;
+ }
+
+ public void setMaxSessionTimeout(int max) {
+ this.maxSessionTimeout = max == -1 ? tickTime * 20 : max;
+ LOG.info("maxSessionTimeout set to {}", this.maxSessionTimeout);
+ }
+
+ public int getClientPort() {
+ return serverCnxnFactory != null ? serverCnxnFactory.getLocalPort() : -1;
+ }
+
+ public int getSecureClientPort() {
+ return secureServerCnxnFactory != null ? secureServerCnxnFactory.getLocalPort() : -1;
+ }
+
+ /** Maximum number of connections allowed from particular host (ip) */
+ public int getMaxClientCnxnsPerHost() {
+ if (serverCnxnFactory != null) {
+ return serverCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ if (secureServerCnxnFactory != null) {
+ return secureServerCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ return -1;
+ }
+
+ public void setTxnLogFactory(FileTxnSnapLog txnLog) {
+ this.txnLogFactory = txnLog;
+ }
+
+ public FileTxnSnapLog getTxnLogFactory() {
+ return this.txnLogFactory;
+ }
+
+ /**
+ * Returns the elapsed sync of time of transaction log in milliseconds.
+ */
+ public long getTxnLogElapsedSyncTime() {
+ return txnLogFactory.getTxnLogElapsedSyncTime();
+ }
+
+ public String getState() {
+ return "standalone";
+ }
+
+ public void dumpEphemerals(PrintWriter pwriter) {
+ zkDb.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return zkDb.getEphemerals();
+ }
+
+ public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
+ BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
+ ConnectRequest connReq = new ConnectRequest();
+ connReq.deserialize(bia, "connect");
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Session establishment request from client "
+ + cnxn.getRemoteSocketAddress()
+ + " client's lastZxid is 0x"
+ + Long.toHexString(connReq.getLastZxidSeen()));
+ }
+ boolean readOnly = false;
+ try {
+ readOnly = bia.readBool("readOnly");
+ cnxn.isOldClient = false;
+ } catch (IOException e) {
+ // this is ok -- just a packet from an old client which
+ // doesn't contain readOnly field
+ LOG.warn("Connection request from old client "
+ + cnxn.getRemoteSocketAddress()
+ + "; will be dropped if server is in r-o mode");
+ }
+ if (!readOnly && this instanceof ReadOnlyZooKeeperServer) {
+ String msg = "Refusing session request for not-read-only client "
+ + cnxn.getRemoteSocketAddress();
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ if (connReq.getLastZxidSeen() > zkDb.dataTree.lastProcessedZxid) {
+ String msg = "Refusing session request for client "
+ + cnxn.getRemoteSocketAddress()
+ + " as it has seen zxid 0x"
+ + Long.toHexString(connReq.getLastZxidSeen())
+ + " our last zxid is 0x"
+ + Long.toHexString(getZKDatabase().getDataTreeLastProcessedZxid())
+ + " client must try another server";
+
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ int sessionTimeout = connReq.getTimeOut();
+ byte passwd[] = connReq.getPasswd();
+ int minSessionTimeout = getMinSessionTimeout();
+ if (sessionTimeout < minSessionTimeout) {
+ sessionTimeout = minSessionTimeout;
+ }
+ int maxSessionTimeout = getMaxSessionTimeout();
+ if (sessionTimeout > maxSessionTimeout) {
+ sessionTimeout = maxSessionTimeout;
+ }
+ cnxn.setSessionTimeout(sessionTimeout);
+ // We don't want to receive any packets until we are sure that the
+ // session is setup
+ cnxn.disableRecv();
+ long sessionId = connReq.getSessionId();
+ if (sessionId == 0) {
+ long id = createSession(cnxn, passwd, sessionTimeout);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.processConnectRequest(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
+ boolean validpacket = Request.isValid(si.type);
+ if (validpacket) {
+ firstProcessor.processRequest(si);
+ if (si.cnxn != null) {
+ incInProcess();
+ }
+ } else {
+ LOG.warn("Received packet at server of unknown type " + si.type);
+ new UnimplementedRequestProcessor().processRequest(si);
+ }
+ } catch (MissingSessionException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Dropping request: " + e.getMessage());
+ }
+ } catch (RequestProcessorException e) {
+ LOG.error("Unable to process request:" + e.getMessage(), e);
+ }
+ }
+
+ public static int getSnapCount() {
+ String sc = System.getProperty("zookeeper.snapCount");
+ try {
+ int snapCount = Integer.parseInt(sc);
+
+ // snapCount must be 2 or more. See org.apache.zookeeper.server.SyncRequestProcessor
+ if( snapCount < 2 ) {
+ LOG.warn("SnapCount should be 2 or more. Now, snapCount is reset to 2");
+ snapCount = 2;
+ }
+ return snapCount;
+ } catch (Exception e) {
+ return 100000;
+ }
+ }
+
+ public int getGlobalOutstandingLimit() {
+ String sc = System.getProperty("zookeeper.globalOutstandingLimit");
+ int limit;
+ try {
+ limit = Integer.parseInt(sc);
+ } catch (Exception e) {
+ limit = 1000;
+ }
+ return limit;
+ }
+
+ public void setServerCnxnFactory(ServerCnxnFactory factory) {
+ serverCnxnFactory = factory;
+ }
+
+ public ServerCnxnFactory getServerCnxnFactory() {
+ return serverCnxnFactory;
+ }
+
+ public ServerCnxnFactory getSecureServerCnxnFactory() {
+ return secureServerCnxnFactory;
+ }
+
+ public void setSecureServerCnxnFactory(ServerCnxnFactory factory) {
+ secureServerCnxnFactory = factory;
+ }
+
+ /**
+ * return the last proceesed id from the
+ * datatree
+ */
+ public long getLastProcessedZxid() {
+ return zkDb.getDataTreeLastProcessedZxid();
+ }
+
+ /**
+ * return the outstanding requests
+ * in the queue, which havent been
+ * processed yet
+ */
+ public long getOutstandingRequests() {
+ return getInProcess();
+ }
+
+ /**
+ * return the total number of client connections that are alive
+ * to this server
+ */
+ public int getNumAliveConnections() {
+ int numAliveConnections = 0;
+
+ if (serverCnxnFactory != null) {
+ numAliveConnections += serverCnxnFactory.getNumAliveConnections();
+ }
+
+ if (secureServerCnxnFactory != null) {
+ numAliveConnections += secureServerCnxnFactory.getNumAliveConnections();
+ }
+
+ return numAliveConnections;
+ }
+
+ /**
+ * trunccate the log to get in sync with others
+ * if in a quorum
+ * @param zxid the zxid that it needs to get in sync
+ * with others
+ * @throws IOException
+ */
+ public void truncateLog(long zxid) throws IOException {
+ this.zkDb.truncateLog(zxid);
+ }
+
+ public int getTickTime() {
+ return tickTime;
+ }
+
+ public void setTickTime(int tickTime) {
+ LOG.info("tickTime set to " + tickTime);
+ this.tickTime = tickTime;
+ }
+
+ public int getMinSessionTimeout() {
+ return minSessionTimeout;
+ }
+
+ public void setMinSessionTimeout(int min) {
+ this.minSessionTimeout = min == -1 ? tickTime * 2 : min;
+ LOG.info("minSessionTimeout set to {}", this.minSessionTimeout);
+ }
+
+ public int getMaxSessionTimeout() {
+ return maxSessionTimeout;
+ }
+
+ public void setMaxSessionTimeout(int max) {
+ this.maxSessionTimeout = max == -1 ? tickTime * 20 : max;
+ LOG.info("maxSessionTimeout set to {}", this.maxSessionTimeout);
+ }
+
+ public int getClientPort() {
+ return serverCnxnFactory != null ? serverCnxnFactory.getLocalPort() : -1;
+ }
+
+ public int getSecureClientPort() {
+ return secureServerCnxnFactory != null ? secureServerCnxnFactory.getLocalPort() : -1;
+ }
+
+ /** Maximum number of connections allowed from particular host (ip) */
+ public int getMaxClientCnxnsPerHost() {
+ if (serverCnxnFactory != null) {
+ return serverCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ if (secureServerCnxnFactory != null) {
+ return secureServerCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ return -1;
+ }
+
+ public void setTxnLogFactory(FileTxnSnapLog txnLog) {
+ this.txnLogFactory = txnLog;
+ }
+
+ public FileTxnSnapLog getTxnLogFactory() {
+ return this.txnLogFactory;
+ }
+
+ /**
+ * Returns the elapsed sync of time of transaction log in milliseconds.
+ */
+ public long getTxnLogElapsedSyncTime() {
+ return txnLogFactory.getTxnLogElapsedSyncTime();
+ }
+
+ public String getState() {
+ return "standalone";
+ }
+
+ public void dumpEphemerals(PrintWriter pwriter) {
+ zkDb.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return zkDb.getEphemerals();
+ }
+
+ public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
+ BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
+ ConnectRequest connReq = new ConnectRequest();
+ connReq.deserialize(bia, "connect");
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Session establishment request from client "
+ + cnxn.getRemoteSocketAddress()
+ + " client's lastZxid is 0x"
+ + Long.toHexString(connReq.getLastZxidSeen()));
+ }
+ boolean readOnly = false;
+ try {
+ readOnly = bia.readBool("readOnly");
+ cnxn.isOldClient = false;
+ } catch (IOException e) {
+ // this is ok -- just a packet from an old client which
+ // doesn't contain readOnly field
+ LOG.warn("Connection request from old client "
+ + cnxn.getRemoteSocketAddress()
+ + "; will be dropped if server is in r-o mode");
+ }
+ if (!readOnly && this instanceof ReadOnlyZooKeeperServer) {
+ String msg = "Refusing session request for not-read-only client "
+ + cnxn.getRemoteSocketAddress();
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ if (connReq.getLastZxidSeen() > zkDb.dataTree.lastProcessedZxid) {
+ String msg = "Refusing session request for client "
+ + cnxn.getRemoteSocketAddress()
+ + " as it has seen zxid 0x"
+ + Long.toHexString(connReq.getLastZxidSeen())
+ + " our last zxid is 0x"
+ + Long.toHexString(getZKDatabase().getDataTreeLastProcessedZxid())
+ + " client must try another server";
+
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ int sessionTimeout = connReq.getTimeOut();
+ byte passwd[] = connReq.getPasswd();
+ int minSessionTimeout = getMinSessionTimeout();
+ if (sessionTimeout < minSessionTimeout) {
+ sessionTimeout = minSessionTimeout;
+ }
+ int maxSessionTimeout = getMaxSessionTimeout();
+ if (sessionTimeout > maxSessionTimeout) {
+ sessionTimeout = maxSessionTimeout;
+ }
+ cnxn.setSessionTimeout(sessionTimeout);
+ // We don't want to receive any packets until we are sure that the
+ // session is setup
+ cnxn.disableRecv();
+ long sessionId = connReq.getSessionId();
+ if (sessionId == 0) {
+ long id = createSession(cnxn, passwd, sessionTimeout);
+ LOG.debug("Client attempting to establish new session:" +
+ " session = 0x{}, zxid = 0x{}, timeout = {}, address = {}",
+ Long.toHexString(id),
+ Long.toHexString(connReq.getLastZxidSeen()),
+ connReq.getTimeOut(),
+ cnxn.getRemoteSocketAddress());
+ } else {
+ long clientSessionId = connReq.getSessionId();
+ LOG.debug("Client attempting to renew session:" +
+ " session = 0x{}, zxid = 0x{}, timeout = {}, address = {}",
+ Long.toHexString(clientSessionId),
+ Long.toHexString(connReq.getLastZxidSeen()),
+ connReq.getTimeOut(),
+ cnxn.getRemoteSocketAddress());
+ if (serverCnxnFactory != null) {
+ serverCnxnFactory.closeSession(sessionId);
+ }
+ if (secureServerCnxnFactory != null) {
+ secureServerCnxnFactory.closeSession(sessionId);
+ }
+ cnxn.setSessionId(sessionId);
+ reopenSession(cnxn, sessionId, passwd, sessionTimeout);
+ }
+ }
+
+ public boolean shouldThrottle(long outStandingCount) {
+ if (getGlobalOutstandingLimit() < getInProcess()) {
+ return outStandingCount > 0;
+ }
+ return false;
+ }
+
+ public void processPacket(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
+ // We have the request, now process and setup for next
+ InputStream bais = new ByteBufferInputStream(incomingBuffer);
+ BinaryInputArchive bia = BinaryInputArchive.getArchive(bais);
+ RequestHeader h = new RequestHeader();
+ h.deserialize(bia, "header");
+ // Through the magic of byte buffers, txn will not be
+ // pointing
+ // to the start of the txn
+ incomingBuffer = incomingBuffer.slice();
+ if (h.getType() == OpCode.auth) {
+ LOG.info("got auth packet " + cnxn.getRemoteSocketAddress());
+ AuthPacket authPacket = new AuthPacket();
+ ByteBufferInputStream.byteBuffer2Record(incomingBuffer, authPacket);
+ String scheme = authPacket.getScheme();
+ AuthenticationProvider ap = ProviderRegistry.getProvider(scheme);
+ Code authReturn = KeeperException.Code.AUTHFAILED;
+ if(ap != null) {
+ try {
+ authReturn = ap.handleAuthentication(cnxn, authPacket.getAuth());
+ } catch(RuntimeException e) {
+ LOG.warn("Caught runtime exception from AuthenticationProvider: " + scheme + " due to " + e);
+ authReturn = KeeperException.Code.AUTHFAILED;
+ }
+ }
+ if (authReturn == KeeperException.Code.OK) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Authentication succeeded for scheme: " + scheme);
+ }
+ LOG.info("auth success " + cnxn.getRemoteSocketAddress());
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0,
+ KeeperException.Code.OK.intValue());
+ cnxn.sendResponse(rh, null, null);
+ } else {
+ if (ap == null) {
+ LOG.warn("No authentication provider for scheme: "
+ + scheme + " has "
+ + ProviderRegistry.listProviders());
+ } else {
+ LOG.warn("Authentication failed for scheme: " + scheme);
+ }
+ // send a response...
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0,
+ KeeperException.Code.AUTHFAILED.intValue());
+ cnxn.sendResponse(rh, null, null);
+ // ... and close connection
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ cnxn.disableRecv();
+ }
+ return;
+ } else {
+ if (h.getType() == OpCode.sasl) {
+ Record rsp = processSasl(incomingBuffer,cnxn);
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0, KeeperException.Code.OK.intValue());
+ cnxn.sendResponse(rh,rsp, "response"); // not sure about 3rd arg..what is it?
+ return;
+ }
+ else {
+ Request si = new Request(cnxn, cnxn.getSessionId(), h.getXid(),
+ h.getType(), incomingBuffer, cnxn.getAuthInfo());
+ si.setOwner(ServerCnxn.me);
+ // Always treat packet from the client as a possible
+ // local request.
+ setLocalSessionFlag(si);
+ submitRequest(si);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.processPacket(...)` indirectly reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.registerJMX()` reads without synchronization from `this.jmxDataTreeBean`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.registerJMX()` reads without synchronization from `this.jmxServerBean`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.startdata()` indirectly reads without synchronization from `this.sessionTracker`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.submitRequest(...)` indirectly reads without synchronization from `this.sessionTracker`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
+ boolean validpacket = Request.isValid(si.type);
+ if (validpacket) {
+ firstProcessor.processRequest(si);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.submitRequest(...)` reads without synchronization from `this.firstProcessor`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
+ boolean validpacket = Request.isValid(si.type);
+ if (validpacket) {
+ firstProcessor.processRequest(si);
+ if (si.cnxn != null) {
+ incInProcess();
+ }
+ } else {
+ LOG.warn("Received packet at server of unknown type " + si.type);
+ new UnimplementedRequestProcessor().processRequest(si);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `ZooKeeperServer.submitRequest(...)` indirectly writes to field `si.e` outside of synchronization.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.unregisterJMX()` reads without synchronization from `this.jmxDataTreeBean`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ZooKeeperServer.unregisterJMX()` reads without synchronization from `this.jmxServerBean`. Potentially races with write in method `ZooKeeperServer.startup()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/auth/ProviderRegistry.java
##########
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.auth;
+
+import java.util.Enumeration;
+import java.util.HashMap;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.zookeeper.server.ZooKeeperServer;
+
+public class ProviderRegistry {
+ private static final Logger LOG = LoggerFactory.getLogger(ProviderRegistry.class);
+
+ private static boolean initialized = false;
+ private static HashMap<String, AuthenticationProvider> authenticationProviders =
+ new HashMap<String, AuthenticationProvider>();
+
+ public static void initialize() {
+ synchronized (ProviderRegistry.class) {
+ if (initialized)
+ return;
+ IPAuthenticationProvider ipp = new IPAuthenticationProvider();
+ DigestAuthenticationProvider digp = new DigestAuthenticationProvider();
+ authenticationProviders.put(ipp.getScheme(), ipp);
+ authenticationProviders.put(digp.getScheme(), digp);
+ Enumeration<Object> en = System.getProperties().keys();
+ while (en.hasMoreElements()) {
+ String k = (String) en.nextElement();
+ if (k.startsWith("zookeeper.authProvider.")) {
+ String className = System.getProperty(k);
+ try {
+ Class<?> c = ZooKeeperServer.class.getClassLoader()
+ .loadClass(className);
+ AuthenticationProvider ap = (AuthenticationProvider) c.getDeclaredConstructor()
+ .newInstance();
+ authenticationProviders.put(ap.getScheme(), ap);
+ } catch (Exception e) {
+ LOG.warn("Problems loading " + className,e);
+ }
+ }
+ }
+ initialized = true;
+ }
+ }
+
+ public static AuthenticationProvider getProvider(String scheme) {
+ if(!initialized)
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ProviderRegistry.getProvider(...)` reads without synchronization from `auth.ProviderRegistry.initialized`. Potentially races with write in method `ProviderRegistry.initialize()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/auth/ProviderRegistry.java
##########
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.auth;
+
+import java.util.Enumeration;
+import java.util.HashMap;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.zookeeper.server.ZooKeeperServer;
+
+public class ProviderRegistry {
+ private static final Logger LOG = LoggerFactory.getLogger(ProviderRegistry.class);
+
+ private static boolean initialized = false;
+ private static HashMap<String, AuthenticationProvider> authenticationProviders =
+ new HashMap<String, AuthenticationProvider>();
+
+ public static void initialize() {
+ synchronized (ProviderRegistry.class) {
+ if (initialized)
+ return;
+ IPAuthenticationProvider ipp = new IPAuthenticationProvider();
+ DigestAuthenticationProvider digp = new DigestAuthenticationProvider();
+ authenticationProviders.put(ipp.getScheme(), ipp);
+ authenticationProviders.put(digp.getScheme(), digp);
+ Enumeration<Object> en = System.getProperties().keys();
+ while (en.hasMoreElements()) {
+ String k = (String) en.nextElement();
+ if (k.startsWith("zookeeper.authProvider.")) {
+ String className = System.getProperty(k);
+ try {
+ Class<?> c = ZooKeeperServer.class.getClassLoader()
+ .loadClass(className);
+ AuthenticationProvider ap = (AuthenticationProvider) c.getDeclaredConstructor()
+ .newInstance();
+ authenticationProviders.put(ap.getScheme(), ap);
+ } catch (Exception e) {
+ LOG.warn("Problems loading " + className,e);
+ }
+ }
+ }
+ initialized = true;
+ }
+ }
+
+ public static AuthenticationProvider getProvider(String scheme) {
+ if(!initialized)
+ initialize();
+ return authenticationProviders.get(scheme);
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ProviderRegistry.getProvider(...)` reads without synchronization from container `auth.ProviderRegistry.authenticationProviders` via call to `HashMap.get(...)`. Potentially races with write in method `ProviderRegistry.initialize()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/auth/ProviderRegistry.java
##########
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.auth;
+
+import java.util.Enumeration;
+import java.util.HashMap;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.zookeeper.server.ZooKeeperServer;
+
+public class ProviderRegistry {
+ private static final Logger LOG = LoggerFactory.getLogger(ProviderRegistry.class);
+
+ private static boolean initialized = false;
+ private static HashMap<String, AuthenticationProvider> authenticationProviders =
+ new HashMap<String, AuthenticationProvider>();
+
+ public static void initialize() {
+ synchronized (ProviderRegistry.class) {
+ if (initialized)
+ return;
+ IPAuthenticationProvider ipp = new IPAuthenticationProvider();
+ DigestAuthenticationProvider digp = new DigestAuthenticationProvider();
+ authenticationProviders.put(ipp.getScheme(), ipp);
+ authenticationProviders.put(digp.getScheme(), digp);
+ Enumeration<Object> en = System.getProperties().keys();
+ while (en.hasMoreElements()) {
+ String k = (String) en.nextElement();
+ if (k.startsWith("zookeeper.authProvider.")) {
+ String className = System.getProperty(k);
+ try {
+ Class<?> c = ZooKeeperServer.class.getClassLoader()
+ .loadClass(className);
+ AuthenticationProvider ap = (AuthenticationProvider) c.getDeclaredConstructor()
+ .newInstance();
+ authenticationProviders.put(ap.getScheme(), ap);
+ } catch (Exception e) {
+ LOG.warn("Problems loading " + className,e);
+ }
+ }
+ }
+ initialized = true;
+ }
+ }
+
+ public static AuthenticationProvider getProvider(String scheme) {
+ if(!initialized)
+ initialize();
+ return authenticationProviders.get(scheme);
+ }
+
+ public static String listProviders() {
+ StringBuilder sb = new StringBuilder();
+ for(String s: authenticationProviders.keySet()) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `ProviderRegistry.listProviders()` reads without synchronization from container `auth.ProviderRegistry.authenticationProviders` via call to `HashMap.keySet()`. Potentially races with write in method `ProviderRegistry.initialize()`.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java
##########
@@ -0,0 +1,1094 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.IOException;
+import java.nio.BufferUnderflowException;
+import java.nio.ByteBuffer;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.quorum.QuorumCnxManager.Message;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.quorum.QuorumPeer.ServerState;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * Implementation of leader election using TCP. It uses an object of the class
+ * QuorumCnxManager to manage connections. Otherwise, the algorithm is push-based
+ * as with the other UDP implementations.
+ *
+ * There are a few parameters that can be tuned to change its behavior. First,
+ * finalizeWait determines the amount of time to wait until deciding upon a leader.
+ * This is part of the leader election algorithm.
+ */
+
+
+public class FastLeaderElection implements Election {
+ private static final Logger LOG = LoggerFactory.getLogger(FastLeaderElection.class);
+
+ /**
+ * Determine how much time a process has to wait
+ * once it believes that it has reached the end of
+ * leader election.
+ */
+ final static int finalizeWait = 200;
+
+
+ /**
+ * Upper bound on the amount of time between two consecutive
+ * notification checks. This impacts the amount of time to get
+ * the system up again after long partitions. Currently 60 seconds.
+ */
+
+ final static int maxNotificationInterval = 60000;
+
+ /**
+ * Connection manager. Fast leader election uses TCP for
+ * communication between peers, and QuorumCnxManager manages
+ * such connections.
+ */
+
+ QuorumCnxManager manager;
+
+
+ /**
+ * Notifications are messages that let other peers know that
+ * a given peer has changed its vote, either because it has
+ * joined leader election or because it learned of another
+ * peer with higher zxid or same zxid and higher server id
+ */
+
+ static public class Notification {
+ /*
+ * Format version, introduced in 3.4.6
+ */
+
+ public final static int CURRENTVERSION = 0x2;
+ int version;
+
+ /*
+ * Proposed leader
+ */
+ long leader;
+
+ /*
+ * zxid of the proposed leader
+ */
+ long zxid;
+
+ /*
+ * Epoch
+ */
+ long electionEpoch;
+
+ /*
+ * current state of sender
+ */
+ QuorumPeer.ServerState state;
+
+ /*
+ * Address of sender
+ */
+ long sid;
+
+ QuorumVerifier qv;
+ /*
+ * epoch of the proposed leader
+ */
+ long peerEpoch;
+ }
+
+ static byte[] dummyData = new byte[0];
+
+ /**
+ * Messages that a peer wants to send to other peers.
+ * These messages can be both Notifications and Acks
+ * of reception of notification.
+ */
+ static public class ToSend {
+ static enum mType {crequest, challenge, notification, ack}
+
+ ToSend(mType type,
+ long leader,
+ long zxid,
+ long electionEpoch,
+ ServerState state,
+ long sid,
+ long peerEpoch,
+ byte[] configData) {
+
+ this.leader = leader;
+ this.zxid = zxid;
+ this.electionEpoch = electionEpoch;
+ this.state = state;
+ this.sid = sid;
+ this.peerEpoch = peerEpoch;
+ this.configData = configData;
+ }
+
+ /*
+ * Proposed leader in the case of notification
+ */
+ long leader;
+
+ /*
+ * id contains the tag for acks, and zxid for notifications
+ */
+ long zxid;
+
+ /*
+ * Epoch
+ */
+ long electionEpoch;
+
+ /*
+ * Current state;
+ */
+ QuorumPeer.ServerState state;
+
+ /*
+ * Address of recipient
+ */
+ long sid;
+
+ /*
+ * Used to send a QuorumVerifier (configuration info)
+ */
+ byte[] configData = dummyData;
+
+ /*
+ * Leader epoch
+ */
+ long peerEpoch;
+ }
+
+ LinkedBlockingQueue<ToSend> sendqueue;
+ LinkedBlockingQueue<Notification> recvqueue;
+
+ /**
+ * Multi-threaded implementation of message handler. Messenger
+ * implements two sub-classes: WorkReceiver and WorkSender. The
+ * functionality of each is obvious from the name. Each of these
+ * spawns a new thread.
+ */
+
+ protected class Messenger {
+
+ /**
+ * Receives messages from instance of QuorumCnxManager on
+ * method run(), and processes such messages.
+ */
+
+ class WorkerReceiver extends ZooKeeperThread {
+ volatile boolean stop;
+ QuorumCnxManager manager;
+
+ WorkerReceiver(QuorumCnxManager manager) {
+ super("WorkerReceiver");
+ this.stop = false;
+ this.manager = manager;
+ }
+
+ public void run() {
+
+ Message response;
+ while (!stop) {
+ // Sleeps on receive
+ try {
+ response = manager.pollRecvQueue(3000, TimeUnit.MILLISECONDS);
+ if(response == null) continue;
+
+ final int capacity = response.buffer.capacity();
+
+ // The current protocol and two previous generations all send at least 28 bytes
+ if (capacity < 28) {
+ LOG.error("Got a short response from server {}: {}", response.sid, capacity);
+ continue;
+ }
+
+ // this is the backwardCompatibility mode in place before ZK-107
+ // It is for a version of the protocol in which we didn't send peer epoch
+ // With peer epoch and version the message became 40 bytes
+ boolean backCompatibility28 = (capacity == 28);
+
+ // this is the backwardCompatibility mode for no version information
+ boolean backCompatibility40 = (capacity == 40);
+
+ response.buffer.clear();
+
+ // Instantiate Notification and set its attributes
+ Notification n = new Notification();
+
+ int rstate = response.buffer.getInt();
+ long rleader = response.buffer.getLong();
+ long rzxid = response.buffer.getLong();
+ long relectionEpoch = response.buffer.getLong();
+ long rpeerepoch;
+
+ int version = 0x0;
+ QuorumVerifier rqv = null;
+
+ try {
+
+ if (!backCompatibility28) {
+ rpeerepoch = response.buffer.getLong();
+ if (!backCompatibility40) {
+ /*
+ * Version added in 3.4.6
+ */
+
+ version = response.buffer.getInt();
+ } else {
+ LOG.info("Backward compatibility mode (36 bits), server id: {}", response.sid);
+ }
+ } else {
+ LOG.info("Backward compatibility mode (28 bits), server id: {}", response.sid);
+ rpeerepoch = ZxidUtils.getEpochFromZxid(rzxid);
+ }
+
+
+ // check if we have a version that includes config. If so extract config info from message.
+ if (version > 0x1) {
+ int configLength = response.buffer.getInt();
+
+ // we want to avoid errors caused by the allocation of a byte array with negative length
+ // (causing NegativeArraySizeException) or huge length (causing e.g. OutOfMemoryError)
+ if (configLength < 0 || configLength > capacity) {
+ throw new IOException(String.format("Invalid configLength in notification message! sid=%d, capacity=%d, version=%d, configLength=%d",
+ response.sid, capacity, version, configLength));
+ }
+
+ byte b[] = new byte[configLength];
+
+ response.buffer.get(b);
+
+ synchronized (self) {
+ try {
+ rqv = self.configFromString(new String(b));
+ QuorumVerifier curQV = self.getQuorumVerifier();
+ if (rqv.getVersion() > curQV.getVersion()) {
+ LOG.info("{} Received version: {} my version: {}", self.getId(),
+ Long.toHexString(rqv.getVersion()),
+ Long.toHexString(self.getQuorumVerifier().getVersion()));
+ if (self.getPeerState() == ServerState.LOOKING) {
+ LOG.debug("Invoking processReconfig(), state: {}", self.getServerState());
+ self.processReconfig(rqv, null, null, false);
+ if (!rqv.equals(curQV)) {
+ LOG.info("restarting leader election");
+ self.shuttingDownLE = true;
+ self.getElectionAlg().shutdown();
+
+ break;
+ }
+ } else {
+ LOG.debug("Skip processReconfig(), state: {}", self.getServerState());
+ }
+ }
+ } catch (IOException | ConfigException e) {
+ LOG.error("Something went wrong while processing config received from {}. " +
+ "Continue to process the notification message without handling the configuration.", response.sid);
+ }
+ }
+ } else {
+ LOG.info("Backward compatibility mode (before reconfig), server id: {}", response.sid);
+ }
+ } catch (BufferUnderflowException | IOException e) {
+ LOG.warn("Skipping the processing of a partial / malformed response message sent by sid={} (message length: {})",
+ response.sid, capacity, e);
+ continue;
+ }
+
+ /*
+ * If it is from a non-voting server (such as an observer or
+ * a non-voting follower), respond right away.
+ */
+ if(!validVoter(response.sid)) {
+ Vote current = self.getCurrentVote();
+ QuorumVerifier qv = self.getQuorumVerifier();
+ ToSend notmsg = new ToSend(ToSend.mType.notification,
+ current.getId(),
+ current.getZxid(),
+ logicalclock.get(),
+ self.getPeerState(),
+ response.sid,
+ current.getPeerEpoch(),
+ qv.toString().getBytes());
+
+ sendqueue.offer(notmsg);
+ } else {
+ // Receive new message
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Receive new notification message. My id = "
+ + self.getId());
+ }
+
+ // State of peer that sent this message
+ QuorumPeer.ServerState ackstate = QuorumPeer.ServerState.LOOKING;
+ switch (rstate) {
+ case 0:
+ ackstate = QuorumPeer.ServerState.LOOKING;
+ break;
+ case 1:
+ ackstate = QuorumPeer.ServerState.FOLLOWING;
+ break;
+ case 2:
+ ackstate = QuorumPeer.ServerState.LEADING;
+ break;
+ case 3:
+ ackstate = QuorumPeer.ServerState.OBSERVING;
+ break;
+ default:
+ continue;
+ }
+
+ n.leader = rleader;
+ n.zxid = rzxid;
+ n.electionEpoch = relectionEpoch;
+ n.state = ackstate;
+ n.sid = response.sid;
+ n.peerEpoch = rpeerepoch;
+ n.version = version;
+ n.qv = rqv;
+ /*
+ * Print notification info
+ */
+ if(LOG.isInfoEnabled()){
+ printNotification(n);
+ }
+
+ /*
+ * If this server is looking, then send proposed leader
+ */
+
+ if(self.getPeerState() == QuorumPeer.ServerState.LOOKING){
+ recvqueue.offer(n);
+
+ /*
+ * Send a notification back if the peer that sent this
+ * message is also looking and its logical clock is
+ * lagging behind.
+ */
+ if((ackstate == QuorumPeer.ServerState.LOOKING)
+ && (n.electionEpoch < logicalclock.get())){
+ Vote v = getVote();
+ QuorumVerifier qv = self.getQuorumVerifier();
+ ToSend notmsg = new ToSend(ToSend.mType.notification,
+ v.getId(),
+ v.getZxid(),
+ logicalclock.get(),
+ self.getPeerState(),
+ response.sid,
+ v.getPeerEpoch(),
+ qv.toString().getBytes());
+ sendqueue.offer(notmsg);
+ }
+ } else {
+ /*
+ * If this server is not looking, but the one that sent the ack
+ * is looking, then send back what it believes to be the leader.
+ */
+ Vote current = self.getCurrentVote();
+ if(ackstate == QuorumPeer.ServerState.LOOKING){
+ if(LOG.isDebugEnabled()){
+ LOG.debug("Sending new notification. My id ={} recipient={} zxid=0x{} leader={} config version = {}",
+ self.getId(),
+ response.sid,
+ Long.toHexString(current.getZxid()),
+ current.getId(),
+ Long.toHexString(self.getQuorumVerifier().getVersion()));
+ }
+
+ QuorumVerifier qv = self.getQuorumVerifier();
+ ToSend notmsg = new ToSend(
+ ToSend.mType.notification,
+ current.getId(),
+ current.getZxid(),
+ current.getElectionEpoch(),
+ self.getPeerState(),
+ response.sid,
+ current.getPeerEpoch(),
+ qv.toString().getBytes());
+ sendqueue.offer(notmsg);
+ }
+ }
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Interrupted Exception while waiting for new message" +
+ e.toString());
+ }
+ }
+ LOG.info("WorkerReceiver is down");
+ }
+ }
+
+ /**
+ * This worker simply dequeues a message to send and
+ * and queues it on the manager's queue.
+ */
+
+ class WorkerSender extends ZooKeeperThread {
+ volatile boolean stop;
+ QuorumCnxManager manager;
+
+ WorkerSender(QuorumCnxManager manager){
+ super("WorkerSender");
+ this.stop = false;
+ this.manager = manager;
+ }
+
+ public void run() {
+ while (!stop) {
+ try {
+ ToSend m = sendqueue.poll(3000, TimeUnit.MILLISECONDS);
+ if(m == null) continue;
+
+ process(m);
+ } catch (InterruptedException e) {
+ break;
+ }
+ }
+ LOG.info("WorkerSender is down");
+ }
+
+ /**
+ * Called by run() once there is a new message to send.
+ *
+ * @param m message to send
+ */
+ void process(ToSend m) {
+ ByteBuffer requestBuffer = buildMsg(m.state.ordinal(),
+ m.leader,
+ m.zxid,
+ m.electionEpoch,
+ m.peerEpoch,
+ m.configData);
+
+ manager.toSend(m.sid, requestBuffer);
+
+ }
+ }
+
+ WorkerSender ws;
+ WorkerReceiver wr;
+ Thread wsThread = null;
+ Thread wrThread = null;
+
+ /**
+ * Constructor of class Messenger.
+ *
+ * @param manager Connection manager
+ */
+ Messenger(QuorumCnxManager manager) {
+
+ this.ws = new WorkerSender(manager);
+
+ this.wsThread = new Thread(this.ws,
+ "WorkerSender[myid=" + self.getId() + "]");
+ this.wsThread.setDaemon(true);
+
+ this.wr = new WorkerReceiver(manager);
+
+ this.wrThread = new Thread(this.wr,
+ "WorkerReceiver[myid=" + self.getId() + "]");
+ this.wrThread.setDaemon(true);
+ }
+
+ /**
+ * Starts instances of WorkerSender and WorkerReceiver
+ */
+ void start(){
+ this.wsThread.start();
+ this.wrThread.start();
+ }
+
+ /**
+ * Stops instances of WorkerSender and WorkerReceiver
+ */
+ void halt(){
+ this.ws.stop = true;
+ this.wr.stop = true;
+ }
+
+ }
+
+ QuorumPeer self;
+ Messenger messenger;
+ AtomicLong logicalclock = new AtomicLong(); /* Election instance */
+ long proposedLeader;
+ long proposedZxid;
+ long proposedEpoch;
+
+
+ /**
+ * Returns the current vlue of the logical clock counter
+ */
+ public long getLogicalClock(){
+ return logicalclock.get();
+ }
+
+ static ByteBuffer buildMsg(int state,
+ long leader,
+ long zxid,
+ long electionEpoch,
+ long epoch) {
+ byte requestBytes[] = new byte[40];
+ ByteBuffer requestBuffer = ByteBuffer.wrap(requestBytes);
+
+ /*
+ * Building notification packet to send, this is called directly only in tests
+ */
+
+ requestBuffer.clear();
+ requestBuffer.putInt(state);
+ requestBuffer.putLong(leader);
+ requestBuffer.putLong(zxid);
+ requestBuffer.putLong(electionEpoch);
+ requestBuffer.putLong(epoch);
+ requestBuffer.putInt(0x1);
+
+ return requestBuffer;
+ }
+
+ static ByteBuffer buildMsg(int state,
+ long leader,
+ long zxid,
+ long electionEpoch,
+ long epoch,
+ byte[] configData) {
+ byte requestBytes[] = new byte[44 + configData.length];
+ ByteBuffer requestBuffer = ByteBuffer.wrap(requestBytes);
+
+ /*
+ * Building notification packet to send
+ */
+
+ requestBuffer.clear();
+ requestBuffer.putInt(state);
+ requestBuffer.putLong(leader);
+ requestBuffer.putLong(zxid);
+ requestBuffer.putLong(electionEpoch);
+ requestBuffer.putLong(epoch);
+ requestBuffer.putInt(Notification.CURRENTVERSION);
+ requestBuffer.putInt(configData.length);
+ requestBuffer.put(configData);
+
+ return requestBuffer;
+ }
+
+ /**
+ * Constructor of FastLeaderElection. It takes two parameters, one
+ * is the QuorumPeer object that instantiated this object, and the other
+ * is the connection manager. Such an object should be created only once
+ * by each peer during an instance of the ZooKeeper service.
+ *
+ * @param self QuorumPeer that created this object
+ * @param manager Connection manager
+ */
+ public FastLeaderElection(QuorumPeer self, QuorumCnxManager manager){
+ this.stop = false;
+ this.manager = manager;
+ starter(self, manager);
+ }
+
+ /**
+ * This method is invoked by the constructor. Because it is a
+ * part of the starting procedure of the object that must be on
+ * any constructor of this class, it is probably best to keep as
+ * a separate method. As we have a single constructor currently,
+ * it is not strictly necessary to have it separate.
+ *
+ * @param self QuorumPeer that created this object
+ * @param manager Connection manager
+ */
+ private void starter(QuorumPeer self, QuorumCnxManager manager) {
+ this.self = self;
+ proposedLeader = -1;
+ proposedZxid = -1;
+
+ sendqueue = new LinkedBlockingQueue<ToSend>();
+ recvqueue = new LinkedBlockingQueue<Notification>();
+ this.messenger = new Messenger(manager);
+ }
+
+ /**
+ * This method starts the sender and receiver threads.
+ */
+ public void start() {
+ this.messenger.start();
+ }
+
+ private void leaveInstance(Vote v) {
+ if(LOG.isDebugEnabled()){
+ LOG.debug("About to leave FLE instance: leader={}, zxid=0x{}, my id={}, my state={}",
+ v.getId(), Long.toHexString(v.getZxid()), self.getId(), self.getPeerState());
+ }
+ recvqueue.clear();
+ }
+
+ public QuorumCnxManager getCnxManager(){
+ return manager;
+ }
+
+ volatile boolean stop;
+ public void shutdown(){
+ stop = true;
+ proposedLeader = -1;
+ proposedZxid = -1;
+ LOG.debug("Shutting down connection manager");
+ manager.halt();
+ LOG.debug("Shutting down messenger");
+ messenger.halt();
+ LOG.debug("FLE is down");
+ }
+
+ /**
+ * Send notifications to all peers upon a change in our vote
+ */
+ private void sendNotifications() {
+ for (long sid : self.getCurrentAndNextConfigVoters()) {
+ QuorumVerifier qv = self.getQuorumVerifier();
+ ToSend notmsg = new ToSend(ToSend.mType.notification,
+ proposedLeader,
+ proposedZxid,
+ logicalclock.get(),
+ QuorumPeer.ServerState.LOOKING,
+ sid,
+ proposedEpoch, qv.toString().getBytes());
+ if(LOG.isDebugEnabled()){
+ LOG.debug("Sending Notification: " + proposedLeader + " (n.leader), 0x" +
+ Long.toHexString(proposedZxid) + " (n.zxid), 0x" + Long.toHexString(logicalclock.get()) +
+ " (n.round), " + sid + " (recipient), " + self.getId() +
+ " (myid), 0x" + Long.toHexString(proposedEpoch) + " (n.peerEpoch)");
+ }
+ sendqueue.offer(notmsg);
+ }
+ }
+
+ private void printNotification(Notification n){
+ LOG.info("Notification: "
+ + Long.toHexString(n.version) + " (message format version), "
+ + n.leader + " (n.leader), 0x"
+ + Long.toHexString(n.zxid) + " (n.zxid), 0x"
+ + Long.toHexString(n.electionEpoch) + " (n.round), " + n.state
+ + " (n.state), " + n.sid + " (n.sid), 0x"
+ + Long.toHexString(n.peerEpoch) + " (n.peerEPoch), "
+ + self.getPeerState() + " (my state)"
+ + (n.qv!=null ? (Long.toHexString(n.qv.getVersion()) + " (n.config version)"):""));
+ }
+
+
+ /**
+ * Check if a pair (server id, zxid) succeeds our
+ * current vote.
+ *
+ * @param id Server identifier
+ * @param zxid Last zxid observed by the issuer of this vote
+ */
+ protected boolean totalOrderPredicate(long newId, long newZxid, long newEpoch, long curId, long curZxid, long curEpoch) {
+ LOG.debug("id: " + newId + ", proposed id: " + curId + ", zxid: 0x" +
+ Long.toHexString(newZxid) + ", proposed zxid: 0x" + Long.toHexString(curZxid));
+ if(self.getQuorumVerifier().getWeight(newId) == 0){
+ return false;
+ }
+
+ /*
+ * We return true if one of the following three cases hold:
+ * 1- New epoch is higher
+ * 2- New epoch is the same as current epoch, but new zxid is higher
+ * 3- New epoch is the same as current epoch, new zxid is the same
+ * as current zxid, but server id is higher.
+ */
+
+ return ((newEpoch > curEpoch) ||
+ ((newEpoch == curEpoch) &&
+ ((newZxid > curZxid) || ((newZxid == curZxid) && (newId > curId)))));
+ }
+
+ /**
+ * Termination predicate. Given a set of votes, determines if have
+ * sufficient to declare the end of the election round.
+ *
+ * @param votes
+ * Set of votes
+ * @param vote
+ * Identifier of the vote received last
+ */
+ protected boolean termPredicate(Map<Long, Vote> votes, Vote vote) {
+ SyncedLearnerTracker voteSet = new SyncedLearnerTracker();
+ voteSet.addQuorumVerifier(self.getQuorumVerifier());
+ if (self.getLastSeenQuorumVerifier() != null
+ && self.getLastSeenQuorumVerifier().getVersion() > self
+ .getQuorumVerifier().getVersion()) {
+ voteSet.addQuorumVerifier(self.getLastSeenQuorumVerifier());
+ }
+
+ /*
+ * First make the views consistent. Sometimes peers will have different
+ * zxids for a server depending on timing.
+ */
+ for (Map.Entry<Long, Vote> entry : votes.entrySet()) {
+ if (vote.equals(entry.getValue())) {
+ voteSet.addAck(entry.getKey());
+ }
+ }
+
+ return voteSet.hasAllQuorums();
+ }
+
+ /**
+ * In the case there is a leader elected, and a quorum supporting
+ * this leader, we have to check if the leader has voted and acked
+ * that it is leading. We need this check to avoid that peers keep
+ * electing over and over a peer that has crashed and it is no
+ * longer leading.
+ *
+ * @param votes set of votes
+ * @param leader leader id
+ * @param electionEpoch epoch id
+ */
+ protected boolean checkLeader(
+ Map<Long, Vote> votes,
+ long leader,
+ long electionEpoch){
+
+ boolean predicate = true;
+
+ /*
+ * If everyone else thinks I'm the leader, I must be the leader.
+ * The other two checks are just for the case in which I'm not the
+ * leader. If I'm not the leader and I haven't received a message
+ * from leader stating that it is leading, then predicate is false.
+ */
+
+ if(leader != self.getId()){
+ if(votes.get(leader) == null) predicate = false;
+ else if(votes.get(leader).getState() != ServerState.LEADING) predicate = false;
+ } else if(logicalclock.get() != electionEpoch) {
+ predicate = false;
+ }
+
+ return predicate;
+ }
+
+ synchronized void updateProposal(long leader, long zxid, long epoch){
+ if(LOG.isDebugEnabled()){
+ LOG.debug("Updating proposal: " + leader + " (newleader), 0x"
+ + Long.toHexString(zxid) + " (newzxid), " + proposedLeader
+ + " (oldleader), 0x" + Long.toHexString(proposedZxid) + " (oldzxid)");
+ }
+ proposedLeader = leader;
+ proposedZxid = zxid;
+ proposedEpoch = epoch;
+ }
+
+ synchronized public Vote getVote(){
+ return new Vote(proposedLeader, proposedZxid, proposedEpoch);
+ }
+
+ /**
+ * A learning state can be either FOLLOWING or OBSERVING.
+ * This method simply decides which one depending on the
+ * role of the server.
+ *
+ * @return ServerState
+ */
+ private ServerState learningState(){
+ if(self.getLearnerType() == LearnerType.PARTICIPANT){
+ LOG.debug("I'm a participant: " + self.getId());
+ return ServerState.FOLLOWING;
+ }
+ else{
+ LOG.debug("I'm an observer: " + self.getId());
+ return ServerState.OBSERVING;
+ }
+ }
+
+ /**
+ * Returns the initial vote value of server identifier.
+ *
+ * @return long
+ */
+ private long getInitId(){
+ if(self.getQuorumVerifier().getVotingMembers().containsKey(self.getId()))
+ return self.getId();
+ else return Long.MIN_VALUE;
+ }
+
+ /**
+ * Returns initial last logged zxid.
+ *
+ * @return long
+ */
+ private long getInitLastLoggedZxid(){
+ if(self.getLearnerType() == LearnerType.PARTICIPANT)
+ return self.getLastLoggedZxid();
+ else return Long.MIN_VALUE;
+ }
+
+ /**
+ * Returns the initial vote value of the peer epoch.
+ *
+ * @return long
+ */
+ private long getPeerEpoch(){
+ if(self.getLearnerType() == LearnerType.PARTICIPANT)
+ try {
+ return self.getCurrentEpoch();
+ } catch(IOException e) {
+ RuntimeException re = new RuntimeException(e.getMessage());
+ re.setStackTrace(e.getStackTrace());
+ throw re;
+ }
+ else return Long.MIN_VALUE;
+ }
+
+ /**
+ * Starts a new round of leader election. Whenever our QuorumPeer
+ * changes its state to LOOKING, this method is invoked, and it
+ * sends notifications to all other peers.
+ */
+ public Vote lookForLeader() throws InterruptedException {
+ try {
+ self.jmxLeaderElectionBean = new LeaderElectionBean();
+ MBeanRegistry.getInstance().register(
+ self.jmxLeaderElectionBean, self.jmxLocalPeerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ self.jmxLeaderElectionBean = null;
+ }
+ if (self.start_fle == 0) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `FastLeaderElection.lookForLeader()` reads without synchronization from `this.self.start_fle`. Potentially races with write in method `FastLeaderElection.lookForLeader()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Leader.java
##########
@@ -0,0 +1,1437 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.IOException;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.net.SocketException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.FinalRequestProcessor;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.RequestProcessor;
+import org.apache.zookeeper.server.ZooKeeperCriticalThread;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class has the control logic for the Leader.
+ */
+public class Leader {
+ private static final Logger LOG = LoggerFactory.getLogger(Leader.class);
+
+ static final private boolean nodelay = System.getProperty("leader.nodelay", "true").equals("true");
+ static {
+ LOG.info("TCP NoDelay set to: " + nodelay);
+ }
+
+ static public class Proposal extends SyncedLearnerTracker {
+ public QuorumPacket packet;
+ public Request request;
+
+ @Override
+ public String toString() {
+ return packet.getType() + ", " + packet.getZxid() + ", " + request;
+ }
+ }
+
+ // Throttle when there are too many concurrent snapshots being sent to observers
+ private static final String MAX_CONCURRENT_SNAPSHOTS = "zookeeper.leader.maxConcurrentSnapshots";
+ private static final int maxConcurrentSnapshots;
+ private static final String MAX_CONCURRENT_SNAPSHOT_TIMEOUT = "zookeeper.leader.maxConcurrentSnapshotTimeout";
+ private static final long maxConcurrentSnapshotTimeout;
+ static {
+ maxConcurrentSnapshots = Integer.getInteger(MAX_CONCURRENT_SNAPSHOTS, 10);
+ LOG.info(MAX_CONCURRENT_SNAPSHOTS + " = " + maxConcurrentSnapshots);
+ maxConcurrentSnapshotTimeout = Long.getLong(MAX_CONCURRENT_SNAPSHOT_TIMEOUT, 5);
+ LOG.info(MAX_CONCURRENT_SNAPSHOT_TIMEOUT + " = " + maxConcurrentSnapshotTimeout);
+ }
+
+ private final LearnerSnapshotThrottler learnerSnapshotThrottler;
+
+ final LeaderZooKeeperServer zk;
+
+ final QuorumPeer self;
+
+ // VisibleForTesting
+ protected boolean quorumFormed = false;
+
+ // the follower acceptor thread
+ volatile LearnerCnxAcceptor cnxAcceptor = null;
+
+ // list of all the followers
+ private final HashSet<LearnerHandler> learners =
+ new HashSet<LearnerHandler>();
+
+ private final BufferStats proposalStats;
+
+ public BufferStats getProposalStats() {
+ return proposalStats;
+ }
+
+ public LearnerSnapshotThrottler createLearnerSnapshotThrottler(
+ int maxConcurrentSnapshots, long maxConcurrentSnapshotTimeout) {
+ return new LearnerSnapshotThrottler(
+ maxConcurrentSnapshots, maxConcurrentSnapshotTimeout);
+ }
+
+ /**
+ * Returns a copy of the current learner snapshot
+ */
+ public List<LearnerHandler> getLearners() {
+ synchronized (learners) {
+ return new ArrayList<LearnerHandler>(learners);
+ }
+ }
+
+ // list of followers that are ready to follow (i.e synced with the leader)
+ private final HashSet<LearnerHandler> forwardingFollowers =
+ new HashSet<LearnerHandler>();
+
+ /**
+ * Returns a copy of the current forwarding follower snapshot
+ */
+ public List<LearnerHandler> getForwardingFollowers() {
+ synchronized (forwardingFollowers) {
+ return new ArrayList<LearnerHandler>(forwardingFollowers);
+ }
+ }
+
+ private void addForwardingFollower(LearnerHandler lh) {
+ synchronized (forwardingFollowers) {
+ forwardingFollowers.add(lh);
+ }
+ }
+
+ private final HashSet<LearnerHandler> observingLearners =
+ new HashSet<LearnerHandler>();
+
+ /**
+ * Returns a copy of the current observer snapshot
+ */
+ public List<LearnerHandler> getObservingLearners() {
+ synchronized (observingLearners) {
+ return new ArrayList<LearnerHandler>(observingLearners);
+ }
+ }
+
+ private void addObserverLearnerHandler(LearnerHandler lh) {
+ synchronized (observingLearners) {
+ observingLearners.add(lh);
+ }
+ }
+
+ // Pending sync requests. Must access under 'this' lock.
+ private final HashMap<Long,List<LearnerSyncRequest>> pendingSyncs =
+ new HashMap<Long,List<LearnerSyncRequest>>();
+
+ synchronized public int getNumPendingSyncs() {
+ return pendingSyncs.size();
+ }
+
+ //Follower counter
+ final AtomicLong followerCounter = new AtomicLong(-1);
+
+ /**
+ * Adds peer to the leader.
+ *
+ * @param learner
+ * instance of learner handle
+ */
+ void addLearnerHandler(LearnerHandler learner) {
+ synchronized (learners) {
+ learners.add(learner);
+ }
+ }
+
+ /**
+ * Remove the learner from the learner list
+ *
+ * @param peer
+ */
+ void removeLearnerHandler(LearnerHandler peer) {
+ synchronized (forwardingFollowers) {
+ forwardingFollowers.remove(peer);
+ }
+ synchronized (learners) {
+ learners.remove(peer);
+ }
+ synchronized (observingLearners) {
+ observingLearners.remove(peer);
+ }
+ }
+
+ boolean isLearnerSynced(LearnerHandler peer){
+ synchronized (forwardingFollowers) {
+ return forwardingFollowers.contains(peer);
+ }
+ }
+
+
+ /**
+ * Returns true if a quorum in qv is connected and synced with the leader
+ * and false otherwise
+ *
+ * @param qv, a QuorumVerifier
+ */
+ public boolean isQuorumSynced(QuorumVerifier qv) {
+ HashSet<Long> ids = new HashSet<Long>();
+ if (qv.getVotingMembers().containsKey(self.getId()))
+ ids.add(self.getId());
+ synchronized (forwardingFollowers) {
+ for (LearnerHandler learnerHandler: forwardingFollowers){
+ if (learnerHandler.synced() && qv.getVotingMembers().containsKey(learnerHandler.getSid())){
+ ids.add(learnerHandler.getSid());
+ }
+ }
+ }
+ return qv.containsQuorum(ids);
+ }
+
+ private final ServerSocket ss;
+
+ Leader(QuorumPeer self,LeaderZooKeeperServer zk) throws IOException {
+ this.self = self;
+ this.proposalStats = new BufferStats();
+ try {
+ if (self.shouldUsePortUnification() || self.isSslQuorum()) {
+ boolean allowInsecureConnection = self.shouldUsePortUnification();
+ if (self.getQuorumListenOnAllIPs()) {
+ ss = new UnifiedServerSocket(self.getX509Util(), allowInsecureConnection, self.getQuorumAddress().getPort());
+ } else {
+ ss = new UnifiedServerSocket(self.getX509Util(), allowInsecureConnection);
+ }
+ } else {
+ if (self.getQuorumListenOnAllIPs()) {
+ ss = new ServerSocket(self.getQuorumAddress().getPort());
+ } else {
+ ss = new ServerSocket();
+ }
+ }
+ ss.setReuseAddress(true);
+ if (!self.getQuorumListenOnAllIPs()) {
+ ss.bind(self.getQuorumAddress());
+ }
+ } catch (BindException e) {
+ if (self.getQuorumListenOnAllIPs()) {
+ LOG.error("Couldn't bind to port " + self.getQuorumAddress().getPort(), e);
+ } else {
+ LOG.error("Couldn't bind to " + self.getQuorumAddress(), e);
+ }
+ throw e;
+ }
+ this.zk = zk;
+ this.learnerSnapshotThrottler = createLearnerSnapshotThrottler(
+ maxConcurrentSnapshots, maxConcurrentSnapshotTimeout);
+ }
+
+ /**
+ * This message is for follower to expect diff
+ */
+ final static int DIFF = 13;
+
+ /**
+ * This is for follower to truncate its logs
+ */
+ final static int TRUNC = 14;
+
+ /**
+ * This is for follower to download the snapshots
+ */
+ final static int SNAP = 15;
+
+ /**
+ * This tells the leader that the connecting peer is actually an observer
+ */
+ final static int OBSERVERINFO = 16;
+
+ /**
+ * This message type is sent by the leader to indicate it's zxid and if
+ * needed, its database.
+ */
+ final static int NEWLEADER = 10;
+
+ /**
+ * This message type is sent by a follower to pass the last zxid. This is here
+ * for backward compatibility purposes.
+ */
+ final static int FOLLOWERINFO = 11;
+
+ /**
+ * This message type is sent by the leader to indicate that the follower is
+ * now uptodate andt can start responding to clients.
+ */
+ final static int UPTODATE = 12;
+
+ /**
+ * This message is the first that a follower receives from the leader.
+ * It has the protocol version and the epoch of the leader.
+ */
+ public static final int LEADERINFO = 17;
+
+ /**
+ * This message is used by the follow to ack a proposed epoch.
+ */
+ public static final int ACKEPOCH = 18;
+
+ /**
+ * This message type is sent to a leader to request and mutation operation.
+ * The payload will consist of a request header followed by a request.
+ */
+ final static int REQUEST = 1;
+
+ /**
+ * This message type is sent by a leader to propose a mutation.
+ */
+ public final static int PROPOSAL = 2;
+
+ /**
+ * This message type is sent by a follower after it has synced a proposal.
+ */
+ final static int ACK = 3;
+
+ /**
+ * This message type is sent by a leader to commit a proposal and cause
+ * followers to start serving the corresponding data.
+ */
+ final static int COMMIT = 4;
+
+ /**
+ * This message type is enchanged between follower and leader (initiated by
+ * follower) to determine liveliness.
+ */
+ final static int PING = 5;
+
+ /**
+ * This message type is to validate a session that should be active.
+ */
+ final static int REVALIDATE = 6;
+
+ /**
+ * This message is a reply to a synchronize command flushing the pipe
+ * between the leader and the follower.
+ */
+ final static int SYNC = 7;
+
+ /**
+ * This message type informs observers of a committed proposal.
+ */
+ final static int INFORM = 8;
+
+ /**
+ * Similar to COMMIT, only for a reconfig operation.
+ */
+ final static int COMMITANDACTIVATE = 9;
+
+ /**
+ * Similar to INFORM, only for a reconfig operation.
+ */
+ final static int INFORMANDACTIVATE = 19;
+
+ final ConcurrentMap<Long, Proposal> outstandingProposals = new ConcurrentHashMap<Long, Proposal>();
+
+ private final ConcurrentLinkedQueue<Proposal> toBeApplied = new ConcurrentLinkedQueue<Proposal>();
+
+ // VisibleForTesting
+ protected final Proposal newLeaderProposal = new Proposal();
+
+ class LearnerCnxAcceptor extends ZooKeeperCriticalThread {
+ private volatile boolean stop = false;
+
+ public LearnerCnxAcceptor() {
+ super("LearnerCnxAcceptor-" + ss.getLocalSocketAddress(), zk
+ .getZooKeeperServerListener());
+ }
+
+ @Override
+ public void run() {
+ try {
+ while (!stop) {
+ Socket s = null;
+ boolean error = false;
+ try {
+ s = ss.accept();
+
+ // start with the initLimit, once the ack is processed
+ // in LearnerHandler switch to the syncLimit
+ s.setSoTimeout(self.tickTime * self.initLimit);
+ s.setTcpNoDelay(nodelay);
+
+ BufferedInputStream is = new BufferedInputStream(
+ s.getInputStream());
+ LearnerHandler fh = new LearnerHandler(s, is, Leader.this);
+ fh.start();
+ } catch (SocketException e) {
+ error = true;
+ if (stop) {
+ LOG.info("exception while shutting down acceptor: "
+ + e);
+
+ // When Leader.shutdown() calls ss.close(),
+ // the call to accept throws an exception.
+ // We catch and set stop to true.
+ stop = true;
+ } else {
+ throw e;
+ }
+ } catch (SaslException e){
+ LOG.error("Exception while connecting to quorum learner", e);
+ error = true;
+ } catch (Exception e) {
+ error = true;
+ throw e;
+ } finally {
+ // Don't leak sockets on errors
+ if (error && s != null && !s.isClosed()) {
+ try {
+ s.close();
+ } catch (IOException e) {
+ LOG.warn("Error closing socket", e);
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Exception while accepting follower", e.getMessage());
+ handleException(this.getName(), e);
+ }
+ }
+
+ public void halt() {
+ stop = true;
+ }
+ }
+
+ StateSummary leaderStateSummary;
+
+ long epoch = -1;
+ boolean waitingForNewEpoch = true;
+
+ // when a reconfig occurs where the leader is removed or becomes an observer,
+ // it does not commit ops after committing the reconfig
+ boolean allowedToCommit = true;
+ /**
+ * This method is main function that is called to lead
+ *
+ * @throws IOException
+ * @throws InterruptedException
+ */
+ void lead() throws IOException, InterruptedException {
+ self.end_fle = Time.currentElapsedTime();
+ long electionTimeTaken = self.end_fle - self.start_fle;
+ self.setElectionTimeTaken(electionTimeTaken);
+ LOG.info("LEADING - LEADER ELECTION TOOK - {} {}", electionTimeTaken,
+ QuorumPeer.FLE_TIME_UNIT);
+ self.start_fle = 0;
+ self.end_fle = 0;
+
+ zk.registerJMX(new LeaderBean(this, zk), self.jmxLocalPeerBean);
+
+ try {
+ self.tick.set(0);
+ zk.loadData();
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `Leader.lead()` indirectly reads without synchronization from `this.zk.sessionTracker`. Potentially races with write in method `Leader.lead()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Learner.java
##########
@@ -0,0 +1,670 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.ConnectException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.nio.ByteBuffer;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.concurrent.ConcurrentHashMap;
+
+import javax.net.ssl.SSLSocket;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.ServerCnxn;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class is the superclass of two of the three main actors in a ZK
+ * ensemble: Followers and Observers. Both Followers and Observers share
+ * a good deal of code which is moved into Peer to avoid duplication.
+ */
+public class Learner {
+ static class PacketInFlight {
+ TxnHeader hdr;
+ Record rec;
+ }
+ QuorumPeer self;
+ LearnerZooKeeperServer zk;
+
+ protected BufferedOutputStream bufferedOutput;
+
+ protected Socket sock;
+
+ /**
+ * Socket getter
+ * @return
+ */
+ public Socket getSocket() {
+ return sock;
+ }
+
+ protected InputArchive leaderIs;
+ protected OutputArchive leaderOs;
+ /** the protocol version of the leader */
+ protected int leaderProtocolVersion = 0x01;
+
+ protected static final Logger LOG = LoggerFactory.getLogger(Learner.class);
+
+ static final private boolean nodelay = System.getProperty("follower.nodelay", "true").equals("true");
+ static {
+ LOG.info("TCP NoDelay set to: " + nodelay);
+ }
+
+ final ConcurrentHashMap<Long, ServerCnxn> pendingRevalidations =
+ new ConcurrentHashMap<Long, ServerCnxn>();
+
+ public int getPendingRevalidationsCount() {
+ return pendingRevalidations.size();
+ }
+
+ /**
+ * validate a session for a client
+ *
+ * @param clientId
+ * the client to be revalidated
+ * @param timeout
+ * the timeout for which the session is valid
+ * @return
+ * @throws IOException
+ */
+ void validateSession(ServerCnxn cnxn, long clientId, int timeout)
+ throws IOException {
+ LOG.info("Revalidating client: 0x" + Long.toHexString(clientId));
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(baos);
+ dos.writeLong(clientId);
+ dos.writeInt(timeout);
+ dos.close();
+ QuorumPacket qp = new QuorumPacket(Leader.REVALIDATE, -1, baos
+ .toByteArray(), null);
+ pendingRevalidations.put(clientId, cnxn);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG,
+ ZooTrace.SESSION_TRACE_MASK,
+ "To validate session 0x"
+ + Long.toHexString(clientId));
+ }
+ writePacket(qp, true);
+ }
+
+ /**
+ * write a packet to the leader
+ *
+ * @param pp
+ * the proposal packet to be sent to the leader
+ * @throws IOException
+ */
+ void writePacket(QuorumPacket pp, boolean flush) throws IOException {
+ synchronized (leaderOs) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `Learner.writePacket(...)` reads without synchronization from `this.leaderOs`. Potentially races with write in method `Learner.connectToLeader(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LearnerHandler.java
##########
@@ -0,0 +1,1000 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.Socket;
+import java.nio.ByteBuffer;
+import java.util.Iterator;
+import java.util.Queue;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.TxnLogProposalIterator;
+import org.apache.zookeeper.server.ZKDatabase;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * There will be an instance of this class created by the Leader for each
+ * learner. All communication with a learner is handled by this
+ * class.
+ */
+public class LearnerHandler extends ZooKeeperThread {
+ private static final Logger LOG = LoggerFactory.getLogger(LearnerHandler.class);
+
+ protected final Socket sock;
+
+ public Socket getSocket() {
+ return sock;
+ }
+
+ final Leader leader;
+
+ /** Deadline for receiving the next ack. If we are bootstrapping then
+ * it's based on the initLimit, if we are done bootstrapping it's based
+ * on the syncLimit. Once the deadline is past this learner should
+ * be considered no longer "sync'd" with the leader. */
+ volatile long tickOfNextAckDeadline;
+
+ /**
+ * ZooKeeper server identifier of this learner
+ */
+ protected long sid = 0;
+
+ long getSid(){
+ return sid;
+ }
+
+ protected int version = 0x1;
+
+ int getVersion() {
+ return version;
+ }
+
+ /**
+ * The packets to be sent to the learner
+ */
+ final LinkedBlockingQueue<QuorumPacket> queuedPackets =
+ new LinkedBlockingQueue<QuorumPacket>();
+
+ /**
+ * This class controls the time that the Leader has been
+ * waiting for acknowledgement of a proposal from this Learner.
+ * If the time is above syncLimit, the connection will be closed.
+ * It keeps track of only one proposal at a time, when the ACK for
+ * that proposal arrives, it switches to the last proposal received
+ * or clears the value if there is no pending proposal.
+ */
+ private class SyncLimitCheck {
+ private boolean started = false;
+ private long currentZxid = 0;
+ private long currentTime = 0;
+ private long nextZxid = 0;
+ private long nextTime = 0;
+
+ public synchronized void start() {
+ started = true;
+ }
+
+ public synchronized void updateProposal(long zxid, long time) {
+ if (!started) {
+ return;
+ }
+ if (currentTime == 0) {
+ currentTime = time;
+ currentZxid = zxid;
+ } else {
+ nextTime = time;
+ nextZxid = zxid;
+ }
+ }
+
+ public synchronized void updateAck(long zxid) {
+ if (currentZxid == zxid) {
+ currentTime = nextTime;
+ currentZxid = nextZxid;
+ nextTime = 0;
+ nextZxid = 0;
+ } else if (nextZxid == zxid) {
+ LOG.warn("ACK for " + zxid + " received before ACK for " + currentZxid + "!!!!");
+ nextTime = 0;
+ nextZxid = 0;
+ }
+ }
+
+ public synchronized boolean check(long time) {
+ if (currentTime == 0) {
+ return true;
+ } else {
+ long msDelay = (time - currentTime) / 1000000;
+ return (msDelay < (leader.self.tickTime * leader.self.syncLimit));
+ }
+ }
+ };
+
+ private SyncLimitCheck syncLimitCheck = new SyncLimitCheck();
+
+ private BinaryInputArchive ia;
+
+ private BinaryOutputArchive oa;
+
+ private final BufferedInputStream bufferedInput;
+ private BufferedOutputStream bufferedOutput;
+
+ /**
+ * Keep track of whether we have started send packets thread
+ */
+ private volatile boolean sendingThreadStarted = false;
+
+ /**
+ * For testing purpose, force leader to use snapshot to sync with followers
+ */
+ public static final String FORCE_SNAP_SYNC = "zookeeper.forceSnapshotSync";
+ private boolean forceSnapSync = false;
+
+ /**
+ * Keep track of whether we need to queue TRUNC or DIFF into packet queue
+ * that we are going to blast it to the learner
+ */
+ private boolean needOpPacket = true;
+
+ /**
+ * Last zxid sent to the learner as part of synchronization
+ */
+ private long leaderLastZxid;
+
+ LearnerHandler(Socket sock, BufferedInputStream bufferedInput,Leader leader) throws IOException {
+ super("LearnerHandler-" + sock.getRemoteSocketAddress());
+ this.sock = sock;
+ this.leader = leader;
+ this.bufferedInput = bufferedInput;
+
+ if (Boolean.getBoolean(FORCE_SNAP_SYNC)) {
+ forceSnapSync = true;
+ LOG.info("Forcing snapshot sync is enabled");
+ }
+
+ try {
+ if (leader.self != null) {
+ leader.self.authServer.authenticate(sock,
+ new DataInputStream(bufferedInput));
+ }
+ } catch (IOException e) {
+ LOG.error("Server failed to authenticate quorum learner, addr: {}, closing connection",
+ sock.getRemoteSocketAddress(), e);
+ try {
+ sock.close();
+ } catch (IOException ie) {
+ LOG.error("Exception while closing socket", ie);
+ }
+ throw new SaslException("Authentication failure: " + e.getMessage());
+ }
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("LearnerHandler ").append(sock);
+ sb.append(" tickOfNextAckDeadline:").append(tickOfNextAckDeadline());
+ sb.append(" synced?:").append(synced());
+ sb.append(" queuedPacketLength:").append(queuedPackets.size());
+ return sb.toString();
+ }
+
+ /**
+ * If this packet is queued, the sender thread will exit
+ */
+ final QuorumPacket proposalOfDeath = new QuorumPacket();
+
+ private LearnerType learnerType = LearnerType.PARTICIPANT;
+ public LearnerType getLearnerType() {
+ return learnerType;
+ }
+
+ /**
+ * This method will use the thread to send packets added to the
+ * queuedPackets list
+ *
+ * @throws InterruptedException
+ */
+ private void sendPackets() throws InterruptedException {
+ long traceMask = ZooTrace.SERVER_PACKET_TRACE_MASK;
+ while (true) {
+ try {
+ QuorumPacket p;
+ p = queuedPackets.poll();
+ if (p == null) {
+ bufferedOutput.flush();
+ p = queuedPackets.take();
+ }
+
+ if (p == proposalOfDeath) {
+ // Packet of death!
+ break;
+ }
+ if (p.getType() == Leader.PING) {
+ traceMask = ZooTrace.SERVER_PING_TRACE_MASK;
+ }
+ if (p.getType() == Leader.PROPOSAL) {
+ syncLimitCheck.updateProposal(p.getZxid(), System.nanoTime());
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'o', p);
+ }
+ oa.writeRecord(p, "packet");
+ } catch (IOException e) {
+ if (!sock.isClosed()) {
+ LOG.warn("Unexpected exception at " + this, e);
+ try {
+ // this will cause everything to shutdown on
+ // this learner handler and will help notify
+ // the learner/observer instantaneously
+ sock.close();
+ } catch(IOException ie) {
+ LOG.warn("Error closing socket for handler " + this, ie);
+ }
+ }
+ break;
+ }
+ }
+ }
+
+ static public String packetToString(QuorumPacket p) {
+ String type;
+ String mess = null;
+
+ switch (p.getType()) {
+ case Leader.ACK:
+ type = "ACK";
+ break;
+ case Leader.COMMIT:
+ type = "COMMIT";
+ break;
+ case Leader.FOLLOWERINFO:
+ type = "FOLLOWERINFO";
+ break;
+ case Leader.NEWLEADER:
+ type = "NEWLEADER";
+ break;
+ case Leader.PING:
+ type = "PING";
+ break;
+ case Leader.PROPOSAL:
+ type = "PROPOSAL";
+ TxnHeader hdr = new TxnHeader();
+ try {
+ SerializeUtils.deserializeTxn(p.getData(), hdr);
+ // mess = "transaction: " + txn.toString();
+ } catch (IOException e) {
+ LOG.warn("Unexpected exception",e);
+ }
+ break;
+ case Leader.REQUEST:
+ type = "REQUEST";
+ break;
+ case Leader.REVALIDATE:
+ type = "REVALIDATE";
+ ByteArrayInputStream bis = new ByteArrayInputStream(p.getData());
+ DataInputStream dis = new DataInputStream(bis);
+ try {
+ long id = dis.readLong();
+ mess = " sessionid = " + id;
+ } catch (IOException e) {
+ LOG.warn("Unexpected exception", e);
+ }
+
+ break;
+ case Leader.UPTODATE:
+ type = "UPTODATE";
+ break;
+ case Leader.DIFF:
+ type = "DIFF";
+ break;
+ case Leader.TRUNC:
+ type = "TRUNC";
+ break;
+ case Leader.SNAP:
+ type = "SNAP";
+ break;
+ case Leader.ACKEPOCH:
+ type = "ACKEPOCH";
+ break;
+ case Leader.SYNC:
+ type = "SYNC";
+ break;
+ case Leader.INFORM:
+ type = "INFORM";
+ break;
+ case Leader.COMMITANDACTIVATE:
+ type = "COMMITANDACTIVATE";
+ break;
+ case Leader.INFORMANDACTIVATE:
+ type = "INFORMANDACTIVATE";
+ break;
+ default:
+ type = "UNKNOWN" + p.getType();
+ }
+ String entry = null;
+ if (type != null) {
+ entry = type + " " + Long.toHexString(p.getZxid()) + " " + mess;
+ }
+ return entry;
+ }
+
+ /**
+ * This thread will receive packets from the peer and process them and
+ * also listen to new connections from new peers.
+ */
+ @Override
+ public void run() {
+ try {
+ leader.addLearnerHandler(this);
+ tickOfNextAckDeadline = leader.self.tick.get()
+ + leader.self.initLimit + leader.self.syncLimit;
+
+ ia = BinaryInputArchive.getArchive(bufferedInput);
+ bufferedOutput = new BufferedOutputStream(sock.getOutputStream());
+ oa = BinaryOutputArchive.getArchive(bufferedOutput);
+
+ QuorumPacket qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+ if(qp.getType() != Leader.FOLLOWERINFO && qp.getType() != Leader.OBSERVERINFO){
+ LOG.error("First packet " + qp.toString()
+ + " is not FOLLOWERINFO or OBSERVERINFO!");
+ return;
+ }
+
+ byte learnerInfoData[] = qp.getData();
+ if (learnerInfoData != null) {
+ ByteBuffer bbsid = ByteBuffer.wrap(learnerInfoData);
+ if (learnerInfoData.length >= 8) {
+ this.sid = bbsid.getLong();
+ }
+ if (learnerInfoData.length >= 12) {
+ this.version = bbsid.getInt(); // protocolVersion
+ }
+ if (learnerInfoData.length >= 20) {
+ long configVersion = bbsid.getLong();
+ if (configVersion > leader.self.getQuorumVerifier().getVersion()) {
+ throw new IOException("Follower is ahead of the leader (has a later activated configuration)");
+ }
+ }
+ } else {
+ this.sid = leader.followerCounter.getAndDecrement();
+ }
+
+ if (leader.self.getView().containsKey(this.sid)) {
+ LOG.info("Follower sid: " + this.sid + " : info : "
+ + leader.self.getView().get(this.sid).toString());
+ } else {
+ LOG.info("Follower sid: " + this.sid + " not in the current config " + Long.toHexString(leader.self.getQuorumVerifier().getVersion()));
+ }
+
+ if (qp.getType() == Leader.OBSERVERINFO) {
+ learnerType = LearnerType.OBSERVER;
+ }
+
+ long lastAcceptedEpoch = ZxidUtils.getEpochFromZxid(qp.getZxid());
+
+ long peerLastZxid;
+ StateSummary ss = null;
+ long zxid = qp.getZxid();
+ long newEpoch = leader.getEpochToPropose(this.getSid(), lastAcceptedEpoch);
+ long newLeaderZxid = ZxidUtils.makeZxid(newEpoch, 0);
+
+ if (this.getVersion() < 0x10000) {
+ // we are going to have to extrapolate the epoch information
+ long epoch = ZxidUtils.getEpochFromZxid(zxid);
+ ss = new StateSummary(epoch, zxid);
+ // fake the message
+ leader.waitForEpochAck(this.getSid(), ss);
+ } else {
+ byte ver[] = new byte[4];
+ ByteBuffer.wrap(ver).putInt(0x10000);
+ QuorumPacket newEpochPacket = new QuorumPacket(Leader.LEADERINFO, newLeaderZxid, ver, null);
+ oa.writeRecord(newEpochPacket, "packet");
+ bufferedOutput.flush();
+ QuorumPacket ackEpochPacket = new QuorumPacket();
+ ia.readRecord(ackEpochPacket, "packet");
+ if (ackEpochPacket.getType() != Leader.ACKEPOCH) {
+ LOG.error(ackEpochPacket.toString()
+ + " is not ACKEPOCH");
+ return;
+ }
+ ByteBuffer bbepoch = ByteBuffer.wrap(ackEpochPacket.getData());
+ ss = new StateSummary(bbepoch.getInt(), ackEpochPacket.getZxid());
+ leader.waitForEpochAck(this.getSid(), ss);
+ }
+ peerLastZxid = ss.getLastZxid();
+
+ // Take any necessary action if we need to send TRUNC or DIFF
+ // startForwarding() will be called in all cases
+ boolean needSnap = syncFollower(peerLastZxid, leader.zk.getZKDatabase(), leader);
+
+ /* if we are not truncating or sending a diff just send a snapshot */
+ if (needSnap) {
+ boolean exemptFromThrottle = getLearnerType() != LearnerType.OBSERVER;
+ LearnerSnapshot snapshot =
+ leader.getLearnerSnapshotThrottler().beginSnapshot(exemptFromThrottle);
+ try {
+ long zxidToSend = leader.zk.getZKDatabase().getDataTreeLastProcessedZxid();
+ oa.writeRecord(new QuorumPacket(Leader.SNAP, zxidToSend, null, null), "packet");
+ bufferedOutput.flush();
+
+ LOG.info("Sending snapshot last zxid of peer is 0x{}, zxid of leader is 0x{}, "
+ + "send zxid of db as 0x{}, {} concurrent snapshots, "
+ + "snapshot was {} from throttle",
+ Long.toHexString(peerLastZxid),
+ Long.toHexString(leaderLastZxid),
+ Long.toHexString(zxidToSend),
+ snapshot.getConcurrentSnapshotNumber(),
+ snapshot.isEssential() ? "exempt" : "not exempt");
+ // Dump data to peer
+ leader.zk.getZKDatabase().serializeSnapshot(oa);
+ oa.writeString("BenWasHere", "signature");
+ bufferedOutput.flush();
+ } finally {
+ snapshot.close();
+ }
+ }
+
+ LOG.debug("Sending NEWLEADER message to " + sid);
+ // the version of this quorumVerifier will be set by leader.lead() in case
+ // the leader is just being established. waitForEpochAck makes sure that readyToStart is true if
+ // we got here, so the version was set
+ if (getVersion() < 0x10000) {
+ QuorumPacket newLeaderQP = new QuorumPacket(Leader.NEWLEADER,
+ newLeaderZxid, null, null);
+ oa.writeRecord(newLeaderQP, "packet");
+ } else {
+ QuorumPacket newLeaderQP = new QuorumPacket(Leader.NEWLEADER,
+ newLeaderZxid, leader.self.getLastSeenQuorumVerifier()
+ .toString().getBytes(), null);
+ queuedPackets.add(newLeaderQP);
+ }
+ bufferedOutput.flush();
+
+ // Start thread that blast packets in the queue to learner
+ startSendingPackets();
+
+ /*
+ * Have to wait for the first ACK, wait until
+ * the leader is ready, and only then we can
+ * start processing messages.
+ */
+ qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+ if(qp.getType() != Leader.ACK){
+ LOG.error("Next packet was supposed to be an ACK,"
+ + " but received packet: {}", packetToString(qp));
+ return;
+ }
+
+ if(LOG.isDebugEnabled()){
+ LOG.debug("Received NEWLEADER-ACK message from " + sid);
+ }
+ leader.waitForNewLeaderAck(getSid(), qp.getZxid());
+
+ syncLimitCheck.start();
+
+ // now that the ack has been processed expect the syncLimit
+ sock.setSoTimeout(leader.self.tickTime * leader.self.syncLimit);
+
+ /*
+ * Wait until leader starts up
+ */
+ synchronized(leader.zk){
+ while(!leader.zk.isRunning() && !this.isInterrupted()){
+ leader.zk.wait(20);
+ }
+ }
+ // Mutation packets will be queued during the serialize,
+ // so we need to mark when the peer can actually start
+ // using the data
+ //
+ LOG.debug("Sending UPTODATE message to " + sid);
+ queuedPackets.add(new QuorumPacket(Leader.UPTODATE, -1, null, null));
+
+ while (true) {
+ qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+
+ long traceMask = ZooTrace.SERVER_PACKET_TRACE_MASK;
+ if (qp.getType() == Leader.PING) {
+ traceMask = ZooTrace.SERVER_PING_TRACE_MASK;
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'i', qp);
+ }
+ tickOfNextAckDeadline = leader.self.tick.get() + leader.self.syncLimit;
+
+
+ ByteBuffer bb;
+ long sessionId;
+ int cxid;
+ int type;
+
+ switch (qp.getType()) {
+ case Leader.ACK:
+ if (this.learnerType == LearnerType.OBSERVER) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Received ACK from Observer " + this.sid);
+ }
+ }
+ syncLimitCheck.updateAck(qp.getZxid());
+ leader.processAck(this.sid, qp.getZxid(), sock.getLocalSocketAddress());
+ break;
+ case Leader.PING:
+ // Process the touches
+ ByteArrayInputStream bis = new ByteArrayInputStream(qp
+ .getData());
+ DataInputStream dis = new DataInputStream(bis);
+ while (dis.available() > 0) {
+ long sess = dis.readLong();
+ int to = dis.readInt();
+ leader.zk.touch(sess, to);
+ }
+ break;
+ case Leader.REVALIDATE:
+ bis = new ByteArrayInputStream(qp.getData());
+ dis = new DataInputStream(bis);
+ long id = dis.readLong();
+ int to = dis.readInt();
+ ByteArrayOutputStream bos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(bos);
+ dos.writeLong(id);
+ boolean valid = leader.zk.checkIfValidGlobalSession(id, to);
+ if (valid) {
+ try {
+ //set the session owner
+ // as the follower that
+ // owns the session
+ leader.zk.setOwner(id, this);
+ } catch (SessionExpiredException e) {
+ LOG.error("Somehow session " + Long.toHexString(id) +
+ " expired right after being renewed! (impossible)", e);
+ }
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG,
+ ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(id)
+ + " is valid: "+ valid);
+ }
+ dos.writeBoolean(valid);
+ qp.setData(bos.toByteArray());
+ queuedPackets.add(qp);
+ break;
+ case Leader.REQUEST:
+ bb = ByteBuffer.wrap(qp.getData());
+ sessionId = bb.getLong();
+ cxid = bb.getInt();
+ type = bb.getInt();
+ bb = bb.slice();
+ Request si;
+ if(type == OpCode.sync){
+ si = new LearnerSyncRequest(this, sessionId, cxid, type, bb, qp.getAuthinfo());
+ } else {
+ si = new Request(null, sessionId, cxid, type, bb, qp.getAuthinfo());
+ }
+ si.setOwner(this);
+ leader.zk.submitLearnerRequest(si);
+ break;
+ default:
+ LOG.warn("unexpected quorum packet, type: {}", packetToString(qp));
+ break;
+ }
+ }
+ } catch (IOException e) {
+ if (sock != null && !sock.isClosed()) {
+ LOG.error("Unexpected exception causing shutdown while sock "
+ + "still open", e);
+ //close the socket to make sure the
+ //other side can see it being close
+ try {
+ sock.close();
+ } catch(IOException ie) {
+ // do nothing
+ }
+ }
+ } catch (InterruptedException e) {
+ LOG.error("Unexpected exception causing shutdown", e);
+ } catch (SnapshotThrottleException e) {
+ LOG.error("too many concurrent snapshots: " + e);
+ } finally {
+ LOG.warn("******* GOODBYE "
+ + (sock != null ? sock.getRemoteSocketAddress() : "<null>")
+ + " ********");
+ shutdown();
+ }
+ }
+
+ /**
+ * Start thread that will forward any packet in the queue to the follower
+ */
+ protected void startSendingPackets() {
+ if (!sendingThreadStarted) {
+ // Start sending packets
+ new Thread() {
+ public void run() {
+ Thread.currentThread().setName(
+ "Sender-" + sock.getRemoteSocketAddress());
+ try {
+ sendPackets();
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption " + e.getMessage());
+ }
+ }
+ }.start();
+ sendingThreadStarted = true;
+ } else {
+ LOG.error("Attempting to start sending thread after it already started");
+ }
+ }
+
+ /**
+ * Determine if we need to sync with follower using DIFF/TRUNC/SNAP
+ * and setup follower to receive packets from commit processor
+ *
+ * @param peerLastZxid
+ * @param db
+ * @param leader
+ * @return true if snapshot transfer is needed.
+ */
+ public boolean syncFollower(long peerLastZxid, ZKDatabase db, Leader leader) {
+ /*
+ * When leader election is completed, the leader will set its
+ * lastProcessedZxid to be (epoch < 32). There will be no txn associated
+ * with this zxid.
+ *
+ * The learner will set its lastProcessedZxid to the same value if
+ * it get DIFF or SNAP from the leader. If the same learner come
+ * back to sync with leader using this zxid, we will never find this
+ * zxid in our history. In this case, we will ignore TRUNC logic and
+ * always send DIFF if we have old enough history
+ */
+ boolean isPeerNewEpochZxid = (peerLastZxid & 0xffffffffL) == 0;
+ // Keep track of the latest zxid which already queued
+ long currentZxid = peerLastZxid;
+ boolean needSnap = true;
+ boolean txnLogSyncEnabled = db.isTxnLogSyncEnabled();
+ ReentrantReadWriteLock lock = db.getLogLock();
+ ReadLock rl = lock.readLock();
+ try {
+ rl.lock();
+ long maxCommittedLog = db.getmaxCommittedLog();
+ long minCommittedLog = db.getminCommittedLog();
+ long lastProcessedZxid = db.getDataTreeLastProcessedZxid();
+
+ LOG.info("Synchronizing with Follower sid: {} maxCommittedLog=0x{}"
+ + " minCommittedLog=0x{} lastProcessedZxid=0x{}"
+ + " peerLastZxid=0x{}", getSid(),
+ Long.toHexString(maxCommittedLog),
+ Long.toHexString(minCommittedLog),
+ Long.toHexString(lastProcessedZxid),
+ Long.toHexString(peerLastZxid));
+
+ if (db.getCommittedLog().isEmpty()) {
+ /*
+ * It is possible that committedLog is empty. In that case
+ * setting these value to the latest txn in leader db
+ * will reduce the case that we need to handle
+ *
+ * Here is how each case handle by the if block below
+ * 1. lastProcessZxid == peerZxid -> Handle by (2)
+ * 2. lastProcessZxid < peerZxid -> Handle by (3)
+ * 3. lastProcessZxid > peerZxid -> Handle by (5)
+ */
+ minCommittedLog = lastProcessedZxid;
+ maxCommittedLog = lastProcessedZxid;
+ }
+
+ /*
+ * Here are the cases that we want to handle
+ *
+ * 1. Force sending snapshot (for testing purpose)
+ * 2. Peer and leader is already sync, send empty diff
+ * 3. Follower has txn that we haven't seen. This may be old leader
+ * so we need to send TRUNC. However, if peer has newEpochZxid,
+ * we cannot send TRUNC since the follower has no txnlog
+ * 4. Follower is within committedLog range or already in-sync.
+ * We may need to send DIFF or TRUNC depending on follower's zxid
+ * We always send empty DIFF if follower is already in-sync
+ * 5. Follower missed the committedLog. We will try to use on-disk
+ * txnlog + committedLog to sync with follower. If that fail,
+ * we will send snapshot
+ */
+
+ if (forceSnapSync) {
+ // Force leader to use snapshot to sync with follower
+ LOG.warn("Forcing snapshot sync - should not see this in production");
+ } else if (lastProcessedZxid == peerLastZxid) {
+ // Follower is already sync with us, send empty diff
+ LOG.info("Sending DIFF zxid=0x" + Long.toHexString(peerLastZxid) +
+ " for peer sid: " + getSid());
+ queueOpPacket(Leader.DIFF, peerLastZxid);
+ needOpPacket = false;
+ needSnap = false;
+ } else if (peerLastZxid > maxCommittedLog && !isPeerNewEpochZxid) {
+ // Newer than committedLog, send trunc and done
+ LOG.debug("Sending TRUNC to follower zxidToSend=0x" +
+ Long.toHexString(maxCommittedLog) +
+ " for peer sid:" + getSid());
+ queueOpPacket(Leader.TRUNC, maxCommittedLog);
+ currentZxid = maxCommittedLog;
+ needOpPacket = false;
+ needSnap = false;
+ } else if ((maxCommittedLog >= peerLastZxid)
+ && (minCommittedLog <= peerLastZxid)) {
+ // Follower is within commitLog range
+ LOG.info("Using committedLog for peer sid: " + getSid());
+ Iterator<Proposal> itr = db.getCommittedLog().iterator();
+ currentZxid = queueCommittedProposals(itr, peerLastZxid,
+ null, maxCommittedLog);
+ needSnap = false;
+ } else if (peerLastZxid < minCommittedLog && txnLogSyncEnabled) {
+ // Use txnlog and committedLog to sync
+
+ // Calculate sizeLimit that we allow to retrieve txnlog from disk
+ long sizeLimit = db.calculateTxnLogSizeLimit();
+ // This method can return empty iterator if the requested zxid
+ // is older than on-disk txnlog
+ Iterator<Proposal> txnLogItr = db.getProposalsFromTxnLog(
+ peerLastZxid, sizeLimit);
+ if (txnLogItr.hasNext()) {
+ LOG.info("Use txnlog and committedLog for peer sid: " + getSid());
+ currentZxid = queueCommittedProposals(txnLogItr, peerLastZxid,
+ minCommittedLog, maxCommittedLog);
+
+ LOG.debug("Queueing committedLog 0x" + Long.toHexString(currentZxid));
+ Iterator<Proposal> committedLogItr = db.getCommittedLog().iterator();
+ currentZxid = queueCommittedProposals(committedLogItr, currentZxid,
+ null, maxCommittedLog);
+ needSnap = false;
+ }
+ // closing the resources
+ if (txnLogItr instanceof TxnLogProposalIterator) {
+ TxnLogProposalIterator txnProposalItr = (TxnLogProposalIterator) txnLogItr;
+ txnProposalItr.close();
+ }
+ } else {
+ LOG.warn("Unhandled scenario for peer sid: " + getSid());
+ }
+ LOG.debug("Start forwarding 0x" + Long.toHexString(currentZxid) +
+ " for peer sid: " + getSid());
+ leaderLastZxid = leader.startForwarding(this, currentZxid);
+ } finally {
+ rl.unlock();
+ }
+
+ if (needOpPacket && !needSnap) {
+ // This should never happen, but we should fall back to sending
+ // snapshot just in case.
+ LOG.error("Unhandled scenario for peer sid: " + getSid() +
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `LearnerHandler.syncFollower(...)` indirectly reads without synchronization from `this.sid`. Potentially races with write in method `LearnerHandler.run()`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LearnerHandler.java
##########
@@ -0,0 +1,1000 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.Socket;
+import java.nio.ByteBuffer;
+import java.util.Iterator;
+import java.util.Queue;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.server.Request;
+import org.apache.zookeeper.server.TxnLogProposalIterator;
+import org.apache.zookeeper.server.ZKDatabase;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.ZooTrace;
+import org.apache.zookeeper.server.quorum.Leader.Proposal;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * There will be an instance of this class created by the Leader for each
+ * learner. All communication with a learner is handled by this
+ * class.
+ */
+public class LearnerHandler extends ZooKeeperThread {
+ private static final Logger LOG = LoggerFactory.getLogger(LearnerHandler.class);
+
+ protected final Socket sock;
+
+ public Socket getSocket() {
+ return sock;
+ }
+
+ final Leader leader;
+
+ /** Deadline for receiving the next ack. If we are bootstrapping then
+ * it's based on the initLimit, if we are done bootstrapping it's based
+ * on the syncLimit. Once the deadline is past this learner should
+ * be considered no longer "sync'd" with the leader. */
+ volatile long tickOfNextAckDeadline;
+
+ /**
+ * ZooKeeper server identifier of this learner
+ */
+ protected long sid = 0;
+
+ long getSid(){
+ return sid;
+ }
+
+ protected int version = 0x1;
+
+ int getVersion() {
+ return version;
+ }
+
+ /**
+ * The packets to be sent to the learner
+ */
+ final LinkedBlockingQueue<QuorumPacket> queuedPackets =
+ new LinkedBlockingQueue<QuorumPacket>();
+
+ /**
+ * This class controls the time that the Leader has been
+ * waiting for acknowledgement of a proposal from this Learner.
+ * If the time is above syncLimit, the connection will be closed.
+ * It keeps track of only one proposal at a time, when the ACK for
+ * that proposal arrives, it switches to the last proposal received
+ * or clears the value if there is no pending proposal.
+ */
+ private class SyncLimitCheck {
+ private boolean started = false;
+ private long currentZxid = 0;
+ private long currentTime = 0;
+ private long nextZxid = 0;
+ private long nextTime = 0;
+
+ public synchronized void start() {
+ started = true;
+ }
+
+ public synchronized void updateProposal(long zxid, long time) {
+ if (!started) {
+ return;
+ }
+ if (currentTime == 0) {
+ currentTime = time;
+ currentZxid = zxid;
+ } else {
+ nextTime = time;
+ nextZxid = zxid;
+ }
+ }
+
+ public synchronized void updateAck(long zxid) {
+ if (currentZxid == zxid) {
+ currentTime = nextTime;
+ currentZxid = nextZxid;
+ nextTime = 0;
+ nextZxid = 0;
+ } else if (nextZxid == zxid) {
+ LOG.warn("ACK for " + zxid + " received before ACK for " + currentZxid + "!!!!");
+ nextTime = 0;
+ nextZxid = 0;
+ }
+ }
+
+ public synchronized boolean check(long time) {
+ if (currentTime == 0) {
+ return true;
+ } else {
+ long msDelay = (time - currentTime) / 1000000;
+ return (msDelay < (leader.self.tickTime * leader.self.syncLimit));
+ }
+ }
+ };
+
+ private SyncLimitCheck syncLimitCheck = new SyncLimitCheck();
+
+ private BinaryInputArchive ia;
+
+ private BinaryOutputArchive oa;
+
+ private final BufferedInputStream bufferedInput;
+ private BufferedOutputStream bufferedOutput;
+
+ /**
+ * Keep track of whether we have started send packets thread
+ */
+ private volatile boolean sendingThreadStarted = false;
+
+ /**
+ * For testing purpose, force leader to use snapshot to sync with followers
+ */
+ public static final String FORCE_SNAP_SYNC = "zookeeper.forceSnapshotSync";
+ private boolean forceSnapSync = false;
+
+ /**
+ * Keep track of whether we need to queue TRUNC or DIFF into packet queue
+ * that we are going to blast it to the learner
+ */
+ private boolean needOpPacket = true;
+
+ /**
+ * Last zxid sent to the learner as part of synchronization
+ */
+ private long leaderLastZxid;
+
+ LearnerHandler(Socket sock, BufferedInputStream bufferedInput,Leader leader) throws IOException {
+ super("LearnerHandler-" + sock.getRemoteSocketAddress());
+ this.sock = sock;
+ this.leader = leader;
+ this.bufferedInput = bufferedInput;
+
+ if (Boolean.getBoolean(FORCE_SNAP_SYNC)) {
+ forceSnapSync = true;
+ LOG.info("Forcing snapshot sync is enabled");
+ }
+
+ try {
+ if (leader.self != null) {
+ leader.self.authServer.authenticate(sock,
+ new DataInputStream(bufferedInput));
+ }
+ } catch (IOException e) {
+ LOG.error("Server failed to authenticate quorum learner, addr: {}, closing connection",
+ sock.getRemoteSocketAddress(), e);
+ try {
+ sock.close();
+ } catch (IOException ie) {
+ LOG.error("Exception while closing socket", ie);
+ }
+ throw new SaslException("Authentication failure: " + e.getMessage());
+ }
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append("LearnerHandler ").append(sock);
+ sb.append(" tickOfNextAckDeadline:").append(tickOfNextAckDeadline());
+ sb.append(" synced?:").append(synced());
+ sb.append(" queuedPacketLength:").append(queuedPackets.size());
+ return sb.toString();
+ }
+
+ /**
+ * If this packet is queued, the sender thread will exit
+ */
+ final QuorumPacket proposalOfDeath = new QuorumPacket();
+
+ private LearnerType learnerType = LearnerType.PARTICIPANT;
+ public LearnerType getLearnerType() {
+ return learnerType;
+ }
+
+ /**
+ * This method will use the thread to send packets added to the
+ * queuedPackets list
+ *
+ * @throws InterruptedException
+ */
+ private void sendPackets() throws InterruptedException {
+ long traceMask = ZooTrace.SERVER_PACKET_TRACE_MASK;
+ while (true) {
+ try {
+ QuorumPacket p;
+ p = queuedPackets.poll();
+ if (p == null) {
+ bufferedOutput.flush();
+ p = queuedPackets.take();
+ }
+
+ if (p == proposalOfDeath) {
+ // Packet of death!
+ break;
+ }
+ if (p.getType() == Leader.PING) {
+ traceMask = ZooTrace.SERVER_PING_TRACE_MASK;
+ }
+ if (p.getType() == Leader.PROPOSAL) {
+ syncLimitCheck.updateProposal(p.getZxid(), System.nanoTime());
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'o', p);
+ }
+ oa.writeRecord(p, "packet");
+ } catch (IOException e) {
+ if (!sock.isClosed()) {
+ LOG.warn("Unexpected exception at " + this, e);
+ try {
+ // this will cause everything to shutdown on
+ // this learner handler and will help notify
+ // the learner/observer instantaneously
+ sock.close();
+ } catch(IOException ie) {
+ LOG.warn("Error closing socket for handler " + this, ie);
+ }
+ }
+ break;
+ }
+ }
+ }
+
+ static public String packetToString(QuorumPacket p) {
+ String type;
+ String mess = null;
+
+ switch (p.getType()) {
+ case Leader.ACK:
+ type = "ACK";
+ break;
+ case Leader.COMMIT:
+ type = "COMMIT";
+ break;
+ case Leader.FOLLOWERINFO:
+ type = "FOLLOWERINFO";
+ break;
+ case Leader.NEWLEADER:
+ type = "NEWLEADER";
+ break;
+ case Leader.PING:
+ type = "PING";
+ break;
+ case Leader.PROPOSAL:
+ type = "PROPOSAL";
+ TxnHeader hdr = new TxnHeader();
+ try {
+ SerializeUtils.deserializeTxn(p.getData(), hdr);
+ // mess = "transaction: " + txn.toString();
+ } catch (IOException e) {
+ LOG.warn("Unexpected exception",e);
+ }
+ break;
+ case Leader.REQUEST:
+ type = "REQUEST";
+ break;
+ case Leader.REVALIDATE:
+ type = "REVALIDATE";
+ ByteArrayInputStream bis = new ByteArrayInputStream(p.getData());
+ DataInputStream dis = new DataInputStream(bis);
+ try {
+ long id = dis.readLong();
+ mess = " sessionid = " + id;
+ } catch (IOException e) {
+ LOG.warn("Unexpected exception", e);
+ }
+
+ break;
+ case Leader.UPTODATE:
+ type = "UPTODATE";
+ break;
+ case Leader.DIFF:
+ type = "DIFF";
+ break;
+ case Leader.TRUNC:
+ type = "TRUNC";
+ break;
+ case Leader.SNAP:
+ type = "SNAP";
+ break;
+ case Leader.ACKEPOCH:
+ type = "ACKEPOCH";
+ break;
+ case Leader.SYNC:
+ type = "SYNC";
+ break;
+ case Leader.INFORM:
+ type = "INFORM";
+ break;
+ case Leader.COMMITANDACTIVATE:
+ type = "COMMITANDACTIVATE";
+ break;
+ case Leader.INFORMANDACTIVATE:
+ type = "INFORMANDACTIVATE";
+ break;
+ default:
+ type = "UNKNOWN" + p.getType();
+ }
+ String entry = null;
+ if (type != null) {
+ entry = type + " " + Long.toHexString(p.getZxid()) + " " + mess;
+ }
+ return entry;
+ }
+
+ /**
+ * This thread will receive packets from the peer and process them and
+ * also listen to new connections from new peers.
+ */
+ @Override
+ public void run() {
+ try {
+ leader.addLearnerHandler(this);
+ tickOfNextAckDeadline = leader.self.tick.get()
+ + leader.self.initLimit + leader.self.syncLimit;
+
+ ia = BinaryInputArchive.getArchive(bufferedInput);
+ bufferedOutput = new BufferedOutputStream(sock.getOutputStream());
+ oa = BinaryOutputArchive.getArchive(bufferedOutput);
+
+ QuorumPacket qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+ if(qp.getType() != Leader.FOLLOWERINFO && qp.getType() != Leader.OBSERVERINFO){
+ LOG.error("First packet " + qp.toString()
+ + " is not FOLLOWERINFO or OBSERVERINFO!");
+ return;
+ }
+
+ byte learnerInfoData[] = qp.getData();
+ if (learnerInfoData != null) {
+ ByteBuffer bbsid = ByteBuffer.wrap(learnerInfoData);
+ if (learnerInfoData.length >= 8) {
+ this.sid = bbsid.getLong();
+ }
+ if (learnerInfoData.length >= 12) {
+ this.version = bbsid.getInt(); // protocolVersion
+ }
+ if (learnerInfoData.length >= 20) {
+ long configVersion = bbsid.getLong();
+ if (configVersion > leader.self.getQuorumVerifier().getVersion()) {
+ throw new IOException("Follower is ahead of the leader (has a later activated configuration)");
+ }
+ }
+ } else {
+ this.sid = leader.followerCounter.getAndDecrement();
+ }
+
+ if (leader.self.getView().containsKey(this.sid)) {
+ LOG.info("Follower sid: " + this.sid + " : info : "
+ + leader.self.getView().get(this.sid).toString());
+ } else {
+ LOG.info("Follower sid: " + this.sid + " not in the current config " + Long.toHexString(leader.self.getQuorumVerifier().getVersion()));
+ }
+
+ if (qp.getType() == Leader.OBSERVERINFO) {
+ learnerType = LearnerType.OBSERVER;
+ }
+
+ long lastAcceptedEpoch = ZxidUtils.getEpochFromZxid(qp.getZxid());
+
+ long peerLastZxid;
+ StateSummary ss = null;
+ long zxid = qp.getZxid();
+ long newEpoch = leader.getEpochToPropose(this.getSid(), lastAcceptedEpoch);
+ long newLeaderZxid = ZxidUtils.makeZxid(newEpoch, 0);
+
+ if (this.getVersion() < 0x10000) {
+ // we are going to have to extrapolate the epoch information
+ long epoch = ZxidUtils.getEpochFromZxid(zxid);
+ ss = new StateSummary(epoch, zxid);
+ // fake the message
+ leader.waitForEpochAck(this.getSid(), ss);
+ } else {
+ byte ver[] = new byte[4];
+ ByteBuffer.wrap(ver).putInt(0x10000);
+ QuorumPacket newEpochPacket = new QuorumPacket(Leader.LEADERINFO, newLeaderZxid, ver, null);
+ oa.writeRecord(newEpochPacket, "packet");
+ bufferedOutput.flush();
+ QuorumPacket ackEpochPacket = new QuorumPacket();
+ ia.readRecord(ackEpochPacket, "packet");
+ if (ackEpochPacket.getType() != Leader.ACKEPOCH) {
+ LOG.error(ackEpochPacket.toString()
+ + " is not ACKEPOCH");
+ return;
+ }
+ ByteBuffer bbepoch = ByteBuffer.wrap(ackEpochPacket.getData());
+ ss = new StateSummary(bbepoch.getInt(), ackEpochPacket.getZxid());
+ leader.waitForEpochAck(this.getSid(), ss);
+ }
+ peerLastZxid = ss.getLastZxid();
+
+ // Take any necessary action if we need to send TRUNC or DIFF
+ // startForwarding() will be called in all cases
+ boolean needSnap = syncFollower(peerLastZxid, leader.zk.getZKDatabase(), leader);
+
+ /* if we are not truncating or sending a diff just send a snapshot */
+ if (needSnap) {
+ boolean exemptFromThrottle = getLearnerType() != LearnerType.OBSERVER;
+ LearnerSnapshot snapshot =
+ leader.getLearnerSnapshotThrottler().beginSnapshot(exemptFromThrottle);
+ try {
+ long zxidToSend = leader.zk.getZKDatabase().getDataTreeLastProcessedZxid();
+ oa.writeRecord(new QuorumPacket(Leader.SNAP, zxidToSend, null, null), "packet");
+ bufferedOutput.flush();
+
+ LOG.info("Sending snapshot last zxid of peer is 0x{}, zxid of leader is 0x{}, "
+ + "send zxid of db as 0x{}, {} concurrent snapshots, "
+ + "snapshot was {} from throttle",
+ Long.toHexString(peerLastZxid),
+ Long.toHexString(leaderLastZxid),
+ Long.toHexString(zxidToSend),
+ snapshot.getConcurrentSnapshotNumber(),
+ snapshot.isEssential() ? "exempt" : "not exempt");
+ // Dump data to peer
+ leader.zk.getZKDatabase().serializeSnapshot(oa);
+ oa.writeString("BenWasHere", "signature");
+ bufferedOutput.flush();
+ } finally {
+ snapshot.close();
+ }
+ }
+
+ LOG.debug("Sending NEWLEADER message to " + sid);
+ // the version of this quorumVerifier will be set by leader.lead() in case
+ // the leader is just being established. waitForEpochAck makes sure that readyToStart is true if
+ // we got here, so the version was set
+ if (getVersion() < 0x10000) {
+ QuorumPacket newLeaderQP = new QuorumPacket(Leader.NEWLEADER,
+ newLeaderZxid, null, null);
+ oa.writeRecord(newLeaderQP, "packet");
+ } else {
+ QuorumPacket newLeaderQP = new QuorumPacket(Leader.NEWLEADER,
+ newLeaderZxid, leader.self.getLastSeenQuorumVerifier()
+ .toString().getBytes(), null);
+ queuedPackets.add(newLeaderQP);
+ }
+ bufferedOutput.flush();
+
+ // Start thread that blast packets in the queue to learner
+ startSendingPackets();
+
+ /*
+ * Have to wait for the first ACK, wait until
+ * the leader is ready, and only then we can
+ * start processing messages.
+ */
+ qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+ if(qp.getType() != Leader.ACK){
+ LOG.error("Next packet was supposed to be an ACK,"
+ + " but received packet: {}", packetToString(qp));
+ return;
+ }
+
+ if(LOG.isDebugEnabled()){
+ LOG.debug("Received NEWLEADER-ACK message from " + sid);
+ }
+ leader.waitForNewLeaderAck(getSid(), qp.getZxid());
+
+ syncLimitCheck.start();
+
+ // now that the ack has been processed expect the syncLimit
+ sock.setSoTimeout(leader.self.tickTime * leader.self.syncLimit);
+
+ /*
+ * Wait until leader starts up
+ */
+ synchronized(leader.zk){
+ while(!leader.zk.isRunning() && !this.isInterrupted()){
+ leader.zk.wait(20);
+ }
+ }
+ // Mutation packets will be queued during the serialize,
+ // so we need to mark when the peer can actually start
+ // using the data
+ //
+ LOG.debug("Sending UPTODATE message to " + sid);
+ queuedPackets.add(new QuorumPacket(Leader.UPTODATE, -1, null, null));
+
+ while (true) {
+ qp = new QuorumPacket();
+ ia.readRecord(qp, "packet");
+
+ long traceMask = ZooTrace.SERVER_PACKET_TRACE_MASK;
+ if (qp.getType() == Leader.PING) {
+ traceMask = ZooTrace.SERVER_PING_TRACE_MASK;
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logQuorumPacket(LOG, traceMask, 'i', qp);
+ }
+ tickOfNextAckDeadline = leader.self.tick.get() + leader.self.syncLimit;
+
+
+ ByteBuffer bb;
+ long sessionId;
+ int cxid;
+ int type;
+
+ switch (qp.getType()) {
+ case Leader.ACK:
+ if (this.learnerType == LearnerType.OBSERVER) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Received ACK from Observer " + this.sid);
+ }
+ }
+ syncLimitCheck.updateAck(qp.getZxid());
+ leader.processAck(this.sid, qp.getZxid(), sock.getLocalSocketAddress());
+ break;
+ case Leader.PING:
+ // Process the touches
+ ByteArrayInputStream bis = new ByteArrayInputStream(qp
+ .getData());
+ DataInputStream dis = new DataInputStream(bis);
+ while (dis.available() > 0) {
+ long sess = dis.readLong();
+ int to = dis.readInt();
+ leader.zk.touch(sess, to);
+ }
+ break;
+ case Leader.REVALIDATE:
+ bis = new ByteArrayInputStream(qp.getData());
+ dis = new DataInputStream(bis);
+ long id = dis.readLong();
+ int to = dis.readInt();
+ ByteArrayOutputStream bos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(bos);
+ dos.writeLong(id);
+ boolean valid = leader.zk.checkIfValidGlobalSession(id, to);
+ if (valid) {
+ try {
+ //set the session owner
+ // as the follower that
+ // owns the session
+ leader.zk.setOwner(id, this);
+ } catch (SessionExpiredException e) {
+ LOG.error("Somehow session " + Long.toHexString(id) +
+ " expired right after being renewed! (impossible)", e);
+ }
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG,
+ ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(id)
+ + " is valid: "+ valid);
+ }
+ dos.writeBoolean(valid);
+ qp.setData(bos.toByteArray());
+ queuedPackets.add(qp);
+ break;
+ case Leader.REQUEST:
+ bb = ByteBuffer.wrap(qp.getData());
+ sessionId = bb.getLong();
+ cxid = bb.getInt();
+ type = bb.getInt();
+ bb = bb.slice();
+ Request si;
+ if(type == OpCode.sync){
+ si = new LearnerSyncRequest(this, sessionId, cxid, type, bb, qp.getAuthinfo());
+ } else {
+ si = new Request(null, sessionId, cxid, type, bb, qp.getAuthinfo());
+ }
+ si.setOwner(this);
+ leader.zk.submitLearnerRequest(si);
+ break;
+ default:
+ LOG.warn("unexpected quorum packet, type: {}", packetToString(qp));
+ break;
+ }
+ }
+ } catch (IOException e) {
+ if (sock != null && !sock.isClosed()) {
+ LOG.error("Unexpected exception causing shutdown while sock "
+ + "still open", e);
+ //close the socket to make sure the
+ //other side can see it being close
+ try {
+ sock.close();
+ } catch(IOException ie) {
+ // do nothing
+ }
+ }
+ } catch (InterruptedException e) {
+ LOG.error("Unexpected exception causing shutdown", e);
+ } catch (SnapshotThrottleException e) {
+ LOG.error("too many concurrent snapshots: " + e);
+ } finally {
+ LOG.warn("******* GOODBYE "
+ + (sock != null ? sock.getRemoteSocketAddress() : "<null>")
+ + " ********");
+ shutdown();
+ }
+ }
+
+ /**
+ * Start thread that will forward any packet in the queue to the follower
+ */
+ protected void startSendingPackets() {
+ if (!sendingThreadStarted) {
+ // Start sending packets
+ new Thread() {
+ public void run() {
+ Thread.currentThread().setName(
+ "Sender-" + sock.getRemoteSocketAddress());
+ try {
+ sendPackets();
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption " + e.getMessage());
+ }
+ }
+ }.start();
+ sendingThreadStarted = true;
+ } else {
+ LOG.error("Attempting to start sending thread after it already started");
+ }
+ }
+
+ /**
+ * Determine if we need to sync with follower using DIFF/TRUNC/SNAP
+ * and setup follower to receive packets from commit processor
+ *
+ * @param peerLastZxid
+ * @param db
+ * @param leader
+ * @return true if snapshot transfer is needed.
+ */
+ public boolean syncFollower(long peerLastZxid, ZKDatabase db, Leader leader) {
+ /*
+ * When leader election is completed, the leader will set its
+ * lastProcessedZxid to be (epoch < 32). There will be no txn associated
+ * with this zxid.
+ *
+ * The learner will set its lastProcessedZxid to the same value if
+ * it get DIFF or SNAP from the leader. If the same learner come
+ * back to sync with leader using this zxid, we will never find this
+ * zxid in our history. In this case, we will ignore TRUNC logic and
+ * always send DIFF if we have old enough history
+ */
+ boolean isPeerNewEpochZxid = (peerLastZxid & 0xffffffffL) == 0;
+ // Keep track of the latest zxid which already queued
+ long currentZxid = peerLastZxid;
+ boolean needSnap = true;
+ boolean txnLogSyncEnabled = db.isTxnLogSyncEnabled();
+ ReentrantReadWriteLock lock = db.getLogLock();
+ ReadLock rl = lock.readLock();
+ try {
+ rl.lock();
+ long maxCommittedLog = db.getmaxCommittedLog();
+ long minCommittedLog = db.getminCommittedLog();
+ long lastProcessedZxid = db.getDataTreeLastProcessedZxid();
+
+ LOG.info("Synchronizing with Follower sid: {} maxCommittedLog=0x{}"
+ + " minCommittedLog=0x{} lastProcessedZxid=0x{}"
+ + " peerLastZxid=0x{}", getSid(),
+ Long.toHexString(maxCommittedLog),
+ Long.toHexString(minCommittedLog),
+ Long.toHexString(lastProcessedZxid),
+ Long.toHexString(peerLastZxid));
+
+ if (db.getCommittedLog().isEmpty()) {
+ /*
+ * It is possible that committedLog is empty. In that case
+ * setting these value to the latest txn in leader db
+ * will reduce the case that we need to handle
+ *
+ * Here is how each case handle by the if block below
+ * 1. lastProcessZxid == peerZxid -> Handle by (2)
+ * 2. lastProcessZxid < peerZxid -> Handle by (3)
+ * 3. lastProcessZxid > peerZxid -> Handle by (5)
+ */
+ minCommittedLog = lastProcessedZxid;
+ maxCommittedLog = lastProcessedZxid;
+ }
+
+ /*
+ * Here are the cases that we want to handle
+ *
+ * 1. Force sending snapshot (for testing purpose)
+ * 2. Peer and leader is already sync, send empty diff
+ * 3. Follower has txn that we haven't seen. This may be old leader
+ * so we need to send TRUNC. However, if peer has newEpochZxid,
+ * we cannot send TRUNC since the follower has no txnlog
+ * 4. Follower is within committedLog range or already in-sync.
+ * We may need to send DIFF or TRUNC depending on follower's zxid
+ * We always send empty DIFF if follower is already in-sync
+ * 5. Follower missed the committedLog. We will try to use on-disk
+ * txnlog + committedLog to sync with follower. If that fail,
+ * we will send snapshot
+ */
+
+ if (forceSnapSync) {
+ // Force leader to use snapshot to sync with follower
+ LOG.warn("Forcing snapshot sync - should not see this in production");
+ } else if (lastProcessedZxid == peerLastZxid) {
+ // Follower is already sync with us, send empty diff
+ LOG.info("Sending DIFF zxid=0x" + Long.toHexString(peerLastZxid) +
+ " for peer sid: " + getSid());
+ queueOpPacket(Leader.DIFF, peerLastZxid);
+ needOpPacket = false;
+ needSnap = false;
+ } else if (peerLastZxid > maxCommittedLog && !isPeerNewEpochZxid) {
+ // Newer than committedLog, send trunc and done
+ LOG.debug("Sending TRUNC to follower zxidToSend=0x" +
+ Long.toHexString(maxCommittedLog) +
+ " for peer sid:" + getSid());
+ queueOpPacket(Leader.TRUNC, maxCommittedLog);
+ currentZxid = maxCommittedLog;
+ needOpPacket = false;
+ needSnap = false;
+ } else if ((maxCommittedLog >= peerLastZxid)
+ && (minCommittedLog <= peerLastZxid)) {
+ // Follower is within commitLog range
+ LOG.info("Using committedLog for peer sid: " + getSid());
+ Iterator<Proposal> itr = db.getCommittedLog().iterator();
+ currentZxid = queueCommittedProposals(itr, peerLastZxid,
+ null, maxCommittedLog);
+ needSnap = false;
+ } else if (peerLastZxid < minCommittedLog && txnLogSyncEnabled) {
+ // Use txnlog and committedLog to sync
+
+ // Calculate sizeLimit that we allow to retrieve txnlog from disk
+ long sizeLimit = db.calculateTxnLogSizeLimit();
+ // This method can return empty iterator if the requested zxid
+ // is older than on-disk txnlog
+ Iterator<Proposal> txnLogItr = db.getProposalsFromTxnLog(
+ peerLastZxid, sizeLimit);
+ if (txnLogItr.hasNext()) {
+ LOG.info("Use txnlog and committedLog for peer sid: " + getSid());
+ currentZxid = queueCommittedProposals(txnLogItr, peerLastZxid,
+ minCommittedLog, maxCommittedLog);
+
+ LOG.debug("Queueing committedLog 0x" + Long.toHexString(currentZxid));
+ Iterator<Proposal> committedLogItr = db.getCommittedLog().iterator();
+ currentZxid = queueCommittedProposals(committedLogItr, currentZxid,
+ null, maxCommittedLog);
+ needSnap = false;
+ }
+ // closing the resources
+ if (txnLogItr instanceof TxnLogProposalIterator) {
+ TxnLogProposalIterator txnProposalItr = (TxnLogProposalIterator) txnLogItr;
+ txnProposalItr.close();
+ }
+ } else {
+ LOG.warn("Unhandled scenario for peer sid: " + getSid());
+ }
+ LOG.debug("Start forwarding 0x" + Long.toHexString(currentZxid) +
+ " for peer sid: " + getSid());
+ leaderLastZxid = leader.startForwarding(this, currentZxid);
+ } finally {
+ rl.unlock();
+ }
+
+ if (needOpPacket && !needSnap) {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method `LearnerHandler.syncFollower(...)` reads without synchronization from `this.needOpPacket`. Potentially races with write in method `LearnerHandler.setFirstPacket(...)`.
Reporting because this access may occur on a background thread.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
##########
@@ -0,0 +1,2197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.net.DatagramPacket;
+import java.net.DatagramSocket;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.net.SocketException;
+import java.net.UnknownHostException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.zookeeper.KeeperException.BadArgumentsException;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom.WriterStatement;
+import org.apache.zookeeper.common.QuorumX509Util;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.jmx.ZKMBeanInfo;
+import org.apache.zookeeper.server.ServerCnxnFactory;
+import org.apache.zookeeper.server.ZKDatabase;
+import org.apache.zookeeper.server.ZooKeeperServer;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuth;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthServer;
+import org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer;
+import org.apache.zookeeper.server.quorum.auth.NullQuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.NullQuorumAuthServer;
+import org.apache.zookeeper.server.admin.AdminServer;
+import org.apache.zookeeper.server.admin.AdminServer.AdminServerException;
+import org.apache.zookeeper.server.admin.AdminServerFactory;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.ConfigUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.apache.zookeeper.common.NetUtils.formatInetAddr;
+
+/**
+ * This class manages the quorum protocol. There are three states this server
+ * can be in:
+ * <ol>
+ * <li>Leader election - each server will elect a leader (proposing itself as a
+ * leader initially).</li>
+ * <li>Follower - the server will synchronize with the leader and replicate any
+ * transactions.</li>
+ * <li>Leader - the server will process requests and forward them to followers.
+ * A majority of followers must log the request before it can be accepted.
+ * </ol>
+ *
+ * This class will setup a datagram socket that will always respond with its
+ * view of the current leader. The response will take the form of:
+ *
+ * <pre>
+ * int xid;
+ *
+ * long myid;
+ *
+ * long leader_id;
+ *
+ * long leader_zxid;
+ * </pre>
+ *
+ * The request for the current leader will consist solely of an xid: int xid;
+ */
+public class QuorumPeer extends ZooKeeperThread implements QuorumStats.Provider {
+ private static final Logger LOG = LoggerFactory.getLogger(QuorumPeer.class);
+
+ private QuorumBean jmxQuorumBean;
+ LocalPeerBean jmxLocalPeerBean;
+ private Map<Long, RemotePeerBean> jmxRemotePeerBean;
+ LeaderElectionBean jmxLeaderElectionBean;
+
+ // The QuorumCnxManager is held through an AtomicReference to ensure cross-thread visibility
+ // of updates; see the implementation comment at setLastSeenQuorumVerifier().
+ private AtomicReference<QuorumCnxManager> qcmRef = new AtomicReference<>();
+
+ QuorumAuthServer authServer;
+ QuorumAuthLearner authLearner;
+
+ /**
+ * ZKDatabase is a top level member of quorumpeer
+ * which will be used in all the zookeeperservers
+ * instantiated later. Also, it is created once on
+ * bootup and only thrown away in case of a truncate
+ * message from the leader
+ */
+ private ZKDatabase zkDb;
+
+ public static final class AddressTuple {
+ public final InetSocketAddress quorumAddr;
+ public final InetSocketAddress electionAddr;
+ public final InetSocketAddress clientAddr;
+
+ public AddressTuple(InetSocketAddress quorumAddr, InetSocketAddress electionAddr, InetSocketAddress clientAddr) {
+ this.quorumAddr = quorumAddr;
+ this.electionAddr = electionAddr;
+ this.clientAddr = clientAddr;
+ }
+ }
+
+ public static class QuorumServer {
+ public InetSocketAddress addr = null;
+
+ public InetSocketAddress electionAddr = null;
+
+ public InetSocketAddress clientAddr = null;
+
+ public long id;
+
+ public String hostname;
+
+ public LearnerType type = LearnerType.PARTICIPANT;
+
+ private List<InetSocketAddress> myAddrs;
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, InetSocketAddress clientAddr) {
+ this(id, addr, electionAddr, clientAddr, LearnerType.PARTICIPANT);
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr) {
+ this(id, addr, electionAddr, (InetSocketAddress)null, LearnerType.PARTICIPANT);
+ }
+
+ // VisibleForTesting
+ public QuorumServer(long id, InetSocketAddress addr) {
+ this(id, addr, (InetSocketAddress)null, (InetSocketAddress)null, LearnerType.PARTICIPANT);
+ }
+
+ public long getId() {
+ return id;
+ }
+
+ /**
+ * Performs a DNS lookup for server address and election address.
+ *
+ * If the DNS lookup fails, this.addr and electionAddr remain
+ * unmodified.
+ */
+ public void recreateSocketAddresses() {
+ if (this.addr == null) {
+ LOG.warn("Server address has not been initialized");
+ return;
+ }
+ if (this.electionAddr == null) {
+ LOG.warn("Election address has not been initialized");
+ return;
+ }
+ String host = this.addr.getHostString();
+ InetAddress address = null;
+ try {
+ address = InetAddress.getByName(host);
+ } catch (UnknownHostException ex) {
+ LOG.warn("Failed to resolve address: {}", host, ex);
+ return;
+ }
+ LOG.debug("Resolved address for {}: {}", host, address);
+ int port = this.addr.getPort();
+ this.addr = new InetSocketAddress(address, port);
+ port = this.electionAddr.getPort();
+ this.electionAddr = new InetSocketAddress(address, port);
+ }
+
+ private void setType(String s) throws ConfigException {
+ if (s.toLowerCase().equals("observer")) {
+ type = LearnerType.OBSERVER;
+ } else if (s.toLowerCase().equals("participant")) {
+ type = LearnerType.PARTICIPANT;
+ } else {
+ throw new ConfigException("Unrecognised peertype: " + s);
+ }
+ }
+
+ private static final String wrongFormat = " does not have the form server_config or server_config;client_config"+
+ " where server_config is host:port:port or host:port:port:type and client_config is port or host:port";
+
+ public QuorumServer(long sid, String addressStr) throws ConfigException {
+ // LOG.warn("sid = " + sid + " addressStr = " + addressStr);
+ this.id = sid;
+ String serverClientParts[] = addressStr.split(";");
+ String serverParts[] = ConfigUtils.getHostAndPort(serverClientParts[0]);
+ if ((serverClientParts.length > 2) || (serverParts.length < 3)
+ || (serverParts.length > 4)) {
+ throw new ConfigException(addressStr + wrongFormat);
+ }
+
+ if (serverClientParts.length == 2) {
+ //LOG.warn("ClientParts: " + serverClientParts[1]);
+ String clientParts[] = ConfigUtils.getHostAndPort(serverClientParts[1]);
+ if (clientParts.length > 2) {
+ throw new ConfigException(addressStr + wrongFormat);
+ }
+
+ // is client_config a host:port or just a port
+ hostname = (clientParts.length == 2) ? clientParts[0] : "0.0.0.0";
+ try {
+ clientAddr = new InetSocketAddress(hostname,
+ Integer.parseInt(clientParts[clientParts.length - 1]));
+ //LOG.warn("Set clientAddr to " + clientAddr);
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + hostname + ":" + clientParts[clientParts.length - 1]);
+ }
+ }
+
+ // server_config should be either host:port:port or host:port:port:type
+ try {
+ addr = new InetSocketAddress(serverParts[0],
+ Integer.parseInt(serverParts[1]));
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + serverParts[0] + ":" + serverParts[1]);
+ }
+ try {
+ electionAddr = new InetSocketAddress(serverParts[0],
+ Integer.parseInt(serverParts[2]));
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + serverParts[0] + ":" + serverParts[2]);
+ }
+
+ if(addr.getPort() == electionAddr.getPort()) {
+ throw new ConfigException(
+ "Client and election port must be different! Please update the configuration file on server." + sid);
+ }
+
+ if (serverParts.length == 4) {
+ setType(serverParts[3]);
+ }
+
+ this.hostname = serverParts[0];
+
+ setMyAddrs();
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, LearnerType type) {
+ this(id, addr, electionAddr, (InetSocketAddress)null, type);
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, InetSocketAddress clientAddr, LearnerType type) {
+ this.id = id;
+ this.addr = addr;
+ this.electionAddr = electionAddr;
+ this.type = type;
+ this.clientAddr = clientAddr;
+
+ setMyAddrs();
+ }
+
+ private void setMyAddrs() {
+ this.myAddrs = new ArrayList<InetSocketAddress>();
+ this.myAddrs.add(this.addr);
+ this.myAddrs.add(this.clientAddr);
+ this.myAddrs.add(this.electionAddr);
+ this.myAddrs = excludedSpecialAddresses(this.myAddrs);
+ }
+
+ private static String delimitedHostString(InetSocketAddress addr)
+ {
+ String host = addr.getHostString();
+ if (host.contains(":")) {
+ return "[" + host + "]";
+ } else {
+ return host;
+ }
+ }
+
+ public String toString(){
+ StringWriter sw = new StringWriter();
+ //addr should never be null, but just to make sure
+ if (addr !=null) {
+ sw.append(delimitedHostString(addr));
+ sw.append(":");
+ sw.append(String.valueOf(addr.getPort()));
+ }
+ if (electionAddr!=null){
+ sw.append(":");
+ sw.append(String.valueOf(electionAddr.getPort()));
+ }
+ if (type == LearnerType.OBSERVER) sw.append(":observer");
+ else if (type == LearnerType.PARTICIPANT) sw.append(":participant");
+ if (clientAddr!=null){
+ sw.append(";");
+ sw.append(delimitedHostString(clientAddr));
+ sw.append(":");
+ sw.append(String.valueOf(clientAddr.getPort()));
+ }
+ return sw.toString();
+ }
+
+ public int hashCode() {
+ assert false : "hashCode not designed";
+ return 42; // any arbitrary constant will do
+ }
+
+ private boolean checkAddressesEqual(InetSocketAddress addr1, InetSocketAddress addr2){
+ if ((addr1 == null && addr2!=null) ||
+ (addr1!=null && addr2==null) ||
+ (addr1!=null && addr2!=null && !addr1.equals(addr2))) return false;
+ return true;
+ }
+
+ public boolean equals(Object o){
+ if (!(o instanceof QuorumServer)) return false;
+ QuorumServer qs = (QuorumServer)o;
+ if ((qs.id != id) || (qs.type != type)) return false;
+ if (!checkAddressesEqual(addr, qs.addr)) return false;
+ if (!checkAddressesEqual(electionAddr, qs.electionAddr)) return false;
+ if (!checkAddressesEqual(clientAddr, qs.clientAddr)) return false;
+ return true;
+ }
+
+ public void checkAddressDuplicate(QuorumServer s) throws BadArgumentsException {
+ List<InetSocketAddress> otherAddrs = new ArrayList<InetSocketAddress>();
+ otherAddrs.add(s.addr);
+ otherAddrs.add(s.clientAddr);
+ otherAddrs.add(s.electionAddr);
+ otherAddrs = excludedSpecialAddresses(otherAddrs);
+
+ for (InetSocketAddress my: this.myAddrs) {
+
+ for (InetSocketAddress other: otherAddrs) {
+ if (my.equals(other)) {
+ String error = String.format("%s of server.%d conflicts %s of server.%d", my, this.id, other, s.id);
+ throw new BadArgumentsException(error);
+ }
+ }
+ }
+ }
+
+ private List<InetSocketAddress> excludedSpecialAddresses(List<InetSocketAddress> addrs) {
+ List<InetSocketAddress> included = new ArrayList<InetSocketAddress>();
+ InetAddress wcAddr = new InetSocketAddress(0).getAddress();
+
+ for (InetSocketAddress addr : addrs) {
+ if (addr == null) {
+ continue;
+ }
+ InetAddress inetaddr = addr.getAddress();
+
+ if (inetaddr == null ||
+ inetaddr.equals(wcAddr) || // wildCard address(0.0.0.0)
+ inetaddr.isLoopbackAddress()) { // loopback address(localhost/127.0.0.1)
+ continue;
+ }
+ included.add(addr);
+ }
+ return included;
+ }
+ }
+
+
+ public enum ServerState {
+ LOOKING, FOLLOWING, LEADING, OBSERVING;
+ }
+
+ /*
+ * A peer can either be participating, which implies that it is willing to
+ * both vote in instances of consensus and to elect or become a Leader, or
+ * it may be observing in which case it isn't.
+ *
+ * We need this distinction to decide which ServerState to move to when
+ * conditions change (e.g. which state to become after LOOKING).
+ */
+ public enum LearnerType {
+ PARTICIPANT, OBSERVER;
+ }
+
+ /*
+ * To enable observers to have no identifier, we need a generic identifier
+ * at least for QuorumCnxManager. We use the following constant to as the
+ * value of such a generic identifier.
+ */
+
+ static final long OBSERVER_ID = Long.MAX_VALUE;
+
+ /*
+ * Record leader election time
+ */
+ public long start_fle, end_fle; // fle = fast leader election
+ public static final String FLE_TIME_UNIT = "MS";
+
+ /*
+ * Default value of peer is participant
+ */
+ private LearnerType learnerType = LearnerType.PARTICIPANT;
+
+ public LearnerType getLearnerType() {
+ return learnerType;
+ }
+
+ /**
+ * Sets the LearnerType
+ */
+ public void setLearnerType(LearnerType p) {
+ learnerType = p;
+ }
+
+ protected synchronized void setConfigFileName(String s) {
+ configFilename = s;
+ }
+
+ private String configFilename = null;
+
+ public int getQuorumSize(){
+ return getVotingView().size();
+ }
+
+ /**
+ * QuorumVerifier implementation; default (majority).
+ */
+
+ //last committed quorum verifier
+ private QuorumVerifier quorumVerifier;
+
+ //last proposed quorum verifier
+ private QuorumVerifier lastSeenQuorumVerifier = null;
+
+ // Lock object that guard access to quorumVerifier and lastSeenQuorumVerifier.
+ final Object QV_LOCK = new Object();
+
+
+ /**
+ * My id
+ */
+ private long myid;
+
+
+ /**
+ * get the id of this quorum peer.
+ */
+ public long getId() {
+ return myid;
+ }
+
+ // VisibleForTesting
+ void setId(long id) {
+ this.myid = id;
+ }
+
+ private boolean sslQuorum;
+ private boolean shouldUsePortUnification;
+
+ public boolean isSslQuorum() {
+ return sslQuorum;
+ }
+
+ public boolean shouldUsePortUnification() {
+ return shouldUsePortUnification;
+ }
+
+ private final QuorumX509Util x509Util;
+
+ QuorumX509Util getX509Util() {
+ return x509Util;
+ }
+
+ /**
+ * This is who I think the leader currently is.
+ */
+ volatile private Vote currentVote;
+
+ public synchronized Vote getCurrentVote(){
+ return currentVote;
+ }
+
+ public synchronized void setCurrentVote(Vote v){
+ currentVote = v;
+ }
+
+ private volatile boolean running = true;
+
+ /**
+ * The number of milliseconds of each tick
+ */
+ protected int tickTime;
+
+ /**
+ * Whether learners in this quorum should create new sessions as local.
+ * False by default to preserve existing behavior.
+ */
+ protected boolean localSessionsEnabled = false;
+
+ /**
+ * Whether learners in this quorum should upgrade local sessions to
+ * global. Only matters if local sessions are enabled.
+ */
+ protected boolean localSessionsUpgradingEnabled = true;
+
+ /**
+ * Minimum number of milliseconds to allow for session timeout.
+ * A value of -1 indicates unset, use default.
+ */
+ protected int minSessionTimeout = -1;
+
+ /**
+ * Maximum number of milliseconds to allow for session timeout.
+ * A value of -1 indicates unset, use default.
+ */
+ protected int maxSessionTimeout = -1;
+
+ /**
+ * The number of ticks that the initial synchronization phase can take
+ */
+ protected int initLimit;
+
+ /**
+ * The number of ticks that can pass between sending a request and getting
+ * an acknowledgment
+ */
+ protected int syncLimit;
+
+ /**
+ * Enables/Disables sync request processor. This option is enabled
+ * by default and is to be used with observers.
+ */
+ protected boolean syncEnabled = true;
+
+ /**
+ * The current tick
+ */
+ protected AtomicInteger tick = new AtomicInteger();
+
+ /**
+ * Whether or not to listen on all IPs for the two quorum ports
+ * (broadcast and fast leader election).
+ */
+ protected boolean quorumListenOnAllIPs = false;
+
+ /**
+ * Keeps time taken for leader election in milliseconds. Sets the value to
+ * this variable only after the completion of leader election.
+ */
+ private long electionTimeTaken = -1;
+
+ /**
+ * Enable/Disables quorum authentication using sasl. Defaulting to false.
+ */
+ protected boolean quorumSaslEnableAuth;
+
+ /**
+ * If this is false, quorum peer server will accept another quorum peer client
+ * connection even if the authentication did not succeed. This can be used while
+ * upgrading ZooKeeper server. Defaulting to false (required).
+ */
+ protected boolean quorumServerSaslAuthRequired;
+
+ /**
+ * If this is false, quorum peer learner will talk to quorum peer server
+ * without authentication. This can be used while upgrading ZooKeeper
+ * server. Defaulting to false (required).
+ */
+ protected boolean quorumLearnerSaslAuthRequired;
+
+ /**
+ * Kerberos quorum service principal. Defaulting to 'zkquorum/localhost'.
+ */
+ protected String quorumServicePrincipal;
+
+ /**
+ * Quorum learner login context name in jaas-conf file to read the kerberos
+ * security details. Defaulting to 'QuorumLearner'.
+ */
+ protected String quorumLearnerLoginContext;
+
+ /**
+ * Quorum server login context name in jaas-conf file to read the kerberos
+ * security details. Defaulting to 'QuorumServer'.
+ */
+ protected String quorumServerLoginContext;
+
+ // TODO: need to tune the default value of thread size
+ private static final int QUORUM_CNXN_THREADS_SIZE_DEFAULT_VALUE = 20;
+ /**
+ * The maximum number of threads to allow in the connectionExecutors thread
+ * pool which will be used to initiate quorum server connections.
+ */
+ protected int quorumCnxnThreadsSize = QUORUM_CNXN_THREADS_SIZE_DEFAULT_VALUE;
+
+ /**
+ * @deprecated As of release 3.4.0, this class has been deprecated, since
+ * it is used with one of the udp-based versions of leader election, which
+ * we are also deprecating.
+ *
+ * This class simply responds to requests for the current leader of this
+ * node.
+ * <p>
+ * The request contains just an xid generated by the requestor.
+ * <p>
+ * The response has the xid, the id of this server, the id of the leader,
+ * and the zxid of the leader.
+ *
+ *
+ */
+ @Deprecated
+ class ResponderThread extends ZooKeeperThread {
+ ResponderThread() {
+ super("ResponderThread");
+ }
+
+ volatile boolean running = true;
+
+ @Override
+ public void run() {
+ try {
+ byte b[] = new byte[36];
+ ByteBuffer responseBuffer = ByteBuffer.wrap(b);
+ DatagramPacket packet = new DatagramPacket(b, b.length);
+ while (running) {
+ udpSocket.receive(packet);
+ if (packet.getLength() != 4) {
+ LOG.warn("Got more than just an xid! Len = "
+ + packet.getLength());
+ } else {
+ responseBuffer.clear();
+ responseBuffer.getInt(); // Skip the xid
+ responseBuffer.putLong(myid);
+ Vote current = getCurrentVote();
+ switch (getPeerState()) {
+ case LOOKING:
+ responseBuffer.putLong(current.getId());
+ responseBuffer.putLong(current.getZxid());
+ break;
+ case LEADING:
+ responseBuffer.putLong(myid);
+ try {
+ long proposed;
+ synchronized(leader) {
+ proposed = leader.lastProposed;
+ }
+ responseBuffer.putLong(proposed);
+ } catch (NullPointerException npe) {
+ // This can happen in state transitions,
+ // just ignore the request
+ }
+ break;
+ case FOLLOWING:
+ responseBuffer.putLong(current.getId());
+ try {
+ responseBuffer.putLong(follower.getZxid());
+ } catch (NullPointerException npe) {
+ // This can happen in state transitions,
+ // just ignore the request
+ }
+ break;
+ case OBSERVING:
+ // Do nothing, Observers keep themselves to
+ // themselves.
+ break;
+ }
+ packet.setData(b);
+ udpSocket.send(packet);
+ }
+ packet.setLength(b.length);
+ }
+ } catch (RuntimeException e) {
+ LOG.warn("Unexpected runtime exception in ResponderThread",e);
+ } catch (IOException e) {
+ LOG.warn("Unexpected IO exception in ResponderThread",e);
+ } finally {
+ LOG.warn("QuorumPeer responder thread exited");
+ }
+ }
+ }
+
+ private ServerState state = ServerState.LOOKING;
+
+ private boolean reconfigFlag = false; // indicates that a reconfig just committed
+
+ public synchronized void setPeerState(ServerState newState){
+ state=newState;
+ }
+ public synchronized void reconfigFlagSet(){
+ reconfigFlag = true;
+ }
+ public synchronized void reconfigFlagClear(){
+ reconfigFlag = false;
+ }
+ public synchronized boolean isReconfigStateChange(){
+ return reconfigFlag;
+ }
+ public synchronized ServerState getPeerState(){
+ return state;
+ }
+
+ DatagramSocket udpSocket;
+
+ private final AtomicReference<AddressTuple> myAddrs = new AtomicReference<>();
+
+ /**
+ * Resolves hostname for a given server ID.
+ *
+ * This method resolves hostname for a given server ID in both quorumVerifer
+ * and lastSeenQuorumVerifier. If the server ID matches the local server ID,
+ * it also updates myAddrs.
+ */
+ public void recreateSocketAddresses(long id) {
+ QuorumVerifier qv = getQuorumVerifier();
+ if (qv != null) {
+ QuorumServer qs = qv.getAllMembers().get(id);
+ if (qs != null) {
+ qs.recreateSocketAddresses();
+ if (id == getId()) {
+ setAddrs(qs.addr, qs.electionAddr, qs.clientAddr);
+ }
+ }
+ }
+ qv = getLastSeenQuorumVerifier();
+ if (qv != null) {
+ QuorumServer qs = qv.getAllMembers().get(id);
+ if (qs != null) {
+ qs.recreateSocketAddresses();
+ }
+ }
+ }
+
+ private AddressTuple getAddrs(){
+ AddressTuple addrs = myAddrs.get();
+ if (addrs != null) {
+ return addrs;
+ }
+ try {
+ synchronized (QV_LOCK) {
+ addrs = myAddrs.get();
+ while (addrs == null) {
+ QV_LOCK.wait();
+ addrs = myAddrs.get();
+ }
+ return addrs;
+ }
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new RuntimeException(e);
+ }
+ }
+
+ public InetSocketAddress getQuorumAddress(){
+ return getAddrs().quorumAddr;
+ }
+
+ public InetSocketAddress getElectionAddress(){
+ return getAddrs().electionAddr;
+ }
+
+ public InetSocketAddress getClientAddress(){
+ final AddressTuple addrs = myAddrs.get();
+ return (addrs == null) ? null : addrs.clientAddr;
+ }
+
+ private void setAddrs(InetSocketAddress quorumAddr, InetSocketAddress electionAddr, InetSocketAddress clientAddr){
+ synchronized (QV_LOCK) {
+ myAddrs.set(new AddressTuple(quorumAddr, electionAddr, clientAddr));
+ QV_LOCK.notifyAll();
+ }
+ }
+
+ private int electionType;
+
+ Election electionAlg;
+
+ ServerCnxnFactory cnxnFactory;
+ ServerCnxnFactory secureCnxnFactory;
+
+ private FileTxnSnapLog logFactory = null;
+
+ private final QuorumStats quorumStats;
+
+ AdminServer adminServer;
+
+ public static QuorumPeer testingQuorumPeer() throws SaslException {
+ return new QuorumPeer();
+ }
+
+ public QuorumPeer() throws SaslException {
+ super("QuorumPeer");
+ quorumStats = new QuorumStats(this);
+ jmxRemotePeerBean = new HashMap<Long, RemotePeerBean>();
+ adminServer = AdminServerFactory.createAdminServer();
+ x509Util = new QuorumX509Util();
+ initialize();
+ }
+
+ /**
+ * For backward compatibility purposes, we instantiate QuorumMaj by default.
+ */
+
+ public QuorumPeer(Map<Long, QuorumServer> quorumPeers, File dataDir,
+ File dataLogDir, int electionType,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ ServerCnxnFactory cnxnFactory) throws IOException {
+ this(quorumPeers, dataDir, dataLogDir, electionType, myid, tickTime,
+ initLimit, syncLimit, false, cnxnFactory,
+ new QuorumMaj(quorumPeers));
+ }
+
+ public QuorumPeer(Map<Long, QuorumServer> quorumPeers, File dataDir,
+ File dataLogDir, int electionType,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ boolean quorumListenOnAllIPs,
+ ServerCnxnFactory cnxnFactory,
+ QuorumVerifier quorumConfig) throws IOException {
+ this();
+ this.cnxnFactory = cnxnFactory;
+ this.electionType = electionType;
+ this.myid = myid;
+ this.tickTime = tickTime;
+ this.initLimit = initLimit;
+ this.syncLimit = syncLimit;
+ this.quorumListenOnAllIPs = quorumListenOnAllIPs;
+ this.logFactory = new FileTxnSnapLog(dataLogDir, dataDir);
+ this.zkDb = new ZKDatabase(this.logFactory);
+ if(quorumConfig == null) quorumConfig = new QuorumMaj(quorumPeers);
+ setQuorumVerifier(quorumConfig, false);
+ adminServer = AdminServerFactory.createAdminServer();
+ }
+
+ public void initialize() throws SaslException {
+ // init quorum auth server & learner
+ if (isQuorumSaslAuthEnabled()) {
+ Set<String> authzHosts = new HashSet<String>();
+ for (QuorumServer qs : getView().values()) {
+ authzHosts.add(qs.hostname);
+ }
+ authServer = new SaslQuorumAuthServer(isQuorumServerSaslAuthRequired(),
+ quorumServerLoginContext, authzHosts);
+ authLearner = new SaslQuorumAuthLearner(isQuorumLearnerSaslAuthRequired(),
+ quorumServicePrincipal, quorumLearnerLoginContext);
+ } else {
+ authServer = new NullQuorumAuthServer();
+ authLearner = new NullQuorumAuthLearner();
+ }
+ }
+
+ QuorumStats quorumStats() {
+ return quorumStats;
+ }
+
+ @Override
+ public synchronized void start() {
+ if (!getView().containsKey(myid)) {
+ throw new RuntimeException("My id " + myid + " not in the peer list");
+ }
+ loadDataBase();
+ startServerCnxnFactory();
+ try {
+ adminServer.start();
+ } catch (AdminServerException e) {
+ LOG.warn("Problem starting AdminServer", e);
+ System.out.println(e);
+ }
+ startLeaderElection();
+ super.start();
+ }
+
+ private void loadDataBase() {
+ try {
+ zkDb.loadDataBase();
+
+ // load the epochs
+ long lastProcessedZxid = zkDb.getDataTree().lastProcessedZxid;
+ long epochOfZxid = ZxidUtils.getEpochFromZxid(lastProcessedZxid);
+ try {
+ currentEpoch = readLongFromFile(CURRENT_EPOCH_FILENAME);
+ } catch(FileNotFoundException e) {
+ // pick a reasonable epoch number
+ // this should only happen once when moving to a
+ // new code version
+ currentEpoch = epochOfZxid;
+ LOG.info(CURRENT_EPOCH_FILENAME
+ + " not found! Creating with a reasonable default of {}. This should only happen when you are upgrading your installation",
+ currentEpoch);
+ writeLongToFile(CURRENT_EPOCH_FILENAME, currentEpoch);
+ }
+ if (epochOfZxid > currentEpoch) {
+ throw new IOException("The current epoch, " + ZxidUtils.zxidToString(currentEpoch) + ", is older than the last zxid, " + lastProcessedZxid);
+ }
+ try {
+ acceptedEpoch = readLongFromFile(ACCEPTED_EPOCH_FILENAME);
+ } catch(FileNotFoundException e) {
+ // pick a reasonable epoch number
+ // this should only happen once when moving to a
+ // new code version
+ acceptedEpoch = epochOfZxid;
+ LOG.info(ACCEPTED_EPOCH_FILENAME
+ + " not found! Creating with a reasonable default of {}. This should only happen when you are upgrading your installation",
+ acceptedEpoch);
+ writeLongToFile(ACCEPTED_EPOCH_FILENAME, acceptedEpoch);
+ }
+ if (acceptedEpoch < currentEpoch) {
+ throw new IOException("The accepted epoch, " + ZxidUtils.zxidToString(acceptedEpoch) + " is less than the current epoch, " + ZxidUtils.zxidToString(currentEpoch));
+ }
+ } catch(IOException ie) {
+ LOG.error("Unable to load database on disk", ie);
+ throw new RuntimeException("Unable to run quorum server ", ie);
+ }
+ }
+
+ ResponderThread responder;
+
+ synchronized public void stopLeaderElection() {
+ responder.running = false;
+ responder.interrupt();
+ }
+ synchronized public void startLeaderElection() {
+ try {
+ if (getPeerState() == ServerState.LOOKING) {
+ currentVote = new Vote(myid, getLastLoggedZxid(), getCurrentEpoch());
+ }
+ } catch(IOException e) {
+ RuntimeException re = new RuntimeException(e.getMessage());
+ re.setStackTrace(e.getStackTrace());
+ throw re;
+ }
+
+ // if (!getView().containsKey(myid)) {
+ // throw new RuntimeException("My id " + myid + " not in the peer list");
+ //}
+ if (electionType == 0) {
+ try {
+ udpSocket = new DatagramSocket(getQuorumAddress().getPort());
+ responder = new ResponderThread();
+ responder.start();
+ } catch (SocketException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ this.electionAlg = createElectionAlgorithm(electionType);
+ }
+
+ /**
+ * Count the number of nodes in the map that could be followers.
+ * @param peers
+ * @return The number of followers in the map
+ */
+ protected static int countParticipants(Map<Long,QuorumServer> peers) {
+ int count = 0;
+ for (QuorumServer q : peers.values()) {
+ if (q.type == LearnerType.PARTICIPANT) {
+ count++;
+ }
+ }
+ return count;
+ }
+
+ /**
+
+ * This constructor is only used by the existing unit test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public QuorumPeer(Map<Long,QuorumServer> quorumPeers, File snapDir,
+ File logDir, int clientPort, int electionAlg,
+ long myid, int tickTime, int initLimit, int syncLimit)
+ throws IOException
+ {
+ this(quorumPeers, snapDir, logDir, electionAlg, myid, tickTime, initLimit, syncLimit, false,
+ ServerCnxnFactory.createFactory(getClientAddress(quorumPeers, myid, clientPort), -1),
+ new QuorumMaj(quorumPeers));
+ }
+
+ /**
+ * This constructor is only used by the existing unit test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public QuorumPeer(Map<Long,QuorumServer> quorumPeers, File snapDir,
+ File logDir, int clientPort, int electionAlg,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ QuorumVerifier quorumConfig)
+ throws IOException
+ {
+ this(quorumPeers, snapDir, logDir, electionAlg,
+ myid,tickTime, initLimit,syncLimit, false,
+ ServerCnxnFactory.createFactory(getClientAddress(quorumPeers, myid, clientPort), -1),
+ quorumConfig);
+ }
+
+ private static InetSocketAddress getClientAddress(Map<Long, QuorumServer> quorumPeers, long myid, int clientPort)
+ throws IOException {
+ QuorumServer quorumServer = quorumPeers.get(myid);
+ if (null == quorumServer) {
+ throw new IOException("No QuorumServer correspoding to myid " + myid);
+ }
+ if (null == quorumServer.clientAddr) {
+ return new InetSocketAddress(clientPort);
+ }
+ if (quorumServer.clientAddr.getPort() != clientPort) {
+ throw new IOException("QuorumServer port " + quorumServer.clientAddr.getPort()
+ + " does not match with given port " + clientPort);
+ }
+ return quorumServer.clientAddr;
+ }
+
+ /**
+ * returns the highest zxid that this host has seen
+ *
+ * @return the highest zxid for this host
+ */
+ public long getLastLoggedZxid() {
+ if (!zkDb.isInitialized()) {
+ loadDataBase();
+ }
+ return zkDb.getDataTreeLastProcessedZxid();
+ }
+
+ public Follower follower;
+ public Leader leader;
+ public Observer observer;
+
+ protected Follower makeFollower(FileTxnSnapLog logFactory) throws IOException {
+ return new Follower(this, new FollowerZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ protected Leader makeLeader(FileTxnSnapLog logFactory) throws IOException, X509Exception {
+ return new Leader(this, new LeaderZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ protected Observer makeObserver(FileTxnSnapLog logFactory) throws IOException {
+ return new Observer(this, new ObserverZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ @SuppressWarnings("deprecation")
+ protected Election createElectionAlgorithm(int electionAlgorithm){
+ Election le=null;
+
+ //TODO: use a factory rather than a switch
+ switch (electionAlgorithm) {
+ case 0:
+ le = new LeaderElection(this);
+ break;
+ case 1:
+ le = new AuthFastLeaderElection(this);
+ break;
+ case 2:
+ le = new AuthFastLeaderElection(this, true);
+ break;
+ case 3:
+ QuorumCnxManager qcm = createCnxnManager();
+ QuorumCnxManager oldQcm = qcmRef.getAndSet(qcm);
+ if (oldQcm != null) {
+ LOG.warn("Clobbering already-set QuorumCnxManager (restarting leader election?)");
+ oldQcm.halt();
+ }
+ QuorumCnxManager.Listener listener = qcm.listener;
+ if(listener != null){
+ listener.start();
+ FastLeaderElection fle = new FastLeaderElection(this, qcm);
+ fle.start();
+ le = fle;
+ } else {
+ LOG.error("Null listener when initializing cnx manager");
+ }
+ break;
+ default:
+ assert false;
+ }
+ return le;
+ }
+
+ @SuppressWarnings("deprecation")
+ protected Election makeLEStrategy(){
+ LOG.debug("Initializing leader election protocol...");
+ if (getElectionType() == 0) {
+ electionAlg = new LeaderElection(this);
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `QuorumPeer.makeLEStrategy()` writes to field `this.electionAlg` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
##########
@@ -0,0 +1,2197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.net.DatagramPacket;
+import java.net.DatagramSocket;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.net.SocketException;
+import java.net.UnknownHostException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.zookeeper.KeeperException.BadArgumentsException;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom.WriterStatement;
+import org.apache.zookeeper.common.QuorumX509Util;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.common.X509Exception;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.jmx.ZKMBeanInfo;
+import org.apache.zookeeper.server.ServerCnxnFactory;
+import org.apache.zookeeper.server.ZKDatabase;
+import org.apache.zookeeper.server.ZooKeeperServer;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuth;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuthServer;
+import org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer;
+import org.apache.zookeeper.server.quorum.auth.NullQuorumAuthLearner;
+import org.apache.zookeeper.server.quorum.auth.NullQuorumAuthServer;
+import org.apache.zookeeper.server.admin.AdminServer;
+import org.apache.zookeeper.server.admin.AdminServer.AdminServerException;
+import org.apache.zookeeper.server.admin.AdminServerFactory;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.ConfigUtils;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.apache.zookeeper.common.NetUtils.formatInetAddr;
+
+/**
+ * This class manages the quorum protocol. There are three states this server
+ * can be in:
+ * <ol>
+ * <li>Leader election - each server will elect a leader (proposing itself as a
+ * leader initially).</li>
+ * <li>Follower - the server will synchronize with the leader and replicate any
+ * transactions.</li>
+ * <li>Leader - the server will process requests and forward them to followers.
+ * A majority of followers must log the request before it can be accepted.
+ * </ol>
+ *
+ * This class will setup a datagram socket that will always respond with its
+ * view of the current leader. The response will take the form of:
+ *
+ * <pre>
+ * int xid;
+ *
+ * long myid;
+ *
+ * long leader_id;
+ *
+ * long leader_zxid;
+ * </pre>
+ *
+ * The request for the current leader will consist solely of an xid: int xid;
+ */
+public class QuorumPeer extends ZooKeeperThread implements QuorumStats.Provider {
+ private static final Logger LOG = LoggerFactory.getLogger(QuorumPeer.class);
+
+ private QuorumBean jmxQuorumBean;
+ LocalPeerBean jmxLocalPeerBean;
+ private Map<Long, RemotePeerBean> jmxRemotePeerBean;
+ LeaderElectionBean jmxLeaderElectionBean;
+
+ // The QuorumCnxManager is held through an AtomicReference to ensure cross-thread visibility
+ // of updates; see the implementation comment at setLastSeenQuorumVerifier().
+ private AtomicReference<QuorumCnxManager> qcmRef = new AtomicReference<>();
+
+ QuorumAuthServer authServer;
+ QuorumAuthLearner authLearner;
+
+ /**
+ * ZKDatabase is a top level member of quorumpeer
+ * which will be used in all the zookeeperservers
+ * instantiated later. Also, it is created once on
+ * bootup and only thrown away in case of a truncate
+ * message from the leader
+ */
+ private ZKDatabase zkDb;
+
+ public static final class AddressTuple {
+ public final InetSocketAddress quorumAddr;
+ public final InetSocketAddress electionAddr;
+ public final InetSocketAddress clientAddr;
+
+ public AddressTuple(InetSocketAddress quorumAddr, InetSocketAddress electionAddr, InetSocketAddress clientAddr) {
+ this.quorumAddr = quorumAddr;
+ this.electionAddr = electionAddr;
+ this.clientAddr = clientAddr;
+ }
+ }
+
+ public static class QuorumServer {
+ public InetSocketAddress addr = null;
+
+ public InetSocketAddress electionAddr = null;
+
+ public InetSocketAddress clientAddr = null;
+
+ public long id;
+
+ public String hostname;
+
+ public LearnerType type = LearnerType.PARTICIPANT;
+
+ private List<InetSocketAddress> myAddrs;
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, InetSocketAddress clientAddr) {
+ this(id, addr, electionAddr, clientAddr, LearnerType.PARTICIPANT);
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr) {
+ this(id, addr, electionAddr, (InetSocketAddress)null, LearnerType.PARTICIPANT);
+ }
+
+ // VisibleForTesting
+ public QuorumServer(long id, InetSocketAddress addr) {
+ this(id, addr, (InetSocketAddress)null, (InetSocketAddress)null, LearnerType.PARTICIPANT);
+ }
+
+ public long getId() {
+ return id;
+ }
+
+ /**
+ * Performs a DNS lookup for server address and election address.
+ *
+ * If the DNS lookup fails, this.addr and electionAddr remain
+ * unmodified.
+ */
+ public void recreateSocketAddresses() {
+ if (this.addr == null) {
+ LOG.warn("Server address has not been initialized");
+ return;
+ }
+ if (this.electionAddr == null) {
+ LOG.warn("Election address has not been initialized");
+ return;
+ }
+ String host = this.addr.getHostString();
+ InetAddress address = null;
+ try {
+ address = InetAddress.getByName(host);
+ } catch (UnknownHostException ex) {
+ LOG.warn("Failed to resolve address: {}", host, ex);
+ return;
+ }
+ LOG.debug("Resolved address for {}: {}", host, address);
+ int port = this.addr.getPort();
+ this.addr = new InetSocketAddress(address, port);
+ port = this.electionAddr.getPort();
+ this.electionAddr = new InetSocketAddress(address, port);
+ }
+
+ private void setType(String s) throws ConfigException {
+ if (s.toLowerCase().equals("observer")) {
+ type = LearnerType.OBSERVER;
+ } else if (s.toLowerCase().equals("participant")) {
+ type = LearnerType.PARTICIPANT;
+ } else {
+ throw new ConfigException("Unrecognised peertype: " + s);
+ }
+ }
+
+ private static final String wrongFormat = " does not have the form server_config or server_config;client_config"+
+ " where server_config is host:port:port or host:port:port:type and client_config is port or host:port";
+
+ public QuorumServer(long sid, String addressStr) throws ConfigException {
+ // LOG.warn("sid = " + sid + " addressStr = " + addressStr);
+ this.id = sid;
+ String serverClientParts[] = addressStr.split(";");
+ String serverParts[] = ConfigUtils.getHostAndPort(serverClientParts[0]);
+ if ((serverClientParts.length > 2) || (serverParts.length < 3)
+ || (serverParts.length > 4)) {
+ throw new ConfigException(addressStr + wrongFormat);
+ }
+
+ if (serverClientParts.length == 2) {
+ //LOG.warn("ClientParts: " + serverClientParts[1]);
+ String clientParts[] = ConfigUtils.getHostAndPort(serverClientParts[1]);
+ if (clientParts.length > 2) {
+ throw new ConfigException(addressStr + wrongFormat);
+ }
+
+ // is client_config a host:port or just a port
+ hostname = (clientParts.length == 2) ? clientParts[0] : "0.0.0.0";
+ try {
+ clientAddr = new InetSocketAddress(hostname,
+ Integer.parseInt(clientParts[clientParts.length - 1]));
+ //LOG.warn("Set clientAddr to " + clientAddr);
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + hostname + ":" + clientParts[clientParts.length - 1]);
+ }
+ }
+
+ // server_config should be either host:port:port or host:port:port:type
+ try {
+ addr = new InetSocketAddress(serverParts[0],
+ Integer.parseInt(serverParts[1]));
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + serverParts[0] + ":" + serverParts[1]);
+ }
+ try {
+ electionAddr = new InetSocketAddress(serverParts[0],
+ Integer.parseInt(serverParts[2]));
+ } catch (NumberFormatException e) {
+ throw new ConfigException("Address unresolved: " + serverParts[0] + ":" + serverParts[2]);
+ }
+
+ if(addr.getPort() == electionAddr.getPort()) {
+ throw new ConfigException(
+ "Client and election port must be different! Please update the configuration file on server." + sid);
+ }
+
+ if (serverParts.length == 4) {
+ setType(serverParts[3]);
+ }
+
+ this.hostname = serverParts[0];
+
+ setMyAddrs();
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, LearnerType type) {
+ this(id, addr, electionAddr, (InetSocketAddress)null, type);
+ }
+
+ public QuorumServer(long id, InetSocketAddress addr,
+ InetSocketAddress electionAddr, InetSocketAddress clientAddr, LearnerType type) {
+ this.id = id;
+ this.addr = addr;
+ this.electionAddr = electionAddr;
+ this.type = type;
+ this.clientAddr = clientAddr;
+
+ setMyAddrs();
+ }
+
+ private void setMyAddrs() {
+ this.myAddrs = new ArrayList<InetSocketAddress>();
+ this.myAddrs.add(this.addr);
+ this.myAddrs.add(this.clientAddr);
+ this.myAddrs.add(this.electionAddr);
+ this.myAddrs = excludedSpecialAddresses(this.myAddrs);
+ }
+
+ private static String delimitedHostString(InetSocketAddress addr)
+ {
+ String host = addr.getHostString();
+ if (host.contains(":")) {
+ return "[" + host + "]";
+ } else {
+ return host;
+ }
+ }
+
+ public String toString(){
+ StringWriter sw = new StringWriter();
+ //addr should never be null, but just to make sure
+ if (addr !=null) {
+ sw.append(delimitedHostString(addr));
+ sw.append(":");
+ sw.append(String.valueOf(addr.getPort()));
+ }
+ if (electionAddr!=null){
+ sw.append(":");
+ sw.append(String.valueOf(electionAddr.getPort()));
+ }
+ if (type == LearnerType.OBSERVER) sw.append(":observer");
+ else if (type == LearnerType.PARTICIPANT) sw.append(":participant");
+ if (clientAddr!=null){
+ sw.append(";");
+ sw.append(delimitedHostString(clientAddr));
+ sw.append(":");
+ sw.append(String.valueOf(clientAddr.getPort()));
+ }
+ return sw.toString();
+ }
+
+ public int hashCode() {
+ assert false : "hashCode not designed";
+ return 42; // any arbitrary constant will do
+ }
+
+ private boolean checkAddressesEqual(InetSocketAddress addr1, InetSocketAddress addr2){
+ if ((addr1 == null && addr2!=null) ||
+ (addr1!=null && addr2==null) ||
+ (addr1!=null && addr2!=null && !addr1.equals(addr2))) return false;
+ return true;
+ }
+
+ public boolean equals(Object o){
+ if (!(o instanceof QuorumServer)) return false;
+ QuorumServer qs = (QuorumServer)o;
+ if ((qs.id != id) || (qs.type != type)) return false;
+ if (!checkAddressesEqual(addr, qs.addr)) return false;
+ if (!checkAddressesEqual(electionAddr, qs.electionAddr)) return false;
+ if (!checkAddressesEqual(clientAddr, qs.clientAddr)) return false;
+ return true;
+ }
+
+ public void checkAddressDuplicate(QuorumServer s) throws BadArgumentsException {
+ List<InetSocketAddress> otherAddrs = new ArrayList<InetSocketAddress>();
+ otherAddrs.add(s.addr);
+ otherAddrs.add(s.clientAddr);
+ otherAddrs.add(s.electionAddr);
+ otherAddrs = excludedSpecialAddresses(otherAddrs);
+
+ for (InetSocketAddress my: this.myAddrs) {
+
+ for (InetSocketAddress other: otherAddrs) {
+ if (my.equals(other)) {
+ String error = String.format("%s of server.%d conflicts %s of server.%d", my, this.id, other, s.id);
+ throw new BadArgumentsException(error);
+ }
+ }
+ }
+ }
+
+ private List<InetSocketAddress> excludedSpecialAddresses(List<InetSocketAddress> addrs) {
+ List<InetSocketAddress> included = new ArrayList<InetSocketAddress>();
+ InetAddress wcAddr = new InetSocketAddress(0).getAddress();
+
+ for (InetSocketAddress addr : addrs) {
+ if (addr == null) {
+ continue;
+ }
+ InetAddress inetaddr = addr.getAddress();
+
+ if (inetaddr == null ||
+ inetaddr.equals(wcAddr) || // wildCard address(0.0.0.0)
+ inetaddr.isLoopbackAddress()) { // loopback address(localhost/127.0.0.1)
+ continue;
+ }
+ included.add(addr);
+ }
+ return included;
+ }
+ }
+
+
+ public enum ServerState {
+ LOOKING, FOLLOWING, LEADING, OBSERVING;
+ }
+
+ /*
+ * A peer can either be participating, which implies that it is willing to
+ * both vote in instances of consensus and to elect or become a Leader, or
+ * it may be observing in which case it isn't.
+ *
+ * We need this distinction to decide which ServerState to move to when
+ * conditions change (e.g. which state to become after LOOKING).
+ */
+ public enum LearnerType {
+ PARTICIPANT, OBSERVER;
+ }
+
+ /*
+ * To enable observers to have no identifier, we need a generic identifier
+ * at least for QuorumCnxManager. We use the following constant to as the
+ * value of such a generic identifier.
+ */
+
+ static final long OBSERVER_ID = Long.MAX_VALUE;
+
+ /*
+ * Record leader election time
+ */
+ public long start_fle, end_fle; // fle = fast leader election
+ public static final String FLE_TIME_UNIT = "MS";
+
+ /*
+ * Default value of peer is participant
+ */
+ private LearnerType learnerType = LearnerType.PARTICIPANT;
+
+ public LearnerType getLearnerType() {
+ return learnerType;
+ }
+
+ /**
+ * Sets the LearnerType
+ */
+ public void setLearnerType(LearnerType p) {
+ learnerType = p;
+ }
+
+ protected synchronized void setConfigFileName(String s) {
+ configFilename = s;
+ }
+
+ private String configFilename = null;
+
+ public int getQuorumSize(){
+ return getVotingView().size();
+ }
+
+ /**
+ * QuorumVerifier implementation; default (majority).
+ */
+
+ //last committed quorum verifier
+ private QuorumVerifier quorumVerifier;
+
+ //last proposed quorum verifier
+ private QuorumVerifier lastSeenQuorumVerifier = null;
+
+ // Lock object that guard access to quorumVerifier and lastSeenQuorumVerifier.
+ final Object QV_LOCK = new Object();
+
+
+ /**
+ * My id
+ */
+ private long myid;
+
+
+ /**
+ * get the id of this quorum peer.
+ */
+ public long getId() {
+ return myid;
+ }
+
+ // VisibleForTesting
+ void setId(long id) {
+ this.myid = id;
+ }
+
+ private boolean sslQuorum;
+ private boolean shouldUsePortUnification;
+
+ public boolean isSslQuorum() {
+ return sslQuorum;
+ }
+
+ public boolean shouldUsePortUnification() {
+ return shouldUsePortUnification;
+ }
+
+ private final QuorumX509Util x509Util;
+
+ QuorumX509Util getX509Util() {
+ return x509Util;
+ }
+
+ /**
+ * This is who I think the leader currently is.
+ */
+ volatile private Vote currentVote;
+
+ public synchronized Vote getCurrentVote(){
+ return currentVote;
+ }
+
+ public synchronized void setCurrentVote(Vote v){
+ currentVote = v;
+ }
+
+ private volatile boolean running = true;
+
+ /**
+ * The number of milliseconds of each tick
+ */
+ protected int tickTime;
+
+ /**
+ * Whether learners in this quorum should create new sessions as local.
+ * False by default to preserve existing behavior.
+ */
+ protected boolean localSessionsEnabled = false;
+
+ /**
+ * Whether learners in this quorum should upgrade local sessions to
+ * global. Only matters if local sessions are enabled.
+ */
+ protected boolean localSessionsUpgradingEnabled = true;
+
+ /**
+ * Minimum number of milliseconds to allow for session timeout.
+ * A value of -1 indicates unset, use default.
+ */
+ protected int minSessionTimeout = -1;
+
+ /**
+ * Maximum number of milliseconds to allow for session timeout.
+ * A value of -1 indicates unset, use default.
+ */
+ protected int maxSessionTimeout = -1;
+
+ /**
+ * The number of ticks that the initial synchronization phase can take
+ */
+ protected int initLimit;
+
+ /**
+ * The number of ticks that can pass between sending a request and getting
+ * an acknowledgment
+ */
+ protected int syncLimit;
+
+ /**
+ * Enables/Disables sync request processor. This option is enabled
+ * by default and is to be used with observers.
+ */
+ protected boolean syncEnabled = true;
+
+ /**
+ * The current tick
+ */
+ protected AtomicInteger tick = new AtomicInteger();
+
+ /**
+ * Whether or not to listen on all IPs for the two quorum ports
+ * (broadcast and fast leader election).
+ */
+ protected boolean quorumListenOnAllIPs = false;
+
+ /**
+ * Keeps time taken for leader election in milliseconds. Sets the value to
+ * this variable only after the completion of leader election.
+ */
+ private long electionTimeTaken = -1;
+
+ /**
+ * Enable/Disables quorum authentication using sasl. Defaulting to false.
+ */
+ protected boolean quorumSaslEnableAuth;
+
+ /**
+ * If this is false, quorum peer server will accept another quorum peer client
+ * connection even if the authentication did not succeed. This can be used while
+ * upgrading ZooKeeper server. Defaulting to false (required).
+ */
+ protected boolean quorumServerSaslAuthRequired;
+
+ /**
+ * If this is false, quorum peer learner will talk to quorum peer server
+ * without authentication. This can be used while upgrading ZooKeeper
+ * server. Defaulting to false (required).
+ */
+ protected boolean quorumLearnerSaslAuthRequired;
+
+ /**
+ * Kerberos quorum service principal. Defaulting to 'zkquorum/localhost'.
+ */
+ protected String quorumServicePrincipal;
+
+ /**
+ * Quorum learner login context name in jaas-conf file to read the kerberos
+ * security details. Defaulting to 'QuorumLearner'.
+ */
+ protected String quorumLearnerLoginContext;
+
+ /**
+ * Quorum server login context name in jaas-conf file to read the kerberos
+ * security details. Defaulting to 'QuorumServer'.
+ */
+ protected String quorumServerLoginContext;
+
+ // TODO: need to tune the default value of thread size
+ private static final int QUORUM_CNXN_THREADS_SIZE_DEFAULT_VALUE = 20;
+ /**
+ * The maximum number of threads to allow in the connectionExecutors thread
+ * pool which will be used to initiate quorum server connections.
+ */
+ protected int quorumCnxnThreadsSize = QUORUM_CNXN_THREADS_SIZE_DEFAULT_VALUE;
+
+ /**
+ * @deprecated As of release 3.4.0, this class has been deprecated, since
+ * it is used with one of the udp-based versions of leader election, which
+ * we are also deprecating.
+ *
+ * This class simply responds to requests for the current leader of this
+ * node.
+ * <p>
+ * The request contains just an xid generated by the requestor.
+ * <p>
+ * The response has the xid, the id of this server, the id of the leader,
+ * and the zxid of the leader.
+ *
+ *
+ */
+ @Deprecated
+ class ResponderThread extends ZooKeeperThread {
+ ResponderThread() {
+ super("ResponderThread");
+ }
+
+ volatile boolean running = true;
+
+ @Override
+ public void run() {
+ try {
+ byte b[] = new byte[36];
+ ByteBuffer responseBuffer = ByteBuffer.wrap(b);
+ DatagramPacket packet = new DatagramPacket(b, b.length);
+ while (running) {
+ udpSocket.receive(packet);
+ if (packet.getLength() != 4) {
+ LOG.warn("Got more than just an xid! Len = "
+ + packet.getLength());
+ } else {
+ responseBuffer.clear();
+ responseBuffer.getInt(); // Skip the xid
+ responseBuffer.putLong(myid);
+ Vote current = getCurrentVote();
+ switch (getPeerState()) {
+ case LOOKING:
+ responseBuffer.putLong(current.getId());
+ responseBuffer.putLong(current.getZxid());
+ break;
+ case LEADING:
+ responseBuffer.putLong(myid);
+ try {
+ long proposed;
+ synchronized(leader) {
+ proposed = leader.lastProposed;
+ }
+ responseBuffer.putLong(proposed);
+ } catch (NullPointerException npe) {
+ // This can happen in state transitions,
+ // just ignore the request
+ }
+ break;
+ case FOLLOWING:
+ responseBuffer.putLong(current.getId());
+ try {
+ responseBuffer.putLong(follower.getZxid());
+ } catch (NullPointerException npe) {
+ // This can happen in state transitions,
+ // just ignore the request
+ }
+ break;
+ case OBSERVING:
+ // Do nothing, Observers keep themselves to
+ // themselves.
+ break;
+ }
+ packet.setData(b);
+ udpSocket.send(packet);
+ }
+ packet.setLength(b.length);
+ }
+ } catch (RuntimeException e) {
+ LOG.warn("Unexpected runtime exception in ResponderThread",e);
+ } catch (IOException e) {
+ LOG.warn("Unexpected IO exception in ResponderThread",e);
+ } finally {
+ LOG.warn("QuorumPeer responder thread exited");
+ }
+ }
+ }
+
+ private ServerState state = ServerState.LOOKING;
+
+ private boolean reconfigFlag = false; // indicates that a reconfig just committed
+
+ public synchronized void setPeerState(ServerState newState){
+ state=newState;
+ }
+ public synchronized void reconfigFlagSet(){
+ reconfigFlag = true;
+ }
+ public synchronized void reconfigFlagClear(){
+ reconfigFlag = false;
+ }
+ public synchronized boolean isReconfigStateChange(){
+ return reconfigFlag;
+ }
+ public synchronized ServerState getPeerState(){
+ return state;
+ }
+
+ DatagramSocket udpSocket;
+
+ private final AtomicReference<AddressTuple> myAddrs = new AtomicReference<>();
+
+ /**
+ * Resolves hostname for a given server ID.
+ *
+ * This method resolves hostname for a given server ID in both quorumVerifer
+ * and lastSeenQuorumVerifier. If the server ID matches the local server ID,
+ * it also updates myAddrs.
+ */
+ public void recreateSocketAddresses(long id) {
+ QuorumVerifier qv = getQuorumVerifier();
+ if (qv != null) {
+ QuorumServer qs = qv.getAllMembers().get(id);
+ if (qs != null) {
+ qs.recreateSocketAddresses();
+ if (id == getId()) {
+ setAddrs(qs.addr, qs.electionAddr, qs.clientAddr);
+ }
+ }
+ }
+ qv = getLastSeenQuorumVerifier();
+ if (qv != null) {
+ QuorumServer qs = qv.getAllMembers().get(id);
+ if (qs != null) {
+ qs.recreateSocketAddresses();
+ }
+ }
+ }
+
+ private AddressTuple getAddrs(){
+ AddressTuple addrs = myAddrs.get();
+ if (addrs != null) {
+ return addrs;
+ }
+ try {
+ synchronized (QV_LOCK) {
+ addrs = myAddrs.get();
+ while (addrs == null) {
+ QV_LOCK.wait();
+ addrs = myAddrs.get();
+ }
+ return addrs;
+ }
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new RuntimeException(e);
+ }
+ }
+
+ public InetSocketAddress getQuorumAddress(){
+ return getAddrs().quorumAddr;
+ }
+
+ public InetSocketAddress getElectionAddress(){
+ return getAddrs().electionAddr;
+ }
+
+ public InetSocketAddress getClientAddress(){
+ final AddressTuple addrs = myAddrs.get();
+ return (addrs == null) ? null : addrs.clientAddr;
+ }
+
+ private void setAddrs(InetSocketAddress quorumAddr, InetSocketAddress electionAddr, InetSocketAddress clientAddr){
+ synchronized (QV_LOCK) {
+ myAddrs.set(new AddressTuple(quorumAddr, electionAddr, clientAddr));
+ QV_LOCK.notifyAll();
+ }
+ }
+
+ private int electionType;
+
+ Election electionAlg;
+
+ ServerCnxnFactory cnxnFactory;
+ ServerCnxnFactory secureCnxnFactory;
+
+ private FileTxnSnapLog logFactory = null;
+
+ private final QuorumStats quorumStats;
+
+ AdminServer adminServer;
+
+ public static QuorumPeer testingQuorumPeer() throws SaslException {
+ return new QuorumPeer();
+ }
+
+ public QuorumPeer() throws SaslException {
+ super("QuorumPeer");
+ quorumStats = new QuorumStats(this);
+ jmxRemotePeerBean = new HashMap<Long, RemotePeerBean>();
+ adminServer = AdminServerFactory.createAdminServer();
+ x509Util = new QuorumX509Util();
+ initialize();
+ }
+
+ /**
+ * For backward compatibility purposes, we instantiate QuorumMaj by default.
+ */
+
+ public QuorumPeer(Map<Long, QuorumServer> quorumPeers, File dataDir,
+ File dataLogDir, int electionType,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ ServerCnxnFactory cnxnFactory) throws IOException {
+ this(quorumPeers, dataDir, dataLogDir, electionType, myid, tickTime,
+ initLimit, syncLimit, false, cnxnFactory,
+ new QuorumMaj(quorumPeers));
+ }
+
+ public QuorumPeer(Map<Long, QuorumServer> quorumPeers, File dataDir,
+ File dataLogDir, int electionType,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ boolean quorumListenOnAllIPs,
+ ServerCnxnFactory cnxnFactory,
+ QuorumVerifier quorumConfig) throws IOException {
+ this();
+ this.cnxnFactory = cnxnFactory;
+ this.electionType = electionType;
+ this.myid = myid;
+ this.tickTime = tickTime;
+ this.initLimit = initLimit;
+ this.syncLimit = syncLimit;
+ this.quorumListenOnAllIPs = quorumListenOnAllIPs;
+ this.logFactory = new FileTxnSnapLog(dataLogDir, dataDir);
+ this.zkDb = new ZKDatabase(this.logFactory);
+ if(quorumConfig == null) quorumConfig = new QuorumMaj(quorumPeers);
+ setQuorumVerifier(quorumConfig, false);
+ adminServer = AdminServerFactory.createAdminServer();
+ }
+
+ public void initialize() throws SaslException {
+ // init quorum auth server & learner
+ if (isQuorumSaslAuthEnabled()) {
+ Set<String> authzHosts = new HashSet<String>();
+ for (QuorumServer qs : getView().values()) {
+ authzHosts.add(qs.hostname);
+ }
+ authServer = new SaslQuorumAuthServer(isQuorumServerSaslAuthRequired(),
+ quorumServerLoginContext, authzHosts);
+ authLearner = new SaslQuorumAuthLearner(isQuorumLearnerSaslAuthRequired(),
+ quorumServicePrincipal, quorumLearnerLoginContext);
+ } else {
+ authServer = new NullQuorumAuthServer();
+ authLearner = new NullQuorumAuthLearner();
+ }
+ }
+
+ QuorumStats quorumStats() {
+ return quorumStats;
+ }
+
+ @Override
+ public synchronized void start() {
+ if (!getView().containsKey(myid)) {
+ throw new RuntimeException("My id " + myid + " not in the peer list");
+ }
+ loadDataBase();
+ startServerCnxnFactory();
+ try {
+ adminServer.start();
+ } catch (AdminServerException e) {
+ LOG.warn("Problem starting AdminServer", e);
+ System.out.println(e);
+ }
+ startLeaderElection();
+ super.start();
+ }
+
+ private void loadDataBase() {
+ try {
+ zkDb.loadDataBase();
+
+ // load the epochs
+ long lastProcessedZxid = zkDb.getDataTree().lastProcessedZxid;
+ long epochOfZxid = ZxidUtils.getEpochFromZxid(lastProcessedZxid);
+ try {
+ currentEpoch = readLongFromFile(CURRENT_EPOCH_FILENAME);
+ } catch(FileNotFoundException e) {
+ // pick a reasonable epoch number
+ // this should only happen once when moving to a
+ // new code version
+ currentEpoch = epochOfZxid;
+ LOG.info(CURRENT_EPOCH_FILENAME
+ + " not found! Creating with a reasonable default of {}. This should only happen when you are upgrading your installation",
+ currentEpoch);
+ writeLongToFile(CURRENT_EPOCH_FILENAME, currentEpoch);
+ }
+ if (epochOfZxid > currentEpoch) {
+ throw new IOException("The current epoch, " + ZxidUtils.zxidToString(currentEpoch) + ", is older than the last zxid, " + lastProcessedZxid);
+ }
+ try {
+ acceptedEpoch = readLongFromFile(ACCEPTED_EPOCH_FILENAME);
+ } catch(FileNotFoundException e) {
+ // pick a reasonable epoch number
+ // this should only happen once when moving to a
+ // new code version
+ acceptedEpoch = epochOfZxid;
+ LOG.info(ACCEPTED_EPOCH_FILENAME
+ + " not found! Creating with a reasonable default of {}. This should only happen when you are upgrading your installation",
+ acceptedEpoch);
+ writeLongToFile(ACCEPTED_EPOCH_FILENAME, acceptedEpoch);
+ }
+ if (acceptedEpoch < currentEpoch) {
+ throw new IOException("The accepted epoch, " + ZxidUtils.zxidToString(acceptedEpoch) + " is less than the current epoch, " + ZxidUtils.zxidToString(currentEpoch));
+ }
+ } catch(IOException ie) {
+ LOG.error("Unable to load database on disk", ie);
+ throw new RuntimeException("Unable to run quorum server ", ie);
+ }
+ }
+
+ ResponderThread responder;
+
+ synchronized public void stopLeaderElection() {
+ responder.running = false;
+ responder.interrupt();
+ }
+ synchronized public void startLeaderElection() {
+ try {
+ if (getPeerState() == ServerState.LOOKING) {
+ currentVote = new Vote(myid, getLastLoggedZxid(), getCurrentEpoch());
+ }
+ } catch(IOException e) {
+ RuntimeException re = new RuntimeException(e.getMessage());
+ re.setStackTrace(e.getStackTrace());
+ throw re;
+ }
+
+ // if (!getView().containsKey(myid)) {
+ // throw new RuntimeException("My id " + myid + " not in the peer list");
+ //}
+ if (electionType == 0) {
+ try {
+ udpSocket = new DatagramSocket(getQuorumAddress().getPort());
+ responder = new ResponderThread();
+ responder.start();
+ } catch (SocketException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ this.electionAlg = createElectionAlgorithm(electionType);
+ }
+
+ /**
+ * Count the number of nodes in the map that could be followers.
+ * @param peers
+ * @return The number of followers in the map
+ */
+ protected static int countParticipants(Map<Long,QuorumServer> peers) {
+ int count = 0;
+ for (QuorumServer q : peers.values()) {
+ if (q.type == LearnerType.PARTICIPANT) {
+ count++;
+ }
+ }
+ return count;
+ }
+
+ /**
+
+ * This constructor is only used by the existing unit test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public QuorumPeer(Map<Long,QuorumServer> quorumPeers, File snapDir,
+ File logDir, int clientPort, int electionAlg,
+ long myid, int tickTime, int initLimit, int syncLimit)
+ throws IOException
+ {
+ this(quorumPeers, snapDir, logDir, electionAlg, myid, tickTime, initLimit, syncLimit, false,
+ ServerCnxnFactory.createFactory(getClientAddress(quorumPeers, myid, clientPort), -1),
+ new QuorumMaj(quorumPeers));
+ }
+
+ /**
+ * This constructor is only used by the existing unit test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public QuorumPeer(Map<Long,QuorumServer> quorumPeers, File snapDir,
+ File logDir, int clientPort, int electionAlg,
+ long myid, int tickTime, int initLimit, int syncLimit,
+ QuorumVerifier quorumConfig)
+ throws IOException
+ {
+ this(quorumPeers, snapDir, logDir, electionAlg,
+ myid,tickTime, initLimit,syncLimit, false,
+ ServerCnxnFactory.createFactory(getClientAddress(quorumPeers, myid, clientPort), -1),
+ quorumConfig);
+ }
+
+ private static InetSocketAddress getClientAddress(Map<Long, QuorumServer> quorumPeers, long myid, int clientPort)
+ throws IOException {
+ QuorumServer quorumServer = quorumPeers.get(myid);
+ if (null == quorumServer) {
+ throw new IOException("No QuorumServer correspoding to myid " + myid);
+ }
+ if (null == quorumServer.clientAddr) {
+ return new InetSocketAddress(clientPort);
+ }
+ if (quorumServer.clientAddr.getPort() != clientPort) {
+ throw new IOException("QuorumServer port " + quorumServer.clientAddr.getPort()
+ + " does not match with given port " + clientPort);
+ }
+ return quorumServer.clientAddr;
+ }
+
+ /**
+ * returns the highest zxid that this host has seen
+ *
+ * @return the highest zxid for this host
+ */
+ public long getLastLoggedZxid() {
+ if (!zkDb.isInitialized()) {
+ loadDataBase();
+ }
+ return zkDb.getDataTreeLastProcessedZxid();
+ }
+
+ public Follower follower;
+ public Leader leader;
+ public Observer observer;
+
+ protected Follower makeFollower(FileTxnSnapLog logFactory) throws IOException {
+ return new Follower(this, new FollowerZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ protected Leader makeLeader(FileTxnSnapLog logFactory) throws IOException, X509Exception {
+ return new Leader(this, new LeaderZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ protected Observer makeObserver(FileTxnSnapLog logFactory) throws IOException {
+ return new Observer(this, new ObserverZooKeeperServer(logFactory, this, this.zkDb));
+ }
+
+ @SuppressWarnings("deprecation")
+ protected Election createElectionAlgorithm(int electionAlgorithm){
+ Election le=null;
+
+ //TODO: use a factory rather than a switch
+ switch (electionAlgorithm) {
+ case 0:
+ le = new LeaderElection(this);
+ break;
+ case 1:
+ le = new AuthFastLeaderElection(this);
+ break;
+ case 2:
+ le = new AuthFastLeaderElection(this, true);
+ break;
+ case 3:
+ QuorumCnxManager qcm = createCnxnManager();
+ QuorumCnxManager oldQcm = qcmRef.getAndSet(qcm);
+ if (oldQcm != null) {
+ LOG.warn("Clobbering already-set QuorumCnxManager (restarting leader election?)");
+ oldQcm.halt();
+ }
+ QuorumCnxManager.Listener listener = qcm.listener;
+ if(listener != null){
+ listener.start();
+ FastLeaderElection fle = new FastLeaderElection(this, qcm);
+ fle.start();
+ le = fle;
+ } else {
+ LOG.error("Null listener when initializing cnx manager");
+ }
+ break;
+ default:
+ assert false;
+ }
+ return le;
+ }
+
+ @SuppressWarnings("deprecation")
+ protected Election makeLEStrategy(){
+ LOG.debug("Initializing leader election protocol...");
+ if (getElectionType() == 0) {
+ electionAlg = new LeaderElection(this);
+ }
+ return electionAlg;
+ }
+
+ synchronized protected void setLeader(Leader newLeader){
+ leader=newLeader;
+ }
+
+ synchronized protected void setFollower(Follower newFollower){
+ follower=newFollower;
+ }
+
+ synchronized protected void setObserver(Observer newObserver){
+ observer=newObserver;
+ }
+
+ synchronized public ZooKeeperServer getActiveServer(){
+ if(leader!=null)
+ return leader.zk;
+ else if(follower!=null)
+ return follower.zk;
+ else if (observer != null)
+ return observer.zk;
+ return null;
+ }
+
+ boolean shuttingDownLE = false;
+
+ @Override
+ public void run() {
+ updateThreadName();
+
+ LOG.debug("Starting quorum peer");
+ try {
+ jmxQuorumBean = new QuorumBean(this);
+ MBeanRegistry.getInstance().register(jmxQuorumBean, null);
+ for(QuorumServer s: getView().values()){
+ ZKMBeanInfo p;
+ if (getId() == s.id) {
+ p = jmxLocalPeerBean = new LocalPeerBean(this);
+ try {
+ MBeanRegistry.getInstance().register(p, jmxQuorumBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxLocalPeerBean = null;
+ }
+ } else {
+ RemotePeerBean rBean = new RemotePeerBean(this, s);
+ try {
+ MBeanRegistry.getInstance().register(rBean, jmxQuorumBean);
+ jmxRemotePeerBean.put(s.id, rBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxQuorumBean = null;
+ }
+
+ try {
+ /*
+ * Main loop
+ */
+ while (running) {
+ switch (getPeerState()) {
+ case LOOKING:
+ LOG.info("LOOKING");
+
+ if (Boolean.getBoolean("readonlymode.enabled")) {
+ LOG.info("Attempting to start ReadOnlyZooKeeperServer");
+
+ // Create read-only server but don't start it immediately
+ final ReadOnlyZooKeeperServer roZk =
+ new ReadOnlyZooKeeperServer(logFactory, this, this.zkDb);
+
+ // Instead of starting roZk immediately, wait some grace
+ // period before we decide we're partitioned.
+ //
+ // Thread is used here because otherwise it would require
+ // changes in each of election strategy classes which is
+ // unnecessary code coupling.
+ Thread roZkMgr = new Thread() {
+ public void run() {
+ try {
+ // lower-bound grace period to 2 secs
+ sleep(Math.max(2000, tickTime));
+ if (ServerState.LOOKING.equals(getPeerState())) {
+ roZk.startup();
+ }
+ } catch (InterruptedException e) {
+ LOG.info("Interrupted while attempting to start ReadOnlyZooKeeperServer, not started");
+ } catch (Exception e) {
+ LOG.error("FAILED to start ReadOnlyZooKeeperServer", e);
+ }
+ }
+ };
+ try {
+ roZkMgr.start();
+ reconfigFlagClear();
+ if (shuttingDownLE) {
+ shuttingDownLE = false;
+ startLeaderElection();
+ }
+ setCurrentVote(makeLEStrategy().lookForLeader());
Review comment:
*THREAD_SAFETY_VIOLATION:* Unprotected write. Non-private method `QuorumPeer.run()` indirectly writes to field `this.electionAlg` outside of synchronization.
Reporting because another access to the same memory occurs on a background thread, although this access may not.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/resources/lastRevision.sh
##########
@@ -0,0 +1,21 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
*ShellCheck:* Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] maoling commented on pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
maoling commented on pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695#issuecomment-841764845
@EagerToLeave I close this PR, you can reopen it if I make a wrong judgment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] maoling closed pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
maoling closed pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] muse-dev[bot] commented on a change in pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
muse-dev[bot] commented on a change in pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695#discussion_r630277728
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java
##########
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.EOFException;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.text.DateFormat;
+import java.util.Date;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.zookeeper.server.persistence.FileHeader;
+import org.apache.zookeeper.server.persistence.FileTxnLog;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+
+@InterfaceAudience.Public
+public class LogFormatter {
+ private static final Logger LOG = LoggerFactory.getLogger(LogFormatter.class);
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: LogFormatter log_file");
+ System.exit(2);
+ }
+ FileInputStream fis = new FileInputStream(args[0]);
Review comment:
*PATH_TRAVERSAL_IN:* This API (java/io/FileInputStream.<init>(Ljava/lang/String;)V) reads a file whose location might be specified by user input [(details)](https://find-sec-bugs.github.io/bugs.htm#PATH_TRAVERSAL_IN)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
##########
@@ -0,0 +1,128 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+/**
+ * Dump a snapshot file to stdout.
+ */
+@InterfaceAudience.Public
+public class SnapshotFormatter {
+
+ /**
+ * USAGE: SnapshotFormatter snapshot_file
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: SnapshotFormatter snapshot_file");
+ System.exit(2);
+ }
+
+ new SnapshotFormatter().run(args[0]);
+ }
+
+ public void run(String snapshotFileName) throws IOException {
+ InputStream is = new CheckedInputStream(
Review comment:
*PATH_TRAVERSAL_IN:* This API (java/io/FileInputStream.<init>(Ljava/lang/String;)V) reads a file whose location might be specified by user input [(details)](https://find-sec-bugs.github.io/bugs.htm#PATH_TRAVERSAL_IN)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxn.java
##########
@@ -0,0 +1,1665 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+import java.io.BufferedReader;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.net.ConnectException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.net.SocketAddress;
+import java.net.SocketException;
+import java.nio.ByteBuffer;
+import java.util.HashSet;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.Map.Entry;
+import java.util.concurrent.CopyOnWriteArraySet;
+import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.LinkedBlockingQueue;
+
+import javax.security.auth.login.LoginException;
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.Event.KeeperState;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.ZooKeeper.States;
+import org.apache.zookeeper.ZooKeeper.WatchRegistration;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.ExistsResponse;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SetWatches;
+import org.apache.zookeeper.proto.WatcherEvent;
+import org.apache.zookeeper.server.ByteBufferInputStream;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.ZooTrace;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.MDC;
+
+/**
+ * This class manages the socket i/o for the client. ClientCnxn maintains a list
+ * of available servers to connect to and "transparently" switches servers it is
+ * connected to as needed.
+ *
+ */
+@SuppressFBWarnings({"EI_EXPOSE_REP", "EI_EXPOSE_REP2"})
+public class ClientCnxn {
+ private static final Logger LOG = LoggerFactory.getLogger(ClientCnxn.class);
+
+ /* ZOOKEEPER-706: If a session has a large number of watches set then
+ * attempting to re-establish those watches after a connection loss may
+ * fail due to the SetWatches request exceeding the server's configured
+ * jute.maxBuffer value. To avoid this we instead split the watch
+ * re-establishement across multiple SetWatches calls. This constant
+ * controls the size of each call. It is set to 128kB to be conservative
+ * with respect to the server's 1MB default for jute.maxBuffer.
+ */
+ private static final int SET_WATCHES_MAX_LENGTH = 128 * 1024;
+
+ static class AuthData {
+ AuthData(String scheme, byte data[]) {
+ this.scheme = scheme;
+ this.data = data;
+ }
+
+ String scheme;
+
+ byte data[];
+ }
+
+ private final CopyOnWriteArraySet<AuthData> authInfo = new CopyOnWriteArraySet<AuthData>();
+
+ /**
+ * These are the packets that have been sent and are waiting for a response.
+ */
+ private final LinkedList<Packet> pendingQueue = new LinkedList<Packet>();
+
+ /**
+ * These are the packets that need to be sent.
+ */
+ private final LinkedBlockingDeque<Packet> outgoingQueue = new LinkedBlockingDeque<Packet>();
+
+ private int connectTimeout;
+
+ /**
+ * The timeout in ms the client negotiated with the server. This is the
+ * "real" timeout, not the timeout request by the client (which may have
+ * been increased/decreased by the server which applies bounds to this
+ * value.
+ */
+ private volatile int negotiatedSessionTimeout;
+
+ private int readTimeout;
+
+ private final int sessionTimeout;
+
+ private final ZooKeeper zooKeeper;
+
+ private final ClientWatchManager watcher;
+
+ private long sessionId;
+
+ private byte sessionPasswd[] = new byte[16];
+
+ /**
+ * If true, the connection is allowed to go to r-o mode. This field's value
+ * is sent, besides other data, during session creation handshake. If the
+ * server on the other side of the wire is partitioned it'll accept
+ * read-only clients only.
+ */
+ private boolean readOnly;
+
+ final String chrootPath;
+
+ final SendThread sendThread;
+
+ final EventThread eventThread;
+
+ /**
+ * Set to true when close is called. Latches the connection such that we
+ * don't attempt to re-connect to the server if in the middle of closing the
+ * connection (client sends session disconnect to server as part of close
+ * operation)
+ */
+ private volatile boolean closing = false;
+
+ /**
+ * A set of ZooKeeper hosts this client could connect to.
+ */
+ private final HostProvider hostProvider;
+
+ /**
+ * Is set to true when a connection to a r/w server is established for the
+ * first time; never changed afterwards.
+ * <p>
+ * Is used to handle situations when client without sessionId connects to a
+ * read-only server. Such client receives "fake" sessionId from read-only
+ * server, but this sessionId is invalid for other servers. So when such
+ * client finds a r/w server, it sends 0 instead of fake sessionId during
+ * connection handshake and establishes new, valid session.
+ * <p>
+ * If this field is false (which implies we haven't seen r/w server before)
+ * then non-zero sessionId is fake, otherwise it is valid.
+ */
+ volatile boolean seenRwServerBefore = false;
+
+
+ public ZooKeeperSaslClient zooKeeperSaslClient;
+
+ private final ZKClientConfig clientConfig;
+ /**
+ * If any request's response in not received in configured requestTimeout
+ * then it is assumed that the response packet is lost.
+ */
+ private long requestTimeout;
+
+ public long getSessionId() {
+ return sessionId;
+ }
+
+ public byte[] getSessionPasswd() {
+ return sessionPasswd;
+ }
+
+ public int getSessionTimeout() {
+ return negotiatedSessionTimeout;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+
+ SocketAddress local = sendThread.getClientCnxnSocket().getLocalSocketAddress();
+ SocketAddress remote = sendThread.getClientCnxnSocket().getRemoteSocketAddress();
+ sb
+ .append("sessionid:0x").append(Long.toHexString(getSessionId()))
+ .append(" local:").append(local)
+ .append(" remoteserver:").append(remote)
+ .append(" lastZxid:").append(lastZxid)
+ .append(" xid:").append(xid)
+ .append(" sent:").append(sendThread.getClientCnxnSocket().getSentCount())
+ .append(" recv:").append(sendThread.getClientCnxnSocket().getRecvCount())
+ .append(" queuedpkts:").append(outgoingQueue.size())
+ .append(" pendingresp:").append(pendingQueue.size())
+ .append(" queuedevents:").append(eventThread.waitingEvents.size());
+
+ return sb.toString();
+ }
+
+ /**
+ * This class allows us to pass the headers and the relevant records around.
+ */
+ static class Packet {
+ RequestHeader requestHeader;
+
+ ReplyHeader replyHeader;
+
+ Record request;
+
+ Record response;
+
+ ByteBuffer bb;
+
+ /** Client's view of the path (may differ due to chroot) **/
+ String clientPath;
+ /** Servers's view of the path (may differ due to chroot) **/
+ String serverPath;
+
+ boolean finished;
+
+ AsyncCallback cb;
+
+ Object ctx;
+
+ WatchRegistration watchRegistration;
+
+ public boolean readOnly;
+
+ WatchDeregistration watchDeregistration;
+
+ /** Convenience ctor */
+ Packet(RequestHeader requestHeader, ReplyHeader replyHeader,
+ Record request, Record response,
+ WatchRegistration watchRegistration) {
+ this(requestHeader, replyHeader, request, response,
+ watchRegistration, false);
+ }
+
+ Packet(RequestHeader requestHeader, ReplyHeader replyHeader,
+ Record request, Record response,
+ WatchRegistration watchRegistration, boolean readOnly) {
+
+ this.requestHeader = requestHeader;
+ this.replyHeader = replyHeader;
+ this.request = request;
+ this.response = response;
+ this.readOnly = readOnly;
+ this.watchRegistration = watchRegistration;
+ }
+
+ public void createBB() {
+ try {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive boa = BinaryOutputArchive.getArchive(baos);
+ boa.writeInt(-1, "len"); // We'll fill this in later
+ if (requestHeader != null) {
+ requestHeader.serialize(boa, "header");
+ }
+ if (request instanceof ConnectRequest) {
+ request.serialize(boa, "connect");
+ // append "am-I-allowed-to-be-readonly" flag
+ boa.writeBool(readOnly, "readOnly");
+ } else if (request != null) {
+ request.serialize(boa, "request");
+ }
+ baos.close();
+ this.bb = ByteBuffer.wrap(baos.toByteArray());
+ this.bb.putInt(this.bb.capacity() - 4);
+ this.bb.rewind();
+ } catch (IOException e) {
+ LOG.warn("Ignoring unexpected exception", e);
+ }
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+
+ sb.append("clientPath:" + clientPath);
+ sb.append(" serverPath:" + serverPath);
+ sb.append(" finished:" + finished);
+
+ sb.append(" header:: " + requestHeader);
+ sb.append(" replyHeader:: " + replyHeader);
+ sb.append(" request:: " + request);
+ sb.append(" response:: " + response);
+
+ // jute toString is horrible, remove unnecessary newlines
+ return sb.toString().replaceAll("\r*\n+", " ");
+ }
+ }
+
+ /**
+ * Creates a connection object. The actual network connect doesn't get
+ * established until needed. The start() instance method must be called
+ * subsequent to construction.
+ *
+ * @param chrootPath - the chroot of this client. Should be removed from this Class in ZOOKEEPER-838
+ * @param hostProvider
+ * the list of ZooKeeper servers to connect to
+ * @param sessionTimeout
+ * the timeout for connections.
+ * @param zooKeeper
+ * the zookeeper object that this connection is related to.
+ * @param watcher watcher for this connection
+ * @param clientCnxnSocket
+ * the socket implementation used (e.g. NIO/Netty)
+ * @param canBeReadOnly
+ * whether the connection is allowed to go to read-only
+ * mode in case of partitioning
+ * @throws IOException
+ */
+ public ClientCnxn(String chrootPath, HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket, boolean canBeReadOnly)
+ throws IOException {
+ this(chrootPath, hostProvider, sessionTimeout, zooKeeper, watcher,
+ clientCnxnSocket, 0, new byte[16], canBeReadOnly);
+ }
+
+ /**
+ * Creates a connection object. The actual network connect doesn't get
+ * established until needed. The start() instance method must be called
+ * subsequent to construction.
+ *
+ * @param chrootPath - the chroot of this client. Should be removed from this Class in ZOOKEEPER-838
+ * @param hostProvider
+ * the list of ZooKeeper servers to connect to
+ * @param sessionTimeout
+ * the timeout for connections.
+ * @param zooKeeper
+ * the zookeeper object that this connection is related to.
+ * @param watcher watcher for this connection
+ * @param clientCnxnSocket
+ * the socket implementation used (e.g. NIO/Netty)
+ * @param sessionId session id if re-establishing session
+ * @param sessionPasswd session passwd if re-establishing session
+ * @param canBeReadOnly
+ * whether the connection is allowed to go to read-only
+ * mode in case of partitioning
+ * @throws IOException
+ */
+ public ClientCnxn(String chrootPath, HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly) {
+ this.zooKeeper = zooKeeper;
+ this.watcher = watcher;
+ this.sessionId = sessionId;
+ this.sessionPasswd = sessionPasswd;
+ this.sessionTimeout = sessionTimeout;
+ this.hostProvider = hostProvider;
+ this.chrootPath = chrootPath;
+
+ connectTimeout = sessionTimeout / hostProvider.size();
+ readTimeout = sessionTimeout * 2 / 3;
+ readOnly = canBeReadOnly;
+
+ sendThread = new SendThread(clientCnxnSocket);
+ eventThread = new EventThread();
+ this.clientConfig=zooKeeper.getClientConfig();
+ initRequestTimeout();
+ }
+
+ public void start() {
+ sendThread.start();
+ eventThread.start();
+ }
+
+ private Object eventOfDeath = new Object();
+
+ private static class WatcherSetEventPair {
+ private final Set<Watcher> watchers;
+ private final WatchedEvent event;
+
+ public WatcherSetEventPair(Set<Watcher> watchers, WatchedEvent event) {
+ this.watchers = watchers;
+ this.event = event;
+ }
+ }
+
+ /**
+ * Guard against creating "-EventThread-EventThread-EventThread-..." thread
+ * names when ZooKeeper object is being created from within a watcher.
+ * See ZOOKEEPER-795 for details.
+ */
+ private static String makeThreadName(String suffix) {
+ String name = Thread.currentThread().getName().
+ replaceAll("-EventThread", "");
+ return name + suffix;
+ }
+
+ class EventThread extends ZooKeeperThread {
+ private final LinkedBlockingQueue<Object> waitingEvents =
+ new LinkedBlockingQueue<Object>();
+
+ /** This is really the queued session state until the event
+ * thread actually processes the event and hands it to the watcher.
+ * But for all intents and purposes this is the state.
+ */
+ private volatile KeeperState sessionState = KeeperState.Disconnected;
+
+ private volatile boolean wasKilled = false;
+ private volatile boolean isRunning = false;
+
+ EventThread() {
+ super(makeThreadName("-EventThread"));
+ setDaemon(true);
+ }
+
+ public void queueEvent(WatchedEvent event) {
+ queueEvent(event, null);
+ }
+
+ private void queueEvent(WatchedEvent event,
+ Set<Watcher> materializedWatchers) {
+ if (event.getType() == EventType.None
+ && sessionState == event.getState()) {
+ return;
+ }
+ sessionState = event.getState();
+ final Set<Watcher> watchers;
+ if (materializedWatchers == null) {
+ // materialize the watchers based on the event
+ watchers = watcher.materialize(event.getState(),
+ event.getType(), event.getPath());
+ } else {
+ watchers = new HashSet<Watcher>();
+ watchers.addAll(materializedWatchers);
+ }
+ WatcherSetEventPair pair = new WatcherSetEventPair(watchers, event);
+ // queue the pair (watch set & event) for later processing
+ waitingEvents.add(pair);
+ }
+
+ public void queueCallback(AsyncCallback cb, int rc, String path,
+ Object ctx) {
+ waitingEvents.add(new LocalCallback(cb, rc, path, ctx));
+ }
+
+ @SuppressFBWarnings("JLM_JSR166_UTILCONCURRENT_MONITORENTER")
+ public void queuePacket(Packet packet) {
+ if (wasKilled) {
+ synchronized (waitingEvents) {
+ if (isRunning) waitingEvents.add(packet);
+ else processEvent(packet);
+ }
+ } else {
+ waitingEvents.add(packet);
+ }
+ }
+
+ public void queueEventOfDeath() {
+ waitingEvents.add(eventOfDeath);
+ }
+
+ @Override
+ @SuppressFBWarnings("JLM_JSR166_UTILCONCURRENT_MONITORENTER")
+ public void run() {
+ try {
+ isRunning = true;
+ while (true) {
+ Object event = waitingEvents.take();
+ if (event == eventOfDeath) {
+ wasKilled = true;
+ } else {
+ processEvent(event);
+ }
+ if (wasKilled)
+ synchronized (waitingEvents) {
+ if (waitingEvents.isEmpty()) {
+ isRunning = false;
+ break;
+ }
+ }
+ }
+ } catch (InterruptedException e) {
+ LOG.error("Event thread exiting due to interruption", e);
+ }
+
+ LOG.info("EventThread shut down for session: 0x{}",
+ Long.toHexString(getSessionId()));
+ }
+
+ private void processEvent(Object event) {
+ try {
+ if (event instanceof WatcherSetEventPair) {
+ // each watcher will process the event
+ WatcherSetEventPair pair = (WatcherSetEventPair) event;
+ for (Watcher watcher : pair.watchers) {
+ try {
+ watcher.process(pair.event);
+ } catch (Throwable t) {
+ LOG.error("Error while calling watcher ", t);
+ }
+ }
+ } else if (event instanceof LocalCallback) {
+ LocalCallback lcb = (LocalCallback) event;
+ if (lcb.cb instanceof StatCallback) {
+ ((StatCallback) lcb.cb).processResult(lcb.rc, lcb.path,
+ lcb.ctx, null);
+ } else if (lcb.cb instanceof DataCallback) {
+ ((DataCallback) lcb.cb).processResult(lcb.rc, lcb.path,
+ lcb.ctx, null, null);
+ } else if (lcb.cb instanceof ACLCallback) {
+ ((ACLCallback) lcb.cb).processResult(lcb.rc, lcb.path,
+ lcb.ctx, null, null);
+ } else if (lcb.cb instanceof ChildrenCallback) {
+ ((ChildrenCallback) lcb.cb).processResult(lcb.rc,
+ lcb.path, lcb.ctx, null);
+ } else if (lcb.cb instanceof Children2Callback) {
+ ((Children2Callback) lcb.cb).processResult(lcb.rc,
+ lcb.path, lcb.ctx, null, null);
+ } else if (lcb.cb instanceof StringCallback) {
+ ((StringCallback) lcb.cb).processResult(lcb.rc,
+ lcb.path, lcb.ctx, null);
+ } else {
+ ((VoidCallback) lcb.cb).processResult(lcb.rc, lcb.path,
+ lcb.ctx);
+ }
+ } else {
+ Packet p = (Packet) event;
+ int rc = 0;
+ String clientPath = p.clientPath;
+ if (p.replyHeader.getErr() != 0) {
+ rc = p.replyHeader.getErr();
+ }
+ if (p.cb == null) {
+ LOG.warn("Somehow a null cb got to EventThread!");
+ } else if (p.response instanceof ExistsResponse
+ || p.response instanceof SetDataResponse
+ || p.response instanceof SetACLResponse) {
+ StatCallback cb = (StatCallback) p.cb;
+ if (rc == 0) {
+ if (p.response instanceof ExistsResponse) {
+ cb.processResult(rc, clientPath, p.ctx,
+ ((ExistsResponse) p.response)
+ .getStat());
+ } else if (p.response instanceof SetDataResponse) {
+ cb.processResult(rc, clientPath, p.ctx,
+ ((SetDataResponse) p.response)
+ .getStat());
+ } else if (p.response instanceof SetACLResponse) {
+ cb.processResult(rc, clientPath, p.ctx,
+ ((SetACLResponse) p.response)
+ .getStat());
+ }
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null);
+ }
+ } else if (p.response instanceof GetDataResponse) {
+ DataCallback cb = (DataCallback) p.cb;
+ GetDataResponse rsp = (GetDataResponse) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx, rsp
+ .getData(), rsp.getStat());
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null,
+ null);
+ }
+ } else if (p.response instanceof GetACLResponse) {
+ ACLCallback cb = (ACLCallback) p.cb;
+ GetACLResponse rsp = (GetACLResponse) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx, rsp
+ .getAcl(), rsp.getStat());
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null,
+ null);
+ }
+ } else if (p.response instanceof GetChildrenResponse) {
+ ChildrenCallback cb = (ChildrenCallback) p.cb;
+ GetChildrenResponse rsp = (GetChildrenResponse) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx, rsp
+ .getChildren());
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null);
+ }
+ } else if (p.response instanceof GetChildren2Response) {
+ Children2Callback cb = (Children2Callback) p.cb;
+ GetChildren2Response rsp = (GetChildren2Response) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx, rsp
+ .getChildren(), rsp.getStat());
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null, null);
+ }
+ } else if (p.response instanceof CreateResponse) {
+ StringCallback cb = (StringCallback) p.cb;
+ CreateResponse rsp = (CreateResponse) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx,
+ (chrootPath == null
+ ? rsp.getPath()
+ : rsp.getPath()
+ .substring(chrootPath.length())));
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null);
+ }
+ } else if (p.response instanceof Create2Response) {
+ Create2Callback cb = (Create2Callback) p.cb;
+ Create2Response rsp = (Create2Response) p.response;
+ if (rc == 0) {
+ cb.processResult(rc, clientPath, p.ctx,
+ (chrootPath == null
+ ? rsp.getPath()
+ : rsp.getPath()
+ .substring(chrootPath.length())), rsp.getStat());
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null, null);
+ }
+ } else if (p.response instanceof MultiResponse) {
+ MultiCallback cb = (MultiCallback) p.cb;
+ MultiResponse rsp = (MultiResponse) p.response;
+ if (rc == 0) {
+ List<OpResult> results = rsp.getResultList();
+ int newRc = rc;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult
+ && KeeperException.Code.OK.intValue() != (newRc = ((ErrorResult) result)
+ .getErr())) {
+ break;
+ }
+ }
+ cb.processResult(newRc, clientPath, p.ctx, results);
+ } else {
+ cb.processResult(rc, clientPath, p.ctx, null);
+ }
+ } else if (p.cb instanceof VoidCallback) {
+ VoidCallback cb = (VoidCallback) p.cb;
+ cb.processResult(rc, clientPath, p.ctx);
+ }
+ }
+ } catch (Throwable t) {
+ LOG.error("Caught unexpected throwable", t);
+ }
+ }
+ }
+
+ // @VisibleForTesting
+ protected void finishPacket(Packet p) {
+ int err = p.replyHeader.getErr();
+ if (p.watchRegistration != null) {
+ p.watchRegistration.register(err);
+ }
+ // Add all the removed watch events to the event queue, so that the
+ // clients will be notified with 'Data/Child WatchRemoved' event type.
+ if (p.watchDeregistration != null) {
+ Map<EventType, Set<Watcher>> materializedWatchers = null;
+ try {
+ materializedWatchers = p.watchDeregistration.unregister(err);
+ for (Entry<EventType, Set<Watcher>> entry : materializedWatchers
+ .entrySet()) {
+ Set<Watcher> watchers = entry.getValue();
+ if (watchers.size() > 0) {
+ queueEvent(p.watchDeregistration.getClientPath(), err,
+ watchers, entry.getKey());
+ // ignore connectionloss when removing from local
+ // session
+ p.replyHeader.setErr(Code.OK.intValue());
+ }
+ }
+ } catch (KeeperException.NoWatcherException nwe) {
+ p.replyHeader.setErr(nwe.code().intValue());
+ } catch (KeeperException ke) {
+ p.replyHeader.setErr(ke.code().intValue());
+ }
+ }
+
+ if (p.cb == null) {
+ synchronized (p) {
+ p.finished = true;
+ p.notifyAll();
+ }
+ } else {
+ p.finished = true;
+ eventThread.queuePacket(p);
+ }
+ }
+
+ void queueEvent(String clientPath, int err,
+ Set<Watcher> materializedWatchers, EventType eventType) {
+ KeeperState sessionState = KeeperState.SyncConnected;
+ if (KeeperException.Code.SESSIONEXPIRED.intValue() == err
+ || KeeperException.Code.CONNECTIONLOSS.intValue() == err) {
+ sessionState = Event.KeeperState.Disconnected;
+ }
+ WatchedEvent event = new WatchedEvent(eventType, sessionState,
+ clientPath);
+ eventThread.queueEvent(event, materializedWatchers);
+ }
+
+ void queueCallback(AsyncCallback cb, int rc, String path, Object ctx) {
+ eventThread.queueCallback(cb, rc, path, ctx);
+ }
+
+ private void conLossPacket(Packet p) {
+ if (p.replyHeader == null) {
+ return;
+ }
+ switch (state) {
+ case AUTH_FAILED:
+ p.replyHeader.setErr(KeeperException.Code.AUTHFAILED.intValue());
+ break;
+ case CLOSED:
+ p.replyHeader.setErr(KeeperException.Code.SESSIONEXPIRED.intValue());
+ break;
+ default:
+ p.replyHeader.setErr(KeeperException.Code.CONNECTIONLOSS.intValue());
+ }
+ finishPacket(p);
+ }
+
+ private volatile long lastZxid;
+
+ public long getLastZxid() {
+ return lastZxid;
+ }
+
+ static class EndOfStreamException extends IOException {
+ private static final long serialVersionUID = -5438877188796231422L;
+
+ public EndOfStreamException(String msg) {
+ super(msg);
+ }
+
+ @Override
+ public String toString() {
+ return "EndOfStreamException: " + getMessage();
+ }
+ }
+
+ private static class SessionTimeoutException extends IOException {
+ private static final long serialVersionUID = 824482094072071178L;
+
+ public SessionTimeoutException(String msg) {
+ super(msg);
+ }
+ }
+
+ private static class SessionExpiredException extends IOException {
+ private static final long serialVersionUID = -1388816932076193249L;
+
+ public SessionExpiredException(String msg) {
+ super(msg);
+ }
+ }
+
+ private static class RWServerFoundException extends IOException {
+ private static final long serialVersionUID = 90431199887158758L;
+
+ public RWServerFoundException(String msg) {
+ super(msg);
+ }
+ }
+
+ /**
+ * This class services the outgoing request queue and generates the heart
+ * beats. It also spawns the ReadThread.
+ */
+ class SendThread extends ZooKeeperThread {
+ private long lastPingSentNs;
+ private final ClientCnxnSocket clientCnxnSocket;
+ private Random r = new Random();
Review comment:
*PREDICTABLE_RANDOM:* This random generator (java.util.Random) is predictable [(details)](https://find-sec-bugs.github.io/bugs.htm#PREDICTABLE_RANDOM)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/Login.java
##########
@@ -0,0 +1,422 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+/**
+ * This class is responsible for refreshing Kerberos credentials for
+ * logins for both Zookeeper client and server.
+ * See ZooKeeperSaslServer for server-side usage.
+ * See ZooKeeperSaslClient for client-side usage.
+ */
+
+import javax.security.auth.kerberos.KerberosPrincipal;
+import javax.security.auth.login.AppConfigurationEntry;
+import javax.security.auth.login.Configuration;
+import javax.security.auth.login.LoginContext;
+import javax.security.auth.login.LoginException;
+import javax.security.auth.callback.CallbackHandler;
+
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ZKConfig;
+import org.apache.zookeeper.server.ZooKeeperSaslServer;
+import org.apache.zookeeper.common.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.security.auth.kerberos.KerberosTicket;
+import javax.security.auth.Subject;
+
+import java.util.Date;
+import java.util.Random;
+import java.util.Set;
+
+public class Login {
+ private static final String KINIT_COMMAND_DEFAULT = "/usr/bin/kinit";
+ private static final Logger LOG = LoggerFactory.getLogger(Login.class);
+ public CallbackHandler callbackHandler;
+
+ // LoginThread will sleep until 80% of time from last refresh to
+ // ticket's expiry has been reached, at which time it will wake
+ // and try to renew the ticket.
+ private static final float TICKET_RENEW_WINDOW = 0.80f;
+
+ /**
+ * Percentage of random jitter added to the renewal time
+ */
+ private static final float TICKET_RENEW_JITTER = 0.05f;
+
+ // Regardless of TICKET_RENEW_WINDOW setting above and the ticket expiry time,
+ // thread will not sleep between refresh attempts any less than 1 minute (60*1000 milliseconds = 1 minute).
+ // Change the '1' to e.g. 5, to change this to 5 minutes.
+ private static final long MIN_TIME_BEFORE_RELOGIN = 1 * 60 * 1000L;
+
+ private Subject subject = null;
+ private Thread t = null;
+ private boolean isKrbTicket = false;
+ private boolean isUsingTicketCache = false;
+
+ /** Random number generator */
+ private static Random rng = new Random();
Review comment:
*PREDICTABLE_RANDOM:* This random generator (java.util.Random) is predictable [(details)](https://find-sec-bugs.github.io/bugs.htm#PREDICTABLE_RANDOM)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/SyncRequestProcessor.java
##########
@@ -0,0 +1,217 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.Flushable;
+import java.io.IOException;
+import java.util.LinkedList;
+import java.util.Random;
+import java.util.concurrent.LinkedBlockingQueue;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This RequestProcessor logs requests to disk. It batches the requests to do
+ * the io efficiently. The request is not passed to the next RequestProcessor
+ * until its log has been synced to disk.
+ *
+ * SyncRequestProcessor is used in 3 different cases
+ * 1. Leader - Sync request to disk and forward it to AckRequestProcessor which
+ * send ack back to itself.
+ * 2. Follower - Sync request to disk and forward request to
+ * SendAckRequestProcessor which send the packets to leader.
+ * SendAckRequestProcessor is flushable which allow us to force
+ * push packets to leader.
+ * 3. Observer - Sync committed request to disk (received as INFORM packet).
+ * It never send ack back to the leader, so the nextProcessor will
+ * be null. This change the semantic of txnlog on the observer
+ * since it only contains committed txns.
+ */
+public class SyncRequestProcessor extends ZooKeeperCriticalThread implements
+ RequestProcessor {
+ private static final Logger LOG = LoggerFactory.getLogger(SyncRequestProcessor.class);
+ private final ZooKeeperServer zks;
+ private final LinkedBlockingQueue<Request> queuedRequests =
+ new LinkedBlockingQueue<Request>();
+ private final RequestProcessor nextProcessor;
+
+ private Thread snapInProcess = null;
+ volatile private boolean running;
+
+ /**
+ * Transactions that have been written and are waiting to be flushed to
+ * disk. Basically this is the list of SyncItems whose callbacks will be
+ * invoked after flush returns successfully.
+ */
+ private final LinkedList<Request> toFlush = new LinkedList<Request>();
+ private final Random r = new Random();
Review comment:
*PREDICTABLE_RANDOM:* This random generator (java.util.Random) is predictable [(details)](https://find-sec-bugs.github.io/bugs.htm#PREDICTABLE_RANDOM)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/AuthFastLeaderElection.java
##########
@@ -0,0 +1,986 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+import java.io.IOException;
+import java.net.DatagramPacket;
+import java.net.DatagramSocket;
+import java.net.InetSocketAddress;
+import java.net.SocketException;
+import java.nio.ByteBuffer;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.Semaphore;
+
+import java.util.concurrent.TimeUnit;
+import java.util.Random;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.apache.zookeeper.common.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.server.ZooKeeperThread;
+import org.apache.zookeeper.server.quorum.Election;
+import org.apache.zookeeper.server.quorum.Vote;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.ServerState;
+
+/**
+ * @deprecated This class has been deprecated as of release 3.4.0.
+ */
+@Deprecated
+public class AuthFastLeaderElection implements Election {
+ private static final Logger LOG = LoggerFactory.getLogger(AuthFastLeaderElection.class);
+
+ /* Sequence numbers for messages */
+ static int sequencer = 0;
+ static int maxTag = 0;
+
+ /*
+ * Determine how much time a process has to wait once it believes that it
+ * has reached the end of leader election.
+ */
+ static int finalizeWait = 100;
+
+ /*
+ * Challenge counter to avoid replay attacks
+ */
+
+ static int challengeCounter = 0;
+
+ /*
+ * Flag to determine whether to authenticate or not
+ */
+
+ private boolean authEnabled = false;
+
+ static public class Notification {
+ /*
+ * Proposed leader
+ */
+ long leader;
+
+ /*
+ * zxid of the proposed leader
+ */
+ long zxid;
+
+ /*
+ * Epoch
+ */
+ long epoch;
+
+ /*
+ * current state of sender
+ */
+ QuorumPeer.ServerState state;
+
+ /*
+ * Address of the sender
+ */
+ InetSocketAddress addr;
+ }
+
+ /*
+ * Messages to send, both Notifications and Acks
+ */
+ static public class ToSend {
+ static enum mType {
+ crequest, challenge, notification, ack
+ }
+
+ ToSend(mType type, long tag, long leader, long zxid, long epoch,
+ ServerState state, InetSocketAddress addr) {
+
+ switch (type) {
+ case crequest:
+ this.type = 0;
+ this.tag = tag;
+ this.leader = leader;
+ this.zxid = zxid;
+ this.epoch = epoch;
+ this.state = state;
+ this.addr = addr;
+
+ break;
+ case challenge:
+ this.type = 1;
+ this.tag = tag;
+ this.leader = leader;
+ this.zxid = zxid;
+ this.epoch = epoch;
+ this.state = state;
+ this.addr = addr;
+
+ break;
+ case notification:
+ this.type = 2;
+ this.leader = leader;
+ this.zxid = zxid;
+ this.epoch = epoch;
+ this.state = QuorumPeer.ServerState.LOOKING;
+ this.tag = tag;
+ this.addr = addr;
+
+ break;
+ case ack:
+ this.type = 3;
+ this.tag = tag;
+ this.leader = leader;
+ this.zxid = zxid;
+ this.epoch = epoch;
+ this.state = state;
+ this.addr = addr;
+
+ break;
+ default:
+ break;
+ }
+ }
+
+ /*
+ * Message type: 0 notification, 1 acknowledgement
+ */
+ int type;
+
+ /*
+ * Proposed leader in the case of notification
+ */
+ long leader;
+
+ /*
+ * id contains the tag for acks, and zxid for notifications
+ */
+ long zxid;
+
+ /*
+ * Epoch
+ */
+ long epoch;
+
+ /*
+ * Current state;
+ */
+ QuorumPeer.ServerState state;
+
+ /*
+ * Message tag
+ */
+ long tag;
+
+ InetSocketAddress addr;
+ }
+
+ LinkedBlockingQueue<ToSend> sendqueue;
+
+ LinkedBlockingQueue<Notification> recvqueue;
+
+ private class Messenger {
+
+ final DatagramSocket mySocket;
+ long lastProposedLeader;
+ long lastProposedZxid;
+ long lastEpoch;
+ final Set<Long> ackset;
+ final ConcurrentHashMap<Long, Long> challengeMap;
+ final ConcurrentHashMap<Long, Semaphore> challengeMutex;
+ final ConcurrentHashMap<Long, Semaphore> ackMutex;
+ final ConcurrentHashMap<InetSocketAddress, ConcurrentHashMap<Long, Long>> addrChallengeMap;
+
+ class WorkerReceiver extends ZooKeeperThread {
+
+ DatagramSocket mySocket;
+ Messenger myMsg;
+
+ WorkerReceiver(DatagramSocket s, Messenger msg) {
+ super("WorkerReceiver-" + s.getRemoteSocketAddress());
+ mySocket = s;
+ myMsg = msg;
+ }
+
+ boolean saveChallenge(long tag, long challenge) {
+ Semaphore s = challengeMutex.get(tag);
+ if (s != null) {
+ synchronized (Messenger.this) {
+ challengeMap.put(tag, challenge);
+ challengeMutex.remove(tag);
+ }
+
+
+ s.release();
+ } else {
+ LOG.error("No challenge mutex object");
+ }
+
+
+ return true;
+ }
+
+ public void run() {
+ byte responseBytes[] = new byte[48];
+ ByteBuffer responseBuffer = ByteBuffer.wrap(responseBytes);
+ DatagramPacket responsePacket = new DatagramPacket(
+ responseBytes, responseBytes.length);
+ while (true) {
+ // Sleeps on receive
+ try {
+ responseBuffer.clear();
+ mySocket.receive(responsePacket);
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception receiving", e);
+ }
+ // Receive new message
+ if (responsePacket.getLength() != responseBytes.length) {
+ LOG.warn("Got a short response: "
+ + responsePacket.getLength() + " "
+ + responsePacket.toString());
+ continue;
+ }
+ responseBuffer.clear();
+ int type = responseBuffer.getInt();
+ if ((type > 3) || (type < 0)) {
+ LOG.warn("Got bad Msg type: " + type);
+ continue;
+ }
+ long tag = responseBuffer.getLong();
+
+ QuorumPeer.ServerState ackstate = QuorumPeer.ServerState.LOOKING;
+ switch (responseBuffer.getInt()) {
+ case 0:
+ ackstate = QuorumPeer.ServerState.LOOKING;
+ break;
+ case 1:
+ ackstate = QuorumPeer.ServerState.LEADING;
+ break;
+ case 2:
+ ackstate = QuorumPeer.ServerState.FOLLOWING;
+ break;
+ default:
+ LOG.warn("unknown type " + responseBuffer.getInt());
+ break;
+ }
+
+ Vote current = self.getCurrentVote();
+
+ switch (type) {
+ case 0:
+ // Receive challenge request
+ ToSend c = new ToSend(ToSend.mType.challenge, tag,
+ current.getId(), current.getZxid(),
+ logicalclock.get(), self.getPeerState(),
+ (InetSocketAddress) responsePacket
+ .getSocketAddress());
+ sendqueue.offer(c);
+ break;
+ case 1:
+ // Receive challenge and store somewhere else
+ long challenge = responseBuffer.getLong();
+ saveChallenge(tag, challenge);
+
+ break;
+ case 2:
+ Notification n = new Notification();
+ n.leader = responseBuffer.getLong();
+ n.zxid = responseBuffer.getLong();
+ n.epoch = responseBuffer.getLong();
+ n.state = ackstate;
+ n.addr = (InetSocketAddress) responsePacket
+ .getSocketAddress();
+
+ if ((myMsg.lastEpoch <= n.epoch)
+ && ((n.zxid > myMsg.lastProposedZxid)
+ || ((n.zxid == myMsg.lastProposedZxid)
+ && (n.leader > myMsg.lastProposedLeader)))) {
+ myMsg.lastProposedZxid = n.zxid;
+ myMsg.lastProposedLeader = n.leader;
+ myMsg.lastEpoch = n.epoch;
+ }
+
+ long recChallenge;
+ InetSocketAddress addr = (InetSocketAddress) responsePacket
+ .getSocketAddress();
+ if (authEnabled) {
+ ConcurrentHashMap<Long, Long> tmpMap = addrChallengeMap.get(addr);
+ if(tmpMap != null){
+ if (tmpMap.get(tag) != null) {
+ recChallenge = responseBuffer.getLong();
+
+ if (tmpMap.get(tag) == recChallenge) {
+ recvqueue.offer(n);
+
+ ToSend a = new ToSend(ToSend.mType.ack,
+ tag, current.getId(),
+ current.getZxid(),
+ logicalclock.get(), self.getPeerState(),
+ addr);
+
+ sendqueue.offer(a);
+ } else {
+ LOG.warn("Incorrect challenge: "
+ + recChallenge + ", "
+ + addrChallengeMap.toString());
+ }
+ } else {
+ LOG.warn("No challenge for host: " + addr
+ + " " + tag);
+ }
+ }
+ } else {
+ recvqueue.offer(n);
+
+ ToSend a = new ToSend(ToSend.mType.ack, tag,
+ current.getId(), current.getZxid(),
+ logicalclock.get(), self.getPeerState(),
+ (InetSocketAddress) responsePacket
+ .getSocketAddress());
+
+ sendqueue.offer(a);
+ }
+ break;
+
+ // Upon reception of an ack message, remove it from the
+ // queue
+ case 3:
+ Semaphore s = ackMutex.get(tag);
+
+ if(s != null)
+ s.release();
+ else LOG.error("Empty ack semaphore");
+
+ ackset.add(tag);
+
+ if (authEnabled) {
+ ConcurrentHashMap<Long, Long> tmpMap = addrChallengeMap.get(responsePacket
+ .getSocketAddress());
+ if(tmpMap != null) {
+ tmpMap.remove(tag);
+ } else {
+ LOG.warn("No such address in the ensemble configuration " + responsePacket
+ .getSocketAddress());
+ }
+ }
+
+ if (ackstate != QuorumPeer.ServerState.LOOKING) {
+ Notification outofsync = new Notification();
+ outofsync.leader = responseBuffer.getLong();
+ outofsync.zxid = responseBuffer.getLong();
+ outofsync.epoch = responseBuffer.getLong();
+ outofsync.state = ackstate;
+ outofsync.addr = (InetSocketAddress) responsePacket
+ .getSocketAddress();
+
+ recvqueue.offer(outofsync);
+ }
+
+ break;
+ // Default case
+ default:
+ LOG.warn("Received message of incorrect type " + type);
+ break;
+ }
+ }
+ }
+ }
+
+ class WorkerSender extends ZooKeeperThread {
+
+ Random rand;
+ int maxAttempts;
+ int ackWait = finalizeWait;
+
+ /*
+ * Receives a socket and max number of attempts as input
+ */
+
+ WorkerSender(int attempts) {
+ super("WorkerSender");
+ maxAttempts = attempts;
+ rand = new Random(java.lang.Thread.currentThread().getId()
+ + Time.currentElapsedTime());
Review comment:
*PREDICTABLE_RANDOM:* This random generator (java.util.Random) is predictable [(details)](https://find-sec-bugs.github.io/bugs.htm#PREDICTABLE_RANDOM)
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/ZooKeeper.java
##########
@@ -0,0 +1,3114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper;
+
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.AsyncCallback.ACLCallback;
+import org.apache.zookeeper.AsyncCallback.Children2Callback;
+import org.apache.zookeeper.AsyncCallback.ChildrenCallback;
+import org.apache.zookeeper.AsyncCallback.Create2Callback;
+import org.apache.zookeeper.AsyncCallback.DataCallback;
+import org.apache.zookeeper.AsyncCallback.MultiCallback;
+import org.apache.zookeeper.AsyncCallback.StatCallback;
+import org.apache.zookeeper.AsyncCallback.StringCallback;
+import org.apache.zookeeper.AsyncCallback.VoidCallback;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.NoWatcherException;
+import org.apache.zookeeper.OpResult.ErrorResult;
+import org.apache.zookeeper.Watcher.Event.EventType;
+import org.apache.zookeeper.Watcher.WatcherType;
+import org.apache.zookeeper.client.ConnectStringParser;
+import org.apache.zookeeper.client.HostProvider;
+import org.apache.zookeeper.client.StaticHostProvider;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.client.ZooKeeperSaslClient;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Stat;
+import org.apache.zookeeper.proto.CheckWatchesRequest;
+import org.apache.zookeeper.proto.Create2Response;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateResponse;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ExistsRequest;
+import org.apache.zookeeper.proto.GetACLRequest;
+import org.apache.zookeeper.proto.GetACLResponse;
+import org.apache.zookeeper.proto.GetChildren2Request;
+import org.apache.zookeeper.proto.GetChildren2Response;
+import org.apache.zookeeper.proto.GetChildrenRequest;
+import org.apache.zookeeper.proto.GetChildrenResponse;
+import org.apache.zookeeper.proto.GetDataRequest;
+import org.apache.zookeeper.proto.GetDataResponse;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.RemoveWatchesRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetACLResponse;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.proto.SetDataResponse;
+import org.apache.zookeeper.proto.SyncRequest;
+import org.apache.zookeeper.proto.SyncResponse;
+import org.apache.zookeeper.server.DataTree;
+import org.apache.zookeeper.server.EphemeralType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.reflect.Constructor;
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This is the main class of ZooKeeper client library. To use a ZooKeeper
+ * service, an application must first instantiate an object of ZooKeeper class.
+ * All the iterations will be done by calling the methods of ZooKeeper class.
+ * The methods of this class are thread-safe unless otherwise noted.
+ * <p>
+ * Once a connection to a server is established, a session ID is assigned to the
+ * client. The client will send heart beats to the server periodically to keep
+ * the session valid.
+ * <p>
+ * The application can call ZooKeeper APIs through a client as long as the
+ * session ID of the client remains valid.
+ * <p>
+ * If for some reason, the client fails to send heart beats to the server for a
+ * prolonged period of time (exceeding the sessionTimeout value, for instance),
+ * the server will expire the session, and the session ID will become invalid.
+ * The client object will no longer be usable. To make ZooKeeper API calls, the
+ * application must create a new client object.
+ * <p>
+ * If the ZooKeeper server the client currently connects to fails or otherwise
+ * does not respond, the client will automatically try to connect to another
+ * server before its session ID expires. If successful, the application can
+ * continue to use the client.
+ * <p>
+ * The ZooKeeper API methods are either synchronous or asynchronous. Synchronous
+ * methods blocks until the server has responded. Asynchronous methods just queue
+ * the request for sending and return immediately. They take a callback object that
+ * will be executed either on successful execution of the request or on error with
+ * an appropriate return code (rc) indicating the error.
+ * <p>
+ * Some successful ZooKeeper API calls can leave watches on the "data nodes" in
+ * the ZooKeeper server. Other successful ZooKeeper API calls can trigger those
+ * watches. Once a watch is triggered, an event will be delivered to the client
+ * which left the watch at the first place. Each watch can be triggered only
+ * once. Thus, up to one event will be delivered to a client for every watch it
+ * leaves.
+ * <p>
+ * A client needs an object of a class implementing Watcher interface for
+ * processing the events delivered to the client.
+ *
+ * When a client drops the current connection and re-connects to a server, all the
+ * existing watches are considered as being triggered but the undelivered events
+ * are lost. To emulate this, the client will generate a special event to tell
+ * the event handler a connection has been dropped. This special event has
+ * EventType None and KeeperState Disconnected.
+ *
+ */
+/*
+ * We suppress the "try" warning here because the close() method's signature
+ * allows it to throw InterruptedException which is strongly advised against
+ * by AutoCloseable (see: http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html#close()).
+ * close() will never throw an InterruptedException but the exception remains in the
+ * signature for backwards compatibility purposes.
+*/
+@SuppressWarnings("try")
+@InterfaceAudience.Public
+public class ZooKeeper implements AutoCloseable {
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#ZOOKEEPER_CLIENT_CNXN_SOCKET}
+ * instead.
+ */
+ @Deprecated
+ public static final String ZOOKEEPER_CLIENT_CNXN_SOCKET = "zookeeper.clientCnxnSocket";
+ // Setting this to "true" will enable encrypted client-server communication.
+
+ /**
+ * @deprecated Use {@link ZKClientConfig#SECURE_CLIENT}
+ * instead.
+ */
+ @Deprecated
+ public static final String SECURE_CLIENT = "zookeeper.client.secure";
+
+ protected final ClientCnxn cnxn;
+ private static final Logger LOG;
+ static {
+ //Keep these two lines together to keep the initialization order explicit
+ LOG = LoggerFactory.getLogger(ZooKeeper.class);
+ Environment.logEnv("Client environment:", LOG);
+ }
+
+ protected final HostProvider hostProvider;
+
+ /**
+ * This function allows a client to update the connection string by providing
+ * a new comma separated list of host:port pairs, each corresponding to a
+ * ZooKeeper server.
+ * <p>
+ * The function invokes a <a href="https://issues.apache.org/jira/browse/ZOOKEEPER-1355">
+ * probabilistic load-balancing algorithm</a> which may cause the client to disconnect from
+ * its current host with the goal to achieve expected uniform number of connections per server
+ * in the new list. In case the current host to which the client is connected is not in the new
+ * list this call will always cause the connection to be dropped. Otherwise, the decision
+ * is based on whether the number of servers has increased or decreased and by how much.
+ * For example, if the previous connection string contained 3 hosts and now the list contains
+ * these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ * move to one of the new hosts in order to balance the load. The algorithm will disconnect
+ * from the current host with probability 0.4 and in this case cause the client to connect
+ * to one of the 2 new hosts, chosen at random.
+ * <p>
+ * If the connection is dropped, the client moves to a special mode "reconfigMode" where he chooses
+ * a new server to connect to using the probabilistic algorithm. After finding a server,
+ * or exhausting all servers in the new list after trying all of them and failing to connect,
+ * the client moves back to the normal mode of operation where it will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ *
+ * @throws IOException in cases of network failure
+ */
+ public void updateServerList(String connectString) throws IOException {
+ ConnectStringParser connectStringParser = new ConnectStringParser(connectString);
+ Collection<InetSocketAddress> serverAddresses = connectStringParser.getServerAddresses();
+
+ ClientCnxnSocket clientCnxnSocket = cnxn.sendThread.getClientCnxnSocket();
+ InetSocketAddress currentHost = (InetSocketAddress) clientCnxnSocket.getRemoteSocketAddress();
+
+ boolean reconfigMode = hostProvider.updateServerList(serverAddresses, currentHost);
+
+ // cause disconnection - this will cause next to be called
+ // which will in turn call nextReconfigMode
+ if (reconfigMode) clientCnxnSocket.testableCloseSocket();
+ }
+
+ public ZooKeeperSaslClient getSaslClient() {
+ return cnxn.zooKeeperSaslClient;
+ }
+
+ protected final ZKWatchManager watchManager;
+
+ private final ZKClientConfig clientConfig;
+
+ public ZKClientConfig getClientConfig() {
+ return clientConfig;
+ }
+
+ protected List<String> getDataWatches() {
+ synchronized(watchManager.dataWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.dataWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getExistWatches() {
+ synchronized(watchManager.existWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.existWatches.keySet());
+ return rc;
+ }
+ }
+ protected List<String> getChildWatches() {
+ synchronized(watchManager.childWatches) {
+ List<String> rc = new ArrayList<String>(watchManager.childWatches.keySet());
+ return rc;
+ }
+ }
+
+ /**
+ * Manage watchers & handle events generated by the ClientCnxn object.
+ *
+ * We are implementing this as a nested class of ZooKeeper so that
+ * the public methods will not be exposed as part of the ZooKeeper client
+ * API.
+ */
+ static class ZKWatchManager implements ClientWatchManager {
+ private final Map<String, Set<Watcher>> dataWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> existWatches =
+ new HashMap<String, Set<Watcher>>();
+ private final Map<String, Set<Watcher>> childWatches =
+ new HashMap<String, Set<Watcher>>();
+ private boolean disableAutoWatchReset;
+
+ ZKWatchManager(boolean disableAutoWatchReset) {
+ this.disableAutoWatchReset = disableAutoWatchReset;
+ }
+
+ protected volatile Watcher defaultWatcher;
+
+ final private void addTo(Set<Watcher> from, Set<Watcher> to) {
+ if (from != null) {
+ to.addAll(from);
+ }
+ }
+
+ public Map<EventType, Set<Watcher>> removeWatcher(String clientPath,
+ Watcher watcher, WatcherType watcherType, boolean local, int rc)
+ throws KeeperException {
+ // Validate the provided znode path contains the given watcher of
+ // watcherType
+ containsWatcher(clientPath, watcher, watcherType);
+
+ Map<EventType, Set<Watcher>> removedWatchers = new HashMap<EventType, Set<Watcher>>();
+ HashSet<Watcher> childWatchersToRem = new HashSet<Watcher>();
+ removedWatchers
+ .put(EventType.ChildWatchRemoved, childWatchersToRem);
+ HashSet<Watcher> dataWatchersToRem = new HashSet<Watcher>();
+ removedWatchers.put(EventType.DataWatchRemoved, dataWatchersToRem);
+ boolean removedWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ removedWatcher = removeWatches(dataWatches, watcher,
+ clientPath, local, rc, dataWatchersToRem);
+ }
+
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ removedWatcher = removeWatches(childWatches, watcher,
+ clientPath, local, rc, childWatchersToRem);
+ }
+
+ synchronized (dataWatches) {
+ boolean removedDataWatcher = removeWatches(dataWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ synchronized (existWatches) {
+ boolean removedDataWatcher = removeWatches(existWatches,
+ watcher, clientPath, local, rc, dataWatchersToRem);
+ removedWatcher |= removedDataWatcher;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!removedWatcher) {
+ throw new KeeperException.NoWatcherException(clientPath);
+ }
+ return removedWatchers;
+ }
+
+ private boolean contains(String path, Watcher watcherObj,
+ Map<String, Set<Watcher>> pathVsWatchers) {
+ boolean watcherExists = true;
+ if (pathVsWatchers == null || pathVsWatchers.size() == 0) {
+ watcherExists = false;
+ } else {
+ Set<Watcher> watchers = pathVsWatchers.get(path);
+ if (watchers == null) {
+ watcherExists = false;
+ } else if (watcherObj == null) {
+ watcherExists = watchers.size() > 0;
+ } else {
+ watcherExists = watchers.contains(watcherObj);
+ }
+ }
+ return watcherExists;
+ }
+
+ /**
+ * Validate the provided znode path contains the given watcher and
+ * watcherType
+ *
+ * @param path
+ * - client path
+ * @param watcher
+ * - watcher object reference
+ * @param watcherType
+ * - type of the watcher
+ * @throws NoWatcherException
+ */
+ void containsWatcher(String path, Watcher watcher,
+ WatcherType watcherType) throws NoWatcherException{
+ boolean containsWatcher = false;
+ switch (watcherType) {
+ case Children: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+ break;
+ }
+ case Data: {
+ synchronized (dataWatches) {
+ containsWatcher = contains(path, watcher, dataWatches);
+ }
+
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ break;
+ }
+ case Any: {
+ synchronized (childWatches) {
+ containsWatcher = contains(path, watcher, childWatches);
+ }
+
+ synchronized (dataWatches) {
+ boolean contains_temp = contains(path, watcher, dataWatches);
+ containsWatcher |= contains_temp;
+ }
+ synchronized (existWatches) {
+ boolean contains_temp = contains(path, watcher,
+ existWatches);
+ containsWatcher |= contains_temp;
+ }
+ }
+ }
+ // Watcher function doesn't exists for the specified params
+ if (!containsWatcher) {
+ throw new KeeperException.NoWatcherException(path);
+ }
+ }
+
+ protected boolean removeWatches(Map<String, Set<Watcher>> pathVsWatcher,
+ Watcher watcher, String path, boolean local, int rc,
+ Set<Watcher> removedWatchers) throws KeeperException {
+ if (!local && rc != Code.OK.intValue()) {
+ throw KeeperException
+ .create(KeeperException.Code.get(rc), path);
+ }
+ boolean success = false;
+ // When local flag is true, remove watchers for the given path
+ // irrespective of rc. Otherwise shouldn't remove watchers locally
+ // when sees failure from server.
+ if (rc == Code.OK.intValue() || (local && rc != Code.OK.intValue())) {
+ // Remove all the watchers for the given path
+ if (watcher == null) {
+ Set<Watcher> pathWatchers = pathVsWatcher.remove(path);
+ if (pathWatchers != null) {
+ // found path watchers
+ removedWatchers.addAll(pathWatchers);
+ success = true;
+ }
+ } else {
+ Set<Watcher> watchers = pathVsWatcher.get(path);
+ if (watchers != null) {
+ if (watchers.remove(watcher)) {
+ // found path watcher
+ removedWatchers.add(watcher);
+ // cleanup <path vs watchlist>
+ if (watchers.size() <= 0) {
+ pathVsWatcher.remove(path);
+ }
+ success = true;
+ }
+ }
+ }
+ }
+ return success;
+ }
+
+ /* (non-Javadoc)
+ * @see org.apache.zookeeper.ClientWatchManager#materialize(Event.KeeperState,
+ * Event.EventType, java.lang.String)
+ */
+ @Override
+ public Set<Watcher> materialize(Watcher.Event.KeeperState state,
+ Watcher.Event.EventType type,
+ String clientPath)
+ {
+ Set<Watcher> result = new HashSet<Watcher>();
+
+ switch (type) {
+ case None:
+ result.add(defaultWatcher);
+ boolean clear = disableAutoWatchReset && state != Watcher.Event.KeeperState.SyncConnected;
+ synchronized(dataWatches) {
+ for(Set<Watcher> ws: dataWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ dataWatches.clear();
+ }
+ }
+
+ synchronized(existWatches) {
+ for(Set<Watcher> ws: existWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ existWatches.clear();
+ }
+ }
+
+ synchronized(childWatches) {
+ for(Set<Watcher> ws: childWatches.values()) {
+ result.addAll(ws);
+ }
+ if (clear) {
+ childWatches.clear();
+ }
+ }
+
+ return result;
+ case NodeDataChanged:
+ case NodeCreated:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ synchronized (existWatches) {
+ addTo(existWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeChildrenChanged:
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ case NodeDeleted:
+ synchronized (dataWatches) {
+ addTo(dataWatches.remove(clientPath), result);
+ }
+ // XXX This shouldn't be needed, but just in case
+ synchronized (existWatches) {
+ Set<Watcher> list = existWatches.remove(clientPath);
+ if (list != null) {
+ addTo(list, result);
+ LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
+ }
+ }
+ synchronized (childWatches) {
+ addTo(childWatches.remove(clientPath), result);
+ }
+ break;
+ default:
+ String msg = "Unhandled watch event type " + type
+ + " with state " + state + " on path " + clientPath;
+ LOG.error(msg);
+ throw new RuntimeException(msg);
+ }
+
+ return result;
+ }
+ }
+
+ /**
+ * Register a watcher for a particular path.
+ */
+ public abstract class WatchRegistration {
+ private Watcher watcher;
+ private String clientPath;
+ public WatchRegistration(Watcher watcher, String clientPath)
+ {
+ this.watcher = watcher;
+ this.clientPath = clientPath;
+ }
+
+ abstract protected Map<String, Set<Watcher>> getWatches(int rc);
+
+ /**
+ * Register the watcher with the set of watches on path.
+ * @param rc the result code of the operation that attempted to
+ * add the watch on the path.
+ */
+ public void register(int rc) {
+ if (shouldAddWatch(rc)) {
+ Map<String, Set<Watcher>> watches = getWatches(rc);
+ synchronized(watches) {
+ Set<Watcher> watchers = watches.get(clientPath);
+ if (watchers == null) {
+ watchers = new HashSet<Watcher>();
+ watches.put(clientPath, watchers);
+ }
+ watchers.add(watcher);
+ }
+ }
+ }
+ /**
+ * Determine whether the watch should be added based on return code.
+ * @param rc the result code of the operation that attempted to add the
+ * watch on the node
+ * @return true if the watch should be added, otw false
+ */
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0;
+ }
+ }
+
+ /** Handle the special case of exists watches - they add a watcher
+ * even in the case where NONODE result code is returned.
+ */
+ class ExistsWatchRegistration extends WatchRegistration {
+ public ExistsWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return rc == 0 ? watchManager.dataWatches : watchManager.existWatches;
+ }
+
+ @Override
+ protected boolean shouldAddWatch(int rc) {
+ return rc == 0 || rc == KeeperException.Code.NONODE.intValue();
+ }
+ }
+
+ class DataWatchRegistration extends WatchRegistration {
+ public DataWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.dataWatches;
+ }
+ }
+
+ class ChildWatchRegistration extends WatchRegistration {
+ public ChildWatchRegistration(Watcher watcher, String clientPath) {
+ super(watcher, clientPath);
+ }
+
+ @Override
+ protected Map<String, Set<Watcher>> getWatches(int rc) {
+ return watchManager.childWatches;
+ }
+ }
+
+ @InterfaceAudience.Public
+ public enum States {
+ CONNECTING, ASSOCIATING, CONNECTED, CONNECTEDREADONLY,
+ CLOSED, AUTH_FAILED, NOT_CONNECTED;
+
+ public boolean isAlive() {
+ return this != CLOSED && this != AUTH_FAILED;
+ }
+
+ /**
+ * Returns whether we are connected to a server (which
+ * could possibly be read-only, if this client is allowed
+ * to go to read-only mode)
+ * */
+ public boolean isConnected() {
+ return this == CONNECTED || this == CONNECTEDREADONLY;
+ }
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, false, conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ aHostProvider, null);
+ }
+
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, boolean)} which uses default
+ * {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, HostProvider aHostProvider,
+ ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher);
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = createConnection(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), canBeReadOnly);
+ cnxn.start();
+ }
+
+ // @VisibleForTesting
+ protected ClientCnxn createConnection(String chrootPath,
+ HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper,
+ ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
+ boolean canBeReadOnly) throws IOException {
+ return new ClientCnxn(chrootPath, hostProvider, sessionTimeout, this,
+ watchManager, clientCnxnSocket, canBeReadOnly);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ *
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString));
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed.
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" If
+ * the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param conf
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException
+ * in cases of network failure
+ * @throws IllegalArgumentException
+ * if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ boolean canBeReadOnly, ZKClientConfig conf) throws IOException {
+ this(connectString, sessionTimeout, watcher, canBeReadOnly,
+ createDefaultHostProvider(connectString), conf);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ *
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ * @throws IllegalArgumentException for an invalid list of ZooKeeper hosts
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd)
+ throws IOException
+ {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd, false);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider) throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, aHostProvider, null);
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * For backward compatibility, there is another version
+ * {@link #ZooKeeper(String, int, Watcher, long, byte[], boolean)} which uses
+ * default {@link StaticHostProvider}
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @param aHostProvider
+ * use this as HostProvider to enable custom behaviour.
+ * @param clientConfig
+ * (added in 3.5.2) passing this conf object gives each client the flexibility of
+ * configuring properties differently compared to other instances
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ *
+ * @since 3.5.5
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly,
+ HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException {
+ LOG.info("Initiating client connection, connectString=" + connectString
+ + " sessionTimeout=" + sessionTimeout
+ + " watcher=" + watcher
+ + " sessionId=" + Long.toHexString(sessionId)
+ + " sessionPasswd="
+ + (sessionPasswd == null ? "<null>" : "<hidden>"));
+
+ if (clientConfig == null) {
+ clientConfig = new ZKClientConfig();
+ }
+ this.clientConfig = clientConfig;
+ watchManager = defaultWatchManager();
+ watchManager.defaultWatcher = watcher;
+
+ ConnectStringParser connectStringParser = new ConnectStringParser(
+ connectString);
+ hostProvider = aHostProvider;
+
+ cnxn = new ClientCnxn(connectStringParser.getChrootPath(),
+ hostProvider, sessionTimeout, this, watchManager,
+ getClientCnxnSocket(), sessionId, sessionPasswd, canBeReadOnly);
+ cnxn.seenRwServerBefore = true; // since user has provided sessionId
+ cnxn.start();
+ }
+
+ /**
+ * To create a ZooKeeper client object, the application needs to pass a
+ * connection string containing a comma separated list of host:port pairs,
+ * each corresponding to a ZooKeeper server.
+ * <p>
+ * Session establishment is asynchronous. This constructor will initiate
+ * connection to the server and return immediately - potentially (usually)
+ * before the session is fully established. The watcher argument specifies
+ * the watcher that will be notified of any changes in state. This
+ * notification can come at any point before or after the constructor call
+ * has returned.
+ * <p>
+ * The instantiated ZooKeeper client object will pick an arbitrary server
+ * from the connectString and attempt to connect to it. If establishment of
+ * the connection fails, another server in the connect string will be tried
+ * (the order is non-deterministic, as we random shuffle the list), until a
+ * connection is established. The client will continue attempts until the
+ * session is explicitly closed (or the session is expired by the server).
+ * <p>
+ * Added in 3.2.0: An optional "chroot" suffix may also be appended to the
+ * connection string. This will run the client commands while interpreting
+ * all paths relative to this root (similar to the unix chroot command).
+ * <p>
+ * Use {@link #getSessionId} and {@link #getSessionPasswd} on an established
+ * client connection, these values must be passed as sessionId and
+ * sessionPasswd respectively if reconnecting. Otherwise, if not
+ * reconnecting, use the other constructor which does not require these
+ * parameters.
+ * <p>
+ * This constructor uses a StaticHostProvider; there is another one
+ * to enable custom behaviour.
+ *
+ * @param connectString
+ * comma separated host:port pairs, each corresponding to a zk
+ * server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
+ * If the optional chroot suffix is used the example would look
+ * like: "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002/app/a"
+ * where the client would be rooted at "/app/a" and all paths
+ * would be relative to this root - ie getting/setting/etc...
+ * "/foo/bar" would result in operations being run on
+ * "/app/a/foo/bar" (from the server perspective).
+ * @param sessionTimeout
+ * session timeout in milliseconds
+ * @param watcher
+ * a watcher object which will be notified of state changes, may
+ * also be notified for node events
+ * @param sessionId
+ * specific session id to use if reconnecting
+ * @param sessionPasswd
+ * password for this session
+ * @param canBeReadOnly
+ * (added in 3.4) whether the created client is allowed to go to
+ * read-only mode in case of partitioning. Read-only mode
+ * basically means that if the client can't find any majority
+ * servers but there's partitioned server it could reach, it
+ * connects to one in read-only mode, i.e. read requests are
+ * allowed while write requests are not. It continues seeking for
+ * majority in the background.
+ * @throws IOException in cases of network failure
+ * @throws IllegalArgumentException if an invalid chroot path is specified
+ */
+ public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher,
+ long sessionId, byte[] sessionPasswd, boolean canBeReadOnly)
+ throws IOException {
+ this(connectString, sessionTimeout, watcher, sessionId, sessionPasswd,
+ canBeReadOnly, createDefaultHostProvider(connectString));
+ }
+
+ // default hostprovider
+ private static HostProvider createDefaultHostProvider(String connectString) {
+ return new StaticHostProvider(
+ new ConnectStringParser(connectString).getServerAddresses());
+ }
+
+ // VisibleForTesting
+ public Testable getTestable() {
+ return new ZooKeeperTestable(this, cnxn);
+ }
+
+ /* Useful for testing watch handling behavior */
+ protected ZKWatchManager defaultWatchManager() {
+ return new ZKWatchManager(getClientConfig().getBoolean(ZKClientConfig.DISABLE_AUTO_WATCH_RESET));
+ }
+
+ /**
+ * The session id for this ZooKeeper client instance. The value returned is
+ * not valid until the client connects to a server and may change after a
+ * re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session id
+ */
+ public long getSessionId() {
+ return cnxn.getSessionId();
+ }
+
+ /**
+ * The session password for this ZooKeeper client instance. The value
+ * returned is not valid until the client connects to a server and may
+ * change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session password
+ */
+ public byte[] getSessionPasswd() {
+ return cnxn.getSessionPasswd();
+ }
+
+ /**
+ * The negotiated session timeout for this ZooKeeper client instance. The
+ * value returned is not valid until the client connects to a server and
+ * may change after a re-connect.
+ *
+ * This method is NOT thread safe
+ *
+ * @return current session timeout
+ */
+ public int getSessionTimeout() {
+ return cnxn.getSessionTimeout();
+ }
+
+ /**
+ * Add the specified scheme:auth information to this connection.
+ *
+ * This method is NOT thread safe
+ *
+ * @param scheme
+ * @param auth
+ */
+ public void addAuthInfo(String scheme, byte auth[]) {
+ cnxn.addAuthInfo(scheme, auth);
+ }
+
+ /**
+ * Specify the default watcher for the connection (overrides the one
+ * specified during construction).
+ *
+ * @param watcher
+ */
+ public synchronized void register(Watcher watcher) {
+ watchManager.defaultWatcher = watcher;
+ }
+
+ /**
+ * Close this client object. Once the client is closed, its session becomes
+ * invalid. All the ephemeral nodes in the ZooKeeper server associated with
+ * the session will be removed. The watches left on those nodes (and on
+ * their parents) will be triggered.
+ * <p>
+ * Added in 3.5.3: <a href="https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html">try-with-resources</a>
+ * may be used instead of calling close directly.
+ * </p>
+ * <p>
+ * This method does not wait for all internal threads to exit.
+ * Use the {@link #close(int) } method to wait for all resources to be released
+ * </p>
+ *
+ * @throws InterruptedException
+ */
+ public synchronized void close() throws InterruptedException {
+ if (!cnxn.getState().isAlive()) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Close called on already closed client");
+ }
+ return;
+ }
+
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Closing session: 0x" + Long.toHexString(getSessionId()));
+ }
+
+ try {
+ cnxn.close();
+ } catch (IOException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Ignoring unexpected exception during close", e);
+ }
+ }
+
+ LOG.info("Session: 0x" + Long.toHexString(getSessionId()) + " closed");
+ }
+
+ /**
+ * Close this client object as the {@link #close() } method.
+ * This method will wait for internal resources to be released.
+ *
+ * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released.
+ * Use zero or a negative value to skip the wait
+ * @throws InterruptedException
+ * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released
+ *
+ * @since 3.5.4
+ */
+ public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException {
+ close();
+ return testableWaitForShutdown(waitForShutdownTimeoutMs);
+ }
+
+ /**
+ * Prepend the chroot to the client path (if present). The expectation of
+ * this function is that the client path has been validated before this
+ * function is called
+ * @param clientPath path to the node
+ * @return server view of the path (chroot prepended to client path)
+ */
+ private String prependChroot(String clientPath) {
+ if (cnxn.chrootPath != null) {
+ // handle clientPath = "/"
+ if (clientPath.length() == 1) {
+ return cnxn.chrootPath;
+ }
+ return cnxn.chrootPath + clientPath;
+ } else {
+ return clientPath;
+ }
+ }
+
+ /**
+ * Create a node with the given path. The node data will be the given data,
+ * and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ request.setAcl(acl);
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ /**
+ * Create a node with the given path and returns the Stat of that node. The
+ * node data will be the given data and node acl will be the given acl.
+ * <p>
+ * The flags argument specifies whether the created node will be ephemeral
+ * or not.
+ * <p>
+ * An ephemeral node will be removed by the ZooKeeper automatically when the
+ * session associated with the creation of the node expires.
+ * <p>
+ * The flags argument can also specify to create a sequential node. The
+ * actual path name of a sequential node will be the given path plus a
+ * suffix "i" where i is the current sequential number of the node. The sequence
+ * number is always fixed length of 10 digits, 0 padded. Once
+ * such a node is created, the sequential number will be incremented by one.
+ * <p>
+ * If a node with the same actual path already exists in the ZooKeeper, a
+ * KeeperException with error code KeeperException.NodeExists will be
+ * thrown. Note that since a different actual path is used for each
+ * invocation of creating sequential node with the same path argument, the
+ * call will never throw "file exists" KeeperException.
+ * <p>
+ * If the parent node does not exist in the ZooKeeper, a KeeperException
+ * with error code KeeperException.NoNode will be thrown.
+ * <p>
+ * An ephemeral node cannot have children. If the parent node of the given
+ * path is ephemeral, a KeeperException with error code
+ * KeeperException.NoChildrenForEphemerals will be thrown.
+ * <p>
+ * This operation, if successful, will trigger all the watches left on the
+ * node of the given path by exists and getData API calls, and the watches
+ * left on the parent node by getChildren API calls.
+ * <p>
+ * If a node is created successfully, the ZooKeeper server will trigger the
+ * watches on the path left by exists calls, and the watches on the parent
+ * of the node by getChildren calls.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperExecption to be thrown.
+ *
+ * @param path
+ * the path for the node
+ * @param data
+ * the initial data for the node
+ * @param acl
+ * the acl for the node
+ * @param createMode
+ * specifying whether the node to be created is ephemeral
+ * and/or sequential
+ * @param stat
+ * The output Stat object.
+ * @return the actual path of the created node
+ * @throws KeeperException if the server returns a non-zero error code
+ * @throws KeeperException.InvalidACLException if the ACL is invalid, null, or empty
+ * @throws InterruptedException if the transaction is interrupted
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat)
+ throws KeeperException, InterruptedException {
+ return create(path, data, acl, createMode, stat, -1);
+ }
+
+ /**
+ * same as {@link #create(String, byte[], List, CreateMode, Stat)} but
+ * allows for specifying a TTL when mode is {@link CreateMode#PERSISTENT_WITH_TTL}
+ * or {@link CreateMode#PERSISTENT_SEQUENTIAL_WITH_TTL}. If the znode has not been modified
+ * within the given TTL, it will be deleted once it has no children. The TTL unit is
+ * milliseconds and must be greater than 0 and less than or equal to
+ * {@link EphemeralType#maxValue()} for {@link EphemeralType#TTL}.
+ */
+ public String create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Stat stat, long ttl)
+ throws KeeperException, InterruptedException {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ Create2Response response = new Create2Response();
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException();
+ }
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ ReplyHeader r = cnxn.submitRequest(h, record, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ if (cnxn.chrootPath == null) {
+ return response.getPath();
+ } else {
+ return response.getPath().substring(cnxn.chrootPath.length());
+ }
+ }
+
+ private void setCreateHeader(CreateMode createMode, RequestHeader h) {
+ if (createMode.isTTL()) {
+ h.setType(ZooDefs.OpCode.createTTL);
+ } else {
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create2);
+ }
+ }
+
+ private Record makeCreateRecord(CreateMode createMode, String serverPath, byte[] data, List<ACL> acl, long ttl) {
+ Record record;
+ if (createMode.isTTL()) {
+ CreateTTLRequest request = new CreateTTLRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setTtl(ttl);
+ record = request;
+ } else {
+ CreateRequest request = new CreateRequest();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ record = request;
+ }
+ return record;
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, StringCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, -1);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(createMode.isContainer() ? ZooDefs.OpCode.createContainer : ZooDefs.OpCode.create);
+ CreateRequest request = new CreateRequest();
+ CreateResponse response = new CreateResponse();
+ ReplyHeader r = new ReplyHeader();
+ request.setData(data);
+ request.setFlags(createMode.toFlag());
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ cnxn.queuePacket(h, r, request, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * The asynchronous version of create.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx)
+ {
+ create(path, data, acl, createMode, cb, ctx, -1);
+ }
+
+ /**
+ * The asynchronous version of create with ttl.
+ *
+ * @see #create(String, byte[], List, CreateMode, Stat, long)
+ */
+ public void create(final String path, byte data[], List<ACL> acl,
+ CreateMode createMode, Create2Callback cb, Object ctx, long ttl)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath, createMode.isSequential());
+ EphemeralType.validateTTL(createMode, ttl);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ setCreateHeader(createMode, h);
+ ReplyHeader r = new ReplyHeader();
+ Create2Response response = new Create2Response();
+ Record record = makeCreateRecord(createMode, serverPath, data, acl, ttl);
+ cnxn.queuePacket(h, r, record, response, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Delete the node with the given path. The call will succeed if such a node
+ * exists, and the given version matches the node's version (if the given
+ * version is -1, it matches any node's versions).
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the nodes does not exist.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * A KeeperException with error code KeeperException.NotEmpty will be thrown
+ * if the node has children.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by exists API calls, and the watches on the parent
+ * node left by getChildren API calls.
+ *
+ * @param path
+ * the path of the node to be deleted.
+ * @param version
+ * the expected node version.
+ * @throws InterruptedException IF the server transaction is interrupted
+ * @throws KeeperException If the server signals an error with a non-zero
+ * return code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void delete(final String path, int version)
+ throws InterruptedException, KeeperException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ /**
+ * Executes multiple ZooKeeper operations or none of them.
+ * <p>
+ * On success, a list of results is returned.
+ * On failure, an exception is raised which contains partial results and
+ * error details, see {@link KeeperException#getResults}
+ * <p>
+ * Note: The maximum allowable size of all of the data arrays in all of
+ * the setData operations in this single request is typically 1 MB
+ * (1,048,576 bytes). This limit is specified on the server via
+ * <a href="http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Unsafe+Options">jute.maxbuffer</a>.
+ * Requests larger than this will cause a KeeperException to be
+ * thrown.
+ *
+ * @param ops An iterable that contains the operations to be done.
+ * These should be created using the factory methods on {@link Op}.
+ * @return A list of results, one for each input Op, the order of
+ * which exactly matches the order of the <code>ops</code> input
+ * operations.
+ * @throws InterruptedException If the operation was interrupted.
+ * The operation may or may not have succeeded, but will not have
+ * partially succeeded if this exception is thrown.
+ * @throws KeeperException If the operation could not be completed
+ * due to some error in doing one of the specified ops.
+ * @throws IllegalArgumentException if an invalid path is specified
+ *
+ * @since 3.4.0
+ */
+ public List<OpResult> multi(Iterable<Op> ops) throws InterruptedException, KeeperException {
+ for (Op op : ops) {
+ op.validate();
+ }
+ return multiInternal(generateMultiTransaction(ops));
+ }
+
+ /**
+ * The asynchronous version of multi.
+ *
+ * @see #multi(Iterable)
+ */
+ public void multi(Iterable<Op> ops, MultiCallback cb, Object ctx) {
+ List<OpResult> results = validatePath(ops);
+ if (results.size() > 0) {
+ cb.processResult(KeeperException.Code.BADARGUMENTS.intValue(),
+ null, ctx, results);
+ return;
+ }
+ multiInternal(generateMultiTransaction(ops), cb, ctx);
+ }
+
+ private List<OpResult> validatePath(Iterable<Op> ops) {
+ List<OpResult> results = new ArrayList<OpResult>();
+ boolean error = false;
+ for (Op op : ops) {
+ try {
+ op.validate();
+ } catch (IllegalArgumentException iae) {
+ LOG.error("IllegalArgumentException: " + iae.getMessage());
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.BADARGUMENTS.intValue());
+ results.add(err);
+ error = true;
+ continue;
+ } catch (KeeperException ke) {
+ LOG.error("KeeperException: " + ke.getMessage());
+ ErrorResult err = new ErrorResult(ke.code().intValue());
+ results.add(err);
+ error = true;
+ continue;
+ }
+ ErrorResult err = new ErrorResult(
+ KeeperException.Code.RUNTIMEINCONSISTENCY.intValue());
+ results.add(err);
+ }
+ if (false == error) {
+ results.clear();
+ }
+ return results;
+ }
+
+ private MultiTransactionRecord generateMultiTransaction(Iterable<Op> ops) {
+ // reconstructing transaction with the chroot prefix
+ List<Op> transaction = new ArrayList<Op>();
+ for (Op op : ops) {
+ transaction.add(withRootPrefix(op));
+ }
+ return new MultiTransactionRecord(transaction);
+ }
+
+ private Op withRootPrefix(Op op) {
+ if (null != op.getPath()) {
+ final String serverPath = prependChroot(op.getPath());
+ if (!op.getPath().equals(serverPath)) {
+ return op.withChroot(serverPath);
+ }
+ }
+ return op;
+ }
+
+ protected void multiInternal(MultiTransactionRecord request, MultiCallback cb, Object ctx) {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb, null, null, ctx, null);
+ }
+
+ protected List<OpResult> multiInternal(MultiTransactionRecord request)
+ throws InterruptedException, KeeperException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.multi);
+ MultiResponse response = new MultiResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()));
+ }
+
+ List<OpResult> results = response.getResultList();
+
+ ErrorResult fatalError = null;
+ for (OpResult result : results) {
+ if (result instanceof ErrorResult && ((ErrorResult)result).getErr() != KeeperException.Code.OK.intValue()) {
+ fatalError = (ErrorResult) result;
+ break;
+ }
+ }
+
+ if (fatalError != null) {
+ KeeperException ex = KeeperException.create(KeeperException.Code.get(fatalError.getErr()));
+ ex.setMultiResults(results);
+ throw ex;
+ }
+
+ return results;
+ }
+
+ /**
+ * A Transaction is a thin wrapper on the {@link #multi} method
+ * which provides a builder object that can be used to construct
+ * and commit an atomic set of operations.
+ *
+ * @since 3.4.0
+ *
+ * @return a Transaction builder object
+ */
+ public Transaction transaction() {
+ return new Transaction(this);
+ }
+
+ /**
+ * The asynchronous version of delete.
+ *
+ * @see #delete(String, int)
+ */
+ public void delete(final String path, int version, VoidCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath;
+
+ // maintain semantics even in chroot case
+ // specifically - root cannot be deleted
+ // I think this makes sense even in chroot case.
+ if (clientPath.equals("/")) {
+ // a bit of a hack, but delete(/) will never succeed and ensures
+ // that the same semantics are maintained
+ serverPath = clientPath;
+ } else {
+ serverPath = prependChroot(clientPath);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.delete);
+ DeleteRequest request = new DeleteRequest();
+ request.setPath(serverPath);
+ request.setVersion(version);
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null);
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path the node path
+ * @param watcher explicit watcher
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat exists(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ if (r.getErr() == KeeperException.Code.NONODE.intValue()) {
+ return null;
+ }
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+
+ return response.getStat().getCzxid() == -1 ? null : response.getStat();
+ }
+
+ /**
+ * Return the stat of the node of the given path. Return null if no such a
+ * node exists.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that creates/delete the node or sets
+ * the data on the node.
+ *
+ * @param path
+ * the node path
+ * @param watch
+ * whether need to watch this node
+ * @return the stat of the node of the given path; return null if no such a
+ * node exists.
+ * @throws KeeperException If the server signals an error
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public Stat exists(String path, boolean watch) throws KeeperException,
+ InterruptedException
+ {
+ return exists(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, Watcher)
+ */
+ public void exists(final String path, Watcher watcher,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ExistsWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.exists);
+ ExistsRequest request = new ExistsRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of exists.
+ *
+ * @see #exists(String, boolean)
+ */
+ public void exists(String path, boolean watch, StatCallback cb, Object ctx) {
+ exists(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watcher explicit watcher
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public byte[] getData(final String path, Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * Return the data and the stat of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the node with the given path. The watch
+ * will be triggered by a successful operation that sets data on the node, or
+ * deletes the node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path the given path
+ * @param watch whether need to watch this node
+ * @param stat the stat of the node
+ * @return the data of the node
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getData(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getData(path, watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, Watcher, Stat)
+ */
+ public void getData(final String path, Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getData.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getData(String path, boolean watch, DataCallback cb, Object ctx) {
+ getData(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if the configuration node doesn't exists.
+ *
+ * @param watcher explicit watcher
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(Watcher watcher, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ configZnode);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getData();
+ }
+
+ /**
+ * The asynchronous version of getConfig.
+ *
+ * @see #getConfig(Watcher, Stat)
+ */
+ public void getConfig(Watcher watcher,
+ DataCallback cb, Object ctx)
+ {
+ final String configZnode = ZooDefs.CONFIG_NODE;
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new DataWatchRegistration(watcher, configZnode);
+ }
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getData);
+ GetDataRequest request = new GetDataRequest();
+ request.setPath(configZnode);
+ request.setWatch(watcher != null);
+ GetDataResponse response = new GetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ configZnode, configZnode, ctx, wcb);
+ }
+
+
+ /**
+ * Return the last committed configuration (as known to the server to which the client is connected)
+ * and the stat of the configuration.
+ * <p>
+ * If the watch is true and the call is successful (no exception is
+ * thrown), a watch will be left on the configuration node (ZooDefs.CONFIG_NODE). The watch
+ * will be triggered by a successful reconfig operation
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param watch whether need to watch this node
+ * @param stat the stat of the configuration node ZooDefs.CONFIG_NODE
+ * @return configuration data stored in ZooDefs.CONFIG_NODE
+ * @throws KeeperException If the server signals an error with a non-zero error code
+ * @throws InterruptedException If the server transaction is interrupted.
+ */
+ public byte[] getConfig(boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getConfig(watch ? watchManager.defaultWatcher : null, stat);
+ }
+
+ /**
+ * The Asynchronous version of getConfig.
+ *
+ * @see #getData(String, boolean, Stat)
+ */
+ public void getConfig(boolean watch, DataCallback cb, Object ctx) {
+ getConfig(watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public byte[] reconfig(List<String> joiningServers, List<String> leavingServers, List<String> newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ return internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, stat);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(String joiningServers, String leavingServers,
+ String newMembers, long fromConfig, DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * @deprecated instead use the reconfigure() methods instead in {@link org.apache.zookeeper.admin.ZooKeeperAdmin}
+ */
+ @Deprecated
+ public void reconfig(List<String> joiningServers,
+ List<String> leavingServers, List<String> newMembers, long fromConfig,
+ DataCallback cb, Object ctx) {
+ internalReconfig(joiningServers, leavingServers, newMembers, fromConfig, cb, ctx);
+ }
+
+ /**
+ * Set the data for the node of the given path if such a node exists and the
+ * given version matches the version of the node (if the given version is
+ * -1, it matches any node's versions). Return the stat of the node.
+ * <p>
+ * This operation, if successful, will trigger all the watches on the node
+ * of the given path left by getData calls.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given version does not match the node's version.
+ * <p>
+ * The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
+ * Arrays larger than this will cause a KeeperException to be thrown.
+ *
+ * @param path
+ * the path of the node
+ * @param data
+ * the data to set
+ * @param version
+ * the expected matching version
+ * @return the state of the node
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setData(final String path, byte data[], int version)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setData.
+ *
+ * @see #setData(String, byte[], int)
+ */
+ public void setData(final String path, byte data[], int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setData);
+ SetDataRequest request = new SetDataRequest();
+ request.setPath(serverPath);
+ request.setData(data);
+ request.setVersion(version);
+ SetDataResponse response = new SetDataResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the ACL and stat of the node of the given path.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * the given path for the node
+ * @param stat
+ * the stat of the node will be copied to this parameter if
+ * not null.
+ * @return the ACL array of the given node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<ACL> getACL(final String path, Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getAcl();
+ }
+
+ /**
+ * The asynchronous version of getACL.
+ *
+ * @see #getACL(String, Stat)
+ */
+ public void getACL(final String path, Stat stat, ACLCallback cb,
+ Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getACL);
+ GetACLRequest request = new GetACLRequest();
+ request.setPath(serverPath);
+ GetACLResponse response = new GetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Set the ACL for the node of the given path if such a node exists and the
+ * given aclVersion matches the acl version of the node. Return the stat of the
+ * node.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ * <p>
+ * A KeeperException with error code KeeperException.BadVersion will be
+ * thrown if the given aclVersion does not match the node's aclVersion.
+ *
+ * @param path the given path for the node
+ * @param acl the given acl for the node
+ * @param aclVersion the given acl version of the node
+ * @return the stat of the node.
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws org.apache.zookeeper.KeeperException.InvalidACLException If the acl is invalide.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public Stat setACL(final String path, List<ACL> acl, int aclVersion)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ if (acl != null && acl.size() == 0) {
+ throw new KeeperException.InvalidACLException(clientPath);
+ }
+ request.setAcl(acl);
+ request.setVersion(aclVersion);
+ SetACLResponse response = new SetACLResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getStat();
+ }
+
+ /**
+ * The asynchronous version of setACL.
+ *
+ * @see #setACL(String, List, int)
+ */
+ public void setACL(final String path, List<ACL> acl, int version,
+ StatCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.setACL);
+ SetACLRequest request = new SetACLRequest();
+ request.setPath(serverPath);
+ request.setAcl(acl);
+ request.setVersion(version);
+ SetACLResponse response = new SetACLResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * Return the list of the children of the node of the given path.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @param path
+ * @param watch
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ */
+ public List<String> getChildren(String path, boolean watch)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, Watcher)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ ChildrenCallback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren);
+ GetChildrenRequest request = new GetChildrenRequest();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildrenResponse response = new GetChildrenResponse();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @see #getChildren(String, boolean)
+ */
+ public void getChildren(String path, boolean watch, ChildrenCallback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is non-null and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watcher explicit watcher
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public List<String> getChildren(final String path, Watcher watcher,
+ Stat stat)
+ throws KeeperException, InterruptedException
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
+ }
+ return response.getChildren();
+ }
+
+ /**
+ * For the given znode path return the stat and children list.
+ * <p>
+ * If the watch is true and the call is successful (no exception is thrown),
+ * a watch will be left on the node with the given path. The watch will be
+ * triggered by a successful operation that deletes the node of the given
+ * path or creates/delete a child under the node.
+ * <p>
+ * The list of children returned is not sorted and no guarantee is provided
+ * as to its natural or lexical order.
+ * <p>
+ * A KeeperException with error code KeeperException.NoNode will be thrown
+ * if no node with the given path exists.
+ *
+ * @since 3.3.0
+ *
+ * @param path
+ * @param watch
+ * @param stat stat of the znode designated by path
+ * @return an unordered array of children of the node with the given path
+ * @throws InterruptedException If the server transaction is interrupted.
+ * @throws KeeperException If the server signals an error with a non-zero
+ * error code.
+ */
+ public List<String> getChildren(String path, boolean watch, Stat stat)
+ throws KeeperException, InterruptedException {
+ return getChildren(path, watch ? watchManager.defaultWatcher : null,
+ stat);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, Watcher, Stat)
+ */
+ public void getChildren(final String path, Watcher watcher,
+ Children2Callback cb, Object ctx)
+ {
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ // the watch contains the un-chroot path
+ WatchRegistration wcb = null;
+ if (watcher != null) {
+ wcb = new ChildWatchRegistration(watcher, clientPath);
+ }
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.getChildren2);
+ GetChildren2Request request = new GetChildren2Request();
+ request.setPath(serverPath);
+ request.setWatch(watcher != null);
+ GetChildren2Response response = new GetChildren2Response();
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, wcb);
+ }
+
+ /**
+ * The asynchronous version of getChildren.
+ *
+ * @since 3.3.0
+ *
+ * @see #getChildren(String, boolean, Stat)
+ */
+ public void getChildren(String path, boolean watch, Children2Callback cb,
+ Object ctx)
+ {
+ getChildren(path, watch ? watchManager.defaultWatcher : null, cb, ctx);
+ }
+
+ /**
+ * Asynchronous sync. Flushes channel between process and leader.
+ * @param path
+ * @param cb a handler for the callback
+ * @param ctx context to be provided to the callback
+ * @throws IllegalArgumentException if an invalid path is specified
+ */
+ public void sync(final String path, VoidCallback cb, Object ctx){
+ final String clientPath = path;
+ PathUtils.validatePath(clientPath);
+
+ final String serverPath = prependChroot(clientPath);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.sync);
+ SyncRequest request = new SyncRequest();
+ SyncResponse response = new SyncResponse();
+ request.setPath(serverPath);
+ cnxn.queuePacket(h, new ReplyHeader(), request, response, cb,
+ clientPath, serverPath, ctx, null);
+ }
+
+ /**
+ * For the given znode path, removes the specified watcher of given
+ * watcherType.
+ *
+ * <p>
+ * Watcher shouldn't be null. A successful call guarantees that, the
+ * removed watcher won't be triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcher
+ * - a concrete watcher
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether the watcher can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if any of the following is true:
+ * <ul>
+ * <li> {@code path} is invalid
+ * <li> {@code watcher} is null
+ * </ul>
+ *
+ * @since 3.5.0
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local);
+ }
+
+ /**
+ * The asynchronous version of removeWatches.
+ *
+ * @see #removeWatches
+ */
+ public void removeWatches(String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ validateWatcher(watcher);
+ removeWatches(ZooDefs.OpCode.checkWatches, path, watcher,
+ watcherType, local, cb, ctx);
+ }
+
+ /**
+ * For the given znode path, removes all the registered watchers of given
+ * watcherType.
+ *
+ * <p>
+ * A successful call guarantees that, the removed watchers won't be
+ * triggered.
+ * </p>
+ *
+ * @param path
+ * - the path of the node
+ * @param watcherType
+ * - the type of watcher to be removed
+ * @param local
+ * - whether watches can be removed locally when there is no
+ * server connection
+ * @throws InterruptedException
+ * if the server transaction is interrupted.
+ * @throws KeeperException.NoWatcherException
+ * if no watcher exists that match the specified parameters
+ * @throws KeeperException
+ * if the server signals an error with a non-zero error code.
+ * @throws IllegalArgumentException
+ * if an invalid {@code path} is specified
+ *
+ * @since 3.5.0
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local) throws InterruptedException, KeeperException {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null, watcherType,
+ local);
+ }
+
+ /**
+ * The asynchronous version of removeAllWatches.
+ *
+ * @see #removeAllWatches
+ */
+ public void removeAllWatches(String path, WatcherType watcherType,
+ boolean local, VoidCallback cb, Object ctx) {
+
+ removeWatches(ZooDefs.OpCode.removeWatches, path, null,
+ watcherType, local, cb, ctx);
+ }
+
+ private void validateWatcher(Watcher watcher) {
+ if (watcher == null) {
+ throw new IllegalArgumentException(
+ "Invalid Watcher, shouldn't be null!");
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local)
+ throws InterruptedException, KeeperException {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ ReplyHeader r = cnxn.submitRequest(h, request, null, null, wcb);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()),
+ clientPath);
+ }
+ }
+
+ private void removeWatches(int opCode, String path, Watcher watcher,
+ WatcherType watcherType, boolean local, VoidCallback cb, Object ctx) {
+ PathUtils.validatePath(path);
+ final String clientPath = path;
+ final String serverPath = prependChroot(clientPath);
+ WatchDeregistration wcb = new WatchDeregistration(clientPath, watcher,
+ watcherType, local, watchManager);
+
+ RequestHeader h = new RequestHeader();
+ h.setType(opCode);
+ Record request = getRemoveWatchesRequest(opCode, watcherType,
+ serverPath);
+
+ cnxn.queuePacket(h, new ReplyHeader(), request, null, cb, clientPath,
+ serverPath, ctx, null, wcb);
+ }
+
+ private Record getRemoveWatchesRequest(int opCode, WatcherType watcherType,
+ final String serverPath) {
+ Record request = null;
+ switch (opCode) {
+ case ZooDefs.OpCode.checkWatches:
+ CheckWatchesRequest chkReq = new CheckWatchesRequest();
+ chkReq.setPath(serverPath);
+ chkReq.setType(watcherType.getIntValue());
+ request = chkReq;
+ break;
+ case ZooDefs.OpCode.removeWatches:
+ RemoveWatchesRequest rmReq = new RemoveWatchesRequest();
+ rmReq.setPath(serverPath);
+ rmReq.setType(watcherType.getIntValue());
+ request = rmReq;
+ break;
+ default:
+ LOG.warn("unknown type " + opCode);
+ break;
+ }
+ return request;
+ }
+
+ public States getState() {
+ return cnxn.getState();
+ }
+
+ /**
+ * String representation of this ZooKeeper client. Suitable for things
+ * like logging.
+ *
+ * Do NOT count on the format of this string, it may change without
+ * warning.
+ *
+ * @since 3.3.0
+ */
+ @Override
+ public String toString() {
+ States state = getState();
+ return ("State:" + state.toString()
+ + (state.isConnected() ?
+ " Timeout:" + getSessionTimeout() + " " :
+ " ")
+ + cnxn);
+ }
+
+ /*
+ * Methods to aid in testing follow.
+ *
+ * THESE METHODS ARE EXPECTED TO BE USED FOR TESTING ONLY!!!
+ */
+
+ /**
+ * Wait up to wait milliseconds for the underlying threads to shutdown.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @param wait max wait in milliseconds
+ * @return true iff all threads are shutdown, otw false
+ */
+ protected boolean testableWaitForShutdown(int wait)
+ throws InterruptedException
+ {
+ cnxn.sendThread.join(wait);
+ if (cnxn.sendThread.isAlive()) return false;
+ cnxn.eventThread.join(wait);
+ if (cnxn.eventThread.isAlive()) return false;
+ return true;
+ }
+
+ /**
+ * Returns the address to which the socket is connected. Useful for testing
+ * against an ensemble - test client may need to know which server
+ * to shutdown if interested in verifying that the code handles
+ * disconnection/reconnection correctly.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableRemoteSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getRemoteSocketAddress();
+ }
+
+ /**
+ * Returns the local address to which the socket is bound.
+ * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
+ *
+ * @since 3.3.0
+ *
+ * @return ip address of the remote side of the connection or null if
+ * not connected
+ */
+ protected SocketAddress testableLocalSocketAddress() {
+ return cnxn.sendThread.getClientCnxnSocket().getLocalSocketAddress();
+ }
+
+ private ClientCnxnSocket getClientCnxnSocket() throws IOException {
+ String clientCnxnSocketName = getClientConfig().getProperty(
+ ZKClientConfig.ZOOKEEPER_CLIENT_CNXN_SOCKET);
+ if (clientCnxnSocketName == null) {
+ clientCnxnSocketName = ClientCnxnSocketNIO.class.getName();
+ }
+ try {
+ Constructor<?> clientCxnConstructor = Class.forName(clientCnxnSocketName).getDeclaredConstructor(ZKClientConfig.class);
+ ClientCnxnSocket clientCxnSocket = (ClientCnxnSocket) clientCxnConstructor.newInstance(getClientConfig());
+ return clientCxnSocket;
+ } catch (Exception e) {
+ IOException ioe = new IOException("Couldn't instantiate "
+ + clientCnxnSocketName);
+ ioe.initCause(e);
+ throw ioe;
+ }
+ }
+
+ protected byte[] internalReconfig(String joiningServers, String leavingServers, String newMembers, long fromConfig, Stat stat) throws KeeperException, InterruptedException {
+ RequestHeader h = new RequestHeader();
+ h.setType(ZooDefs.OpCode.reconfig);
+ ReconfigRequest request = new ReconfigRequest(joiningServers, leavingServers, newMembers, fromConfig);
+ GetDataResponse response = new GetDataResponse();
+ ReplyHeader r = cnxn.submitRequest(h, request, response, null);
+ if (r.getErr() != 0) {
+ throw KeeperException.create(KeeperException.Code.get(r.getErr()), "");
+ }
+ if (stat != null) {
+ DataTree.copyStat(response.getStat(), stat);
Review comment:
*NULL_DEREFERENCE:* object returned by `getStat(response)` could be null and is dereferenced by call to `copyStat(...)` at line 3084.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java
##########
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.EOFException;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.text.DateFormat;
+import java.util.Date;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.Record;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.zookeeper.server.persistence.FileHeader;
+import org.apache.zookeeper.server.persistence.FileTxnLog;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+
+@InterfaceAudience.Public
+public class LogFormatter {
+ private static final Logger LOG = LoggerFactory.getLogger(LogFormatter.class);
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 1) {
+ System.err.println("USAGE: LogFormatter log_file");
+ System.exit(2);
+ }
+ FileInputStream fis = new FileInputStream(args[0]);
+ BinaryInputArchive logStream = BinaryInputArchive.getArchive(fis);
+ FileHeader fhdr = new FileHeader();
+ fhdr.deserialize(logStream, "fileheader");
+
+ if (fhdr.getMagic() != FileTxnLog.TXNLOG_MAGIC) {
+ System.err.println("Invalid magic number for " + args[0]);
+ System.exit(2);
+ }
+ System.out.println("ZooKeeper Transactional Log File with dbid "
+ + fhdr.getDbid() + " txnlog format version "
+ + fhdr.getVersion());
+
+ int count = 0;
+ while (true) {
+ long crcValue;
+ byte[] bytes;
+ try {
+ crcValue = logStream.readLong("crcvalue");
+
+ bytes = logStream.readBuffer("txnEntry");
+ } catch (EOFException e) {
+ System.out.println("EOF reached after " + count + " txns.");
+ return;
+ }
+ if (bytes.length == 0) {
Review comment:
*NULL_DEREFERENCE:* object `bytes` last assigned on line 71 could be null and is dereferenced at line 76.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java
##########
@@ -0,0 +1,1012 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.BadArgumentsException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.MultiTransactionRecord;
+import org.apache.zookeeper.Op;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.common.Time;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.proto.CheckVersionRequest;
+import org.apache.zookeeper.proto.CreateRequest;
+import org.apache.zookeeper.proto.CreateTTLRequest;
+import org.apache.zookeeper.proto.DeleteRequest;
+import org.apache.zookeeper.proto.ReconfigRequest;
+import org.apache.zookeeper.proto.SetACLRequest;
+import org.apache.zookeeper.proto.SetDataRequest;
+import org.apache.zookeeper.server.ZooKeeperServer.ChangeRecord;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.quorum.Leader.XidRolloverException;
+import org.apache.zookeeper.server.quorum.LeaderZooKeeperServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig.ConfigException;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.txn.CheckVersionTxn;
+import org.apache.zookeeper.txn.CreateContainerTxn;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.CreateTTLTxn;
+import org.apache.zookeeper.txn.CreateTxn;
+import org.apache.zookeeper.txn.DeleteTxn;
+import org.apache.zookeeper.txn.ErrorTxn;
+import org.apache.zookeeper.txn.MultiTxn;
+import org.apache.zookeeper.txn.SetACLTxn;
+import org.apache.zookeeper.txn.SetDataTxn;
+import org.apache.zookeeper.txn.Txn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.StringReader;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.LinkedBlockingQueue;
+
+/**
+ * This request processor is generally at the start of a RequestProcessor
+ * change. It sets up any transactions associated with requests that change the
+ * state of the system. It counts on ZooKeeperServer to update
+ * outstandingRequests, so that it can take into account transactions that are
+ * in the queue to be applied when generating a transaction.
+ */
+public class PrepRequestProcessor extends ZooKeeperCriticalThread implements
+ RequestProcessor {
+ private static final Logger LOG = LoggerFactory.getLogger(PrepRequestProcessor.class);
+
+ static boolean skipACL;
+ static {
+ skipACL = System.getProperty("zookeeper.skipACL", "no").equals("yes");
+ if (skipACL) {
+ LOG.info("zookeeper.skipACL==\"yes\", ACL checks will be skipped");
+ }
+ }
+
+ /**
+ * this is only for testing purposes.
+ * should never be used otherwise
+ */
+ private static boolean failCreate = false;
+
+ LinkedBlockingQueue<Request> submittedRequests = new LinkedBlockingQueue<Request>();
+
+ private final RequestProcessor nextProcessor;
+
+ ZooKeeperServer zks;
+
+ public PrepRequestProcessor(ZooKeeperServer zks,
+ RequestProcessor nextProcessor) {
+ super("ProcessThread(sid:" + zks.getServerId() + " cport:"
+ + zks.getClientPort() + "):", zks.getZooKeeperServerListener());
+ this.nextProcessor = nextProcessor;
+ this.zks = zks;
+ }
+
+ /**
+ * method for tests to set failCreate
+ * @param b
+ */
+ public static void setFailCreate(boolean b) {
+ failCreate = b;
+ }
+ @Override
+ public void run() {
+ try {
+ while (true) {
+ Request request = submittedRequests.take();
+ long traceMask = ZooTrace.CLIENT_REQUEST_TRACE_MASK;
+ if (request.type == OpCode.ping) {
+ traceMask = ZooTrace.CLIENT_PING_TRACE_MASK;
+ }
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logRequest(LOG, traceMask, 'P', request, "");
+ }
+ if (Request.requestOfDeath == request) {
+ break;
+ }
+ pRequest(request);
+ }
+ } catch (RequestProcessorException e) {
+ if (e.getCause() instanceof XidRolloverException) {
+ LOG.info(e.getCause().getMessage());
+ }
+ handleException(this.getName(), e);
+ } catch (Exception e) {
+ handleException(this.getName(), e);
+ }
+ LOG.info("PrepRequestProcessor exited loop!");
+ }
+
+ private ChangeRecord getRecordForPath(String path) throws KeeperException.NoNodeException {
+ ChangeRecord lastChange = null;
+ synchronized (zks.outstandingChanges) {
+ lastChange = zks.outstandingChangesForPath.get(path);
+ if (lastChange == null) {
+ DataNode n = zks.getZKDatabase().getNode(path);
+ if (n != null) {
+ Set<String> children;
+ synchronized(n) {
+ children = n.getChildren();
+ }
+ lastChange = new ChangeRecord(-1, path, n.stat, children.size(),
+ zks.getZKDatabase().aclForNode(n));
+ }
+ }
+ }
+ if (lastChange == null || lastChange.stat == null) {
+ throw new KeeperException.NoNodeException(path);
+ }
+ return lastChange;
+ }
+
+ private ChangeRecord getOutstandingChange(String path) {
+ synchronized (zks.outstandingChanges) {
+ return zks.outstandingChangesForPath.get(path);
+ }
+ }
+
+ private void addChangeRecord(ChangeRecord c) {
+ synchronized (zks.outstandingChanges) {
+ zks.outstandingChanges.add(c);
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+
+ /**
+ * Grab current pending change records for each op in a multi-op.
+ *
+ * This is used inside MultiOp error code path to rollback in the event
+ * of a failed multi-op.
+ *
+ * @param multiRequest
+ * @return a map that contains previously existed records that probably need to be
+ * rolled back in any failure.
+ */
+ private Map<String, ChangeRecord> getPendingChanges(MultiTransactionRecord multiRequest) {
+ HashMap<String, ChangeRecord> pendingChangeRecords = new HashMap<String, ChangeRecord>();
+
+ for (Op op : multiRequest) {
+ String path = op.getPath();
+ ChangeRecord cr = getOutstandingChange(path);
+ // only previously existing records need to be rolled back.
+ if (cr != null) {
+ pendingChangeRecords.put(path, cr);
+ }
+
+ /*
+ * ZOOKEEPER-1624 - We need to store for parent's ChangeRecord
+ * of the parent node of a request. So that if this is a
+ * sequential node creation request, rollbackPendingChanges()
+ * can restore previous parent's ChangeRecord correctly.
+ *
+ * Otherwise, sequential node name generation will be incorrect
+ * for a subsequent request.
+ */
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1) {
+ continue;
+ }
+ String parentPath = path.substring(0, lastSlash);
+ ChangeRecord parentCr = getOutstandingChange(parentPath);
+ if (parentCr != null) {
+ pendingChangeRecords.put(parentPath, parentCr);
+ }
+ }
+
+ return pendingChangeRecords;
+ }
+
+ /**
+ * Rollback pending changes records from a failed multi-op.
+ *
+ * If a multi-op fails, we can't leave any invalid change records we created
+ * around. We also need to restore their prior value (if any) if their prior
+ * value is still valid.
+ *
+ * @param zxid
+ * @param pendingChangeRecords
+ */
+ void rollbackPendingChanges(long zxid, Map<String, ChangeRecord>pendingChangeRecords) {
+ synchronized (zks.outstandingChanges) {
+ // Grab a list iterator starting at the END of the list so we can iterate in reverse
+ Iterator<ChangeRecord> iter = zks.outstandingChanges.descendingIterator();
+ while (iter.hasNext()) {
+ ChangeRecord c = iter.next();
+ if (c.zxid == zxid) {
+ iter.remove();
+ // Remove all outstanding changes for paths of this multi.
+ // Previous records will be added back later.
+ zks.outstandingChangesForPath.remove(c.path);
+ } else {
+ break;
+ }
+ }
+
+ // we don't need to roll back any records because there is nothing left.
+ if (zks.outstandingChanges.isEmpty()) {
+ return;
+ }
+
+ long firstZxid = zks.outstandingChanges.peek().zxid;
+
+ for (ChangeRecord c : pendingChangeRecords.values()) {
+ // Don't apply any prior change records less than firstZxid.
+ // Note that previous outstanding requests might have been removed
+ // once they are completed.
+ if (c.zxid < firstZxid) {
+ continue;
+ }
+
+ // add previously existing records back.
+ zks.outstandingChangesForPath.put(c.path, c);
+ }
+ }
+ }
+
+ /**
+ * Grant or deny authorization to an operation on a node as a function of:
+ *
+ * @param zks: not used.
+ * @param acl: set of ACLs for the node
+ * @param perm: the permission that the client is requesting
+ * @param ids: the credentials supplied by the client
+ */
+ static void checkACL(ZooKeeperServer zks, List<ACL> acl, int perm,
+ List<Id> ids) throws KeeperException.NoAuthException {
+ if (skipACL) {
+ return;
+ }
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Permission requested: {} ", perm);
+ LOG.debug("ACLs for node: {}", acl);
+ LOG.debug("Client credentials: {}", ids);
+ }
+ if (acl == null || acl.size() == 0) {
+ return;
+ }
+ for (Id authId : ids) {
+ if (authId.getScheme().equals("super")) {
+ return;
+ }
+ }
+ for (ACL a : acl) {
+ Id id = a.getId();
+ if ((a.getPerms() & perm) != 0) {
+ if (id.getScheme().equals("world")
+ && id.getId().equals("anyone")) {
+ return;
+ }
+ AuthenticationProvider ap = ProviderRegistry.getProvider(id
+ .getScheme());
+ if (ap != null) {
+ for (Id authId : ids) {
+ if (authId.getScheme().equals(id.getScheme())
+ && ap.matches(authId.getId(), id.getId())) {
+ return;
+ }
+ }
+ }
+ }
+ }
+ throw new KeeperException.NoAuthException();
+ }
+
+ /**
+ * Performs basic validation of a path for a create request.
+ * Throws if the path is not valid and returns the parent path.
+ * @throws BadArgumentsException
+ */
+ private String validatePathForCreate(String path, long sessionId)
+ throws BadArgumentsException {
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1 || failCreate) {
+ LOG.info("Invalid path %s with session 0x%s",
+ path, Long.toHexString(sessionId));
+ throw new KeeperException.BadArgumentsException(path);
+ }
+ return path.substring(0, lastSlash);
+ }
+
+ /**
+ * This method will be called inside the ProcessRequestThread, which is a
+ * singleton, so there will be a single thread calling this code.
+ *
+ * @param type
+ * @param zxid
+ * @param request
+ * @param record
+ */
+ protected void pRequest2Txn(int type, long zxid, Request request,
+ Record record, boolean deserialize)
+ throws KeeperException, IOException, RequestProcessorException
+ {
+ request.setHdr(new TxnHeader(request.sessionId, request.cxid, zxid,
+ Time.currentWallTime(), type));
+
+ switch (type) {
+ case OpCode.create:
+ case OpCode.create2:
+ case OpCode.createTTL:
+ case OpCode.createContainer: {
+ pRequest2TxnCreate(type, request, record, deserialize);
+ break;
+ }
+ case OpCode.deleteContainer: {
+ String path = new String(request.request.array());
+ String parentPath = getParentPathAndValidate(path);
+ ChangeRecord parentRecord = getRecordForPath(parentPath);
+ ChangeRecord nodeRecord = getRecordForPath(path);
+ if (nodeRecord.childCount > 0) {
+ throw new KeeperException.NotEmptyException(path);
+ }
+ if (EphemeralType.get(nodeRecord.stat.getEphemeralOwner()) == EphemeralType.NORMAL) {
+ throw new KeeperException.BadVersionException(path);
+ }
+ request.setTxn(new DeleteTxn(path));
+ parentRecord = parentRecord.duplicate(request.getHdr().getZxid());
+ parentRecord.childCount--;
+ addChangeRecord(parentRecord);
+ addChangeRecord(new ChangeRecord(request.getHdr().getZxid(), path, null, -1, null));
+ break;
+ }
+ case OpCode.delete:
+ zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ DeleteRequest deleteRequest = (DeleteRequest)record;
+ if(deserialize)
+ ByteBufferInputStream.byteBuffer2Record(request.request, deleteRequest);
+ String path = deleteRequest.getPath();
+ String parentPath = getParentPathAndValidate(path);
+ ChangeRecord parentRecord = getRecordForPath(parentPath);
+ ChangeRecord nodeRecord = getRecordForPath(path);
+ checkACL(zks, parentRecord.acl, ZooDefs.Perms.DELETE, request.authInfo);
+ checkAndIncVersion(nodeRecord.stat.getVersion(), deleteRequest.getVersion(), path);
+ if (nodeRecord.childCount > 0) {
+ throw new KeeperException.NotEmptyException(path);
+ }
+ request.setTxn(new DeleteTxn(path));
+ parentRecord = parentRecord.duplicate(request.getHdr().getZxid());
+ parentRecord.childCount--;
+ addChangeRecord(parentRecord);
+ addChangeRecord(new ChangeRecord(request.getHdr().getZxid(), path, null, -1, null));
+ break;
+ case OpCode.setData:
+ zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ SetDataRequest setDataRequest = (SetDataRequest)record;
+ if(deserialize)
+ ByteBufferInputStream.byteBuffer2Record(request.request, setDataRequest);
+ path = setDataRequest.getPath();
+ validatePath(path, request.sessionId);
+ nodeRecord = getRecordForPath(path);
+ checkACL(zks, nodeRecord.acl, ZooDefs.Perms.WRITE, request.authInfo);
+ int newVersion = checkAndIncVersion(nodeRecord.stat.getVersion(), setDataRequest.getVersion(), path);
+ request.setTxn(new SetDataTxn(path, setDataRequest.getData(), newVersion));
+ nodeRecord = nodeRecord.duplicate(request.getHdr().getZxid());
+ nodeRecord.stat.setVersion(newVersion);
+ addChangeRecord(nodeRecord);
+ break;
+ case OpCode.reconfig:
+ if (!QuorumPeerConfig.isReconfigEnabled()) {
+ LOG.error("Reconfig operation requested but reconfig feature is disabled.");
+ throw new KeeperException.ReconfigDisabledException();
+ }
+
+ if (skipACL) {
+ LOG.warn("skipACL is set, reconfig operation will skip ACL checks!");
+ }
+
+ zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ ReconfigRequest reconfigRequest = (ReconfigRequest)record;
+ LeaderZooKeeperServer lzks;
+ try {
+ lzks = (LeaderZooKeeperServer)zks;
+ } catch (ClassCastException e) {
+ // standalone mode - reconfiguration currently not supported
+ throw new KeeperException.UnimplementedException();
+ }
+ QuorumVerifier lastSeenQV = lzks.self.getLastSeenQuorumVerifier();
+ // check that there's no reconfig in progress
+ if (lastSeenQV.getVersion()!=lzks.self.getQuorumVerifier().getVersion()) {
+ throw new KeeperException.ReconfigInProgress();
+ }
+ long configId = reconfigRequest.getCurConfigId();
+
+ if (configId != -1 && configId!=lzks.self.getLastSeenQuorumVerifier().getVersion()){
+ String msg = "Reconfiguration from version " + configId + " failed -- last seen version is " +
+ lzks.self.getLastSeenQuorumVerifier().getVersion();
+ throw new KeeperException.BadVersionException(msg);
+ }
+
+ String newMembers = reconfigRequest.getNewMembers();
+
+ if (newMembers != null) { //non-incremental membership change
+ LOG.info("Non-incremental reconfig");
+
+ // Input may be delimited by either commas or newlines so convert to common newline separated format
+ newMembers = newMembers.replaceAll(",", "\n");
+
+ try{
+ Properties props = new Properties();
+ props.load(new StringReader(newMembers));
+ request.qv = QuorumPeerConfig.parseDynamicConfig(props, lzks.self.getElectionType(), true, false);
+ request.qv.setVersion(request.getHdr().getZxid());
+ } catch (IOException | ConfigException e) {
+ throw new KeeperException.BadArgumentsException(e.getMessage());
+ }
+ } else { //incremental change - must be a majority quorum system
+ LOG.info("Incremental reconfig");
+
+ List<String> joiningServers = null;
+ String joiningServersString = reconfigRequest.getJoiningServers();
+ if (joiningServersString != null)
+ {
+ joiningServers = StringUtils.split(joiningServersString,",");
+ }
+
+ List<String> leavingServers = null;
+ String leavingServersString = reconfigRequest.getLeavingServers();
+ if (leavingServersString != null)
+ {
+ leavingServers = StringUtils.split(leavingServersString, ",");
+ }
+
+ if (!(lastSeenQV instanceof QuorumMaj)) {
+ String msg = "Incremental reconfiguration requested but last configuration seen has a non-majority quorum system";
+ LOG.warn(msg);
+ throw new KeeperException.BadArgumentsException(msg);
+ }
+ Map<Long, QuorumServer> nextServers = new HashMap<Long, QuorumServer>(lastSeenQV.getAllMembers());
+ try {
+ if (leavingServers != null) {
+ for (String leaving: leavingServers){
+ long sid = Long.parseLong(leaving);
+ nextServers.remove(sid);
+ }
+ }
+ if (joiningServers != null) {
+ for (String joiner: joiningServers){
+ // joiner should have the following format: server.x = server_spec;client_spec
+ String[] parts = StringUtils.split(joiner, "=").toArray(new String[0]);
+ if (parts.length != 2) {
+ throw new KeeperException.BadArgumentsException("Wrong format of server string");
+ }
+ // extract server id x from first part of joiner: server.x
+ Long sid = Long.parseLong(parts[0].substring(parts[0].lastIndexOf('.') + 1));
+ QuorumServer qs = new QuorumServer(sid, parts[1]);
+ if (qs.clientAddr == null || qs.electionAddr == null || qs.addr == null) {
+ throw new KeeperException.BadArgumentsException("Wrong format of server string - each server should have 3 ports specified");
+ }
+
+ // check duplication of addresses and ports
+ for (QuorumServer nqs: nextServers.values()) {
+ if (qs.id == nqs.id) {
+ continue;
+ }
+ qs.checkAddressDuplicate(nqs);
+ }
+
+ nextServers.remove(qs.id);
+ nextServers.put(qs.id, qs);
+ }
+ }
+ } catch (ConfigException e){
+ throw new KeeperException.BadArgumentsException("Reconfiguration failed");
+ }
+ request.qv = new QuorumMaj(nextServers);
+ request.qv.setVersion(request.getHdr().getZxid());
+ }
+ if (QuorumPeerConfig.isStandaloneEnabled() && request.qv.getVotingMembers().size() < 2) {
+ String msg = "Reconfig failed - new configuration must include at least 2 followers";
+ LOG.warn(msg);
+ throw new KeeperException.BadArgumentsException(msg);
+ } else if (request.qv.getVotingMembers().size() < 1) {
+ String msg = "Reconfig failed - new configuration must include at least 1 follower";
+ LOG.warn(msg);
+ throw new KeeperException.BadArgumentsException(msg);
+ }
+
+ if (!lzks.getLeader().isQuorumSynced(request.qv)) {
+ String msg2 = "Reconfig failed - there must be a connected and synced quorum in new configuration";
+ LOG.warn(msg2);
+ throw new KeeperException.NewConfigNoQuorum();
+ }
+
+ nodeRecord = getRecordForPath(ZooDefs.CONFIG_NODE);
+ checkACL(zks, nodeRecord.acl, ZooDefs.Perms.WRITE, request.authInfo);
+ request.setTxn(new SetDataTxn(ZooDefs.CONFIG_NODE, request.qv.toString().getBytes(), -1));
+ nodeRecord = nodeRecord.duplicate(request.getHdr().getZxid());
+ nodeRecord.stat.setVersion(-1);
+ addChangeRecord(nodeRecord);
+ break;
+ case OpCode.setACL:
+ zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ SetACLRequest setAclRequest = (SetACLRequest)record;
+ if(deserialize)
+ ByteBufferInputStream.byteBuffer2Record(request.request, setAclRequest);
+ path = setAclRequest.getPath();
+ validatePath(path, request.sessionId);
+ List<ACL> listACL = fixupACL(path, request.authInfo, setAclRequest.getAcl());
+ nodeRecord = getRecordForPath(path);
+ checkACL(zks, nodeRecord.acl, ZooDefs.Perms.ADMIN, request.authInfo);
+ newVersion = checkAndIncVersion(nodeRecord.stat.getAversion(), setAclRequest.getVersion(), path);
+ request.setTxn(new SetACLTxn(path, listACL, newVersion));
+ nodeRecord = nodeRecord.duplicate(request.getHdr().getZxid());
+ nodeRecord.stat.setAversion(newVersion);
+ addChangeRecord(nodeRecord);
+ break;
+ case OpCode.createSession:
+ request.request.rewind();
+ int to = request.request.getInt();
+ request.setTxn(new CreateSessionTxn(to));
+ request.request.rewind();
+ if (request.isLocalSession()) {
+ // This will add to local session tracker if it is enabled
+ zks.sessionTracker.addSession(request.sessionId, to);
+ } else {
+ // Explicitly add to global session if the flag is not set
+ zks.sessionTracker.addGlobalSession(request.sessionId, to);
+ }
+ zks.setOwner(request.sessionId, request.getOwner());
+ break;
+ case OpCode.closeSession:
+ // We don't want to do this check since the session expiration thread
+ // queues up this operation without being the session owner.
+ // this request is the last of the session so it should be ok
+ //zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ Set<String> es = zks.getZKDatabase()
+ .getEphemerals(request.sessionId);
+ synchronized (zks.outstandingChanges) {
+ for (ChangeRecord c : zks.outstandingChanges) {
+ if (c.stat == null) {
+ // Doing a delete
+ es.remove(c.path);
+ } else if (c.stat.getEphemeralOwner() == request.sessionId) {
+ es.add(c.path);
+ }
+ }
+ for (String path2Delete : es) {
+ addChangeRecord(new ChangeRecord(request.getHdr().getZxid(), path2Delete, null, 0, null));
+ }
+
+ zks.sessionTracker.setSessionClosing(request.sessionId);
+ }
+ break;
+ case OpCode.check:
+ zks.sessionTracker.checkSession(request.sessionId, request.getOwner());
+ CheckVersionRequest checkVersionRequest = (CheckVersionRequest)record;
+ if(deserialize)
+ ByteBufferInputStream.byteBuffer2Record(request.request, checkVersionRequest);
+ path = checkVersionRequest.getPath();
+ validatePath(path, request.sessionId);
+ nodeRecord = getRecordForPath(path);
+ checkACL(zks, nodeRecord.acl, ZooDefs.Perms.READ, request.authInfo);
+ request.setTxn(new CheckVersionTxn(path, checkAndIncVersion(nodeRecord.stat.getVersion(),
+ checkVersionRequest.getVersion(), path)));
+ break;
+ default:
+ LOG.warn("unknown type " + type);
+ break;
+ }
+ }
+
+ private void pRequest2TxnCreate(int type, Request request, Record record, boolean deserialize) throws IOException, KeeperException {
+ if (deserialize) {
+ ByteBufferInputStream.byteBuffer2Record(request.request, record);
+ }
+
+ int flags;
+ String path;
+ List<ACL> acl;
+ byte[] data;
+ long ttl;
+ if (type == OpCode.createTTL) {
+ CreateTTLRequest createTtlRequest = (CreateTTLRequest)record;
+ flags = createTtlRequest.getFlags();
+ path = createTtlRequest.getPath();
+ acl = createTtlRequest.getAcl();
+ data = createTtlRequest.getData();
+ ttl = createTtlRequest.getTtl();
+ } else {
+ CreateRequest createRequest = (CreateRequest)record;
+ flags = createRequest.getFlags();
+ path = createRequest.getPath();
+ acl = createRequest.getAcl();
+ data = createRequest.getData();
+ ttl = -1;
+ }
+ CreateMode createMode = CreateMode.fromFlag(flags);
+ validateCreateRequest(path, createMode, request, ttl);
+ String parentPath = validatePathForCreate(path, request.sessionId);
+
+ List<ACL> listACL = fixupACL(path, request.authInfo, acl);
+ ChangeRecord parentRecord = getRecordForPath(parentPath);
+
+ checkACL(zks, parentRecord.acl, ZooDefs.Perms.CREATE, request.authInfo);
+ int parentCVersion = parentRecord.stat.getCversion();
+ if (createMode.isSequential()) {
+ path = path + String.format(Locale.ENGLISH, "%010d", parentCVersion);
+ }
+ validatePath(path, request.sessionId);
+ try {
+ if (getRecordForPath(path) != null) {
+ throw new KeeperException.NodeExistsException(path);
+ }
+ } catch (KeeperException.NoNodeException e) {
+ // ignore this one
+ }
+ boolean ephemeralParent = EphemeralType.get(parentRecord.stat.getEphemeralOwner()) == EphemeralType.NORMAL;
+ if (ephemeralParent) {
+ throw new KeeperException.NoChildrenForEphemeralsException(path);
+ }
+ int newCversion = parentRecord.stat.getCversion()+1;
+ if (type == OpCode.createContainer) {
+ request.setTxn(new CreateContainerTxn(path, data, listACL, newCversion));
+ } else if (type == OpCode.createTTL) {
+ request.setTxn(new CreateTTLTxn(path, data, listACL, newCversion, ttl));
+ } else {
+ request.setTxn(new CreateTxn(path, data, listACL, createMode.isEphemeral(),
+ newCversion));
+ }
+ StatPersisted s = new StatPersisted();
+ if (createMode.isEphemeral()) {
+ s.setEphemeralOwner(request.sessionId);
+ }
+ parentRecord = parentRecord.duplicate(request.getHdr().getZxid());
+ parentRecord.childCount++;
+ parentRecord.stat.setCversion(newCversion);
+ addChangeRecord(parentRecord);
+ addChangeRecord(new ChangeRecord(request.getHdr().getZxid(), path, s, 0, listACL));
+ }
+
+ private void validatePath(String path, long sessionId) throws BadArgumentsException {
+ try {
+ PathUtils.validatePath(path);
+ } catch(IllegalArgumentException ie) {
+ LOG.info("Invalid path {} with session 0x{}, reason: {}",
+ path, Long.toHexString(sessionId), ie.getMessage());
+ throw new BadArgumentsException(path);
+ }
+ }
+
+ private String getParentPathAndValidate(String path)
+ throws BadArgumentsException {
+ int lastSlash = path.lastIndexOf('/');
+ if (lastSlash == -1 || path.indexOf('\0') != -1
+ || zks.getZKDatabase().isSpecialPath(path)) {
+ throw new BadArgumentsException(path);
+ }
+ return path.substring(0, lastSlash);
+ }
+
+ private static int checkAndIncVersion(int currentVersion, int expectedVersion, String path)
+ throws KeeperException.BadVersionException {
+ if (expectedVersion != -1 && expectedVersion != currentVersion) {
+ throw new KeeperException.BadVersionException(path);
+ }
+ return currentVersion + 1;
+ }
+
+ /**
+ * This method will be called inside the ProcessRequestThread, which is a
+ * singleton, so there will be a single thread calling this code.
+ *
+ * @param request
+ */
+ protected void pRequest(Request request) throws RequestProcessorException {
+ // LOG.info("Prep>>> cxid = " + request.cxid + " type = " +
+ // request.type + " id = 0x" + Long.toHexString(request.sessionId));
+ request.setHdr(null);
+ request.setTxn(null);
+
+ try {
+ switch (request.type) {
+ case OpCode.createContainer:
+ case OpCode.create:
+ case OpCode.create2:
+ CreateRequest create2Request = new CreateRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, create2Request, true);
+ break;
+ case OpCode.createTTL:
+ CreateTTLRequest createTtlRequest = new CreateTTLRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, createTtlRequest, true);
+ break;
+ case OpCode.deleteContainer:
+ case OpCode.delete:
+ DeleteRequest deleteRequest = new DeleteRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, deleteRequest, true);
+ break;
+ case OpCode.setData:
+ SetDataRequest setDataRequest = new SetDataRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, setDataRequest, true);
+ break;
+ case OpCode.reconfig:
+ ReconfigRequest reconfigRequest = new ReconfigRequest();
+ ByteBufferInputStream.byteBuffer2Record(request.request, reconfigRequest);
+ pRequest2Txn(request.type, zks.getNextZxid(), request, reconfigRequest, true);
+ break;
+ case OpCode.setACL:
+ SetACLRequest setAclRequest = new SetACLRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, setAclRequest, true);
+ break;
+ case OpCode.check:
+ CheckVersionRequest checkRequest = new CheckVersionRequest();
+ pRequest2Txn(request.type, zks.getNextZxid(), request, checkRequest, true);
+ break;
+ case OpCode.multi:
+ MultiTransactionRecord multiRequest = new MultiTransactionRecord();
+ try {
+ ByteBufferInputStream.byteBuffer2Record(request.request, multiRequest);
+ } catch(IOException e) {
+ request.setHdr(new TxnHeader(request.sessionId, request.cxid, zks.getNextZxid(),
+ Time.currentWallTime(), OpCode.multi));
+ throw e;
+ }
+ List<Txn> txns = new ArrayList<Txn>();
+ //Each op in a multi-op must have the same zxid!
+ long zxid = zks.getNextZxid();
+ KeeperException ke = null;
+
+ //Store off current pending change records in case we need to rollback
+ Map<String, ChangeRecord> pendingChanges = getPendingChanges(multiRequest);
+
+ for(Op op: multiRequest) {
+ Record subrequest = op.toRequestRecord();
+ int type;
+ Record txn;
+
+ /* If we've already failed one of the ops, don't bother
+ * trying the rest as we know it's going to fail and it
+ * would be confusing in the logfiles.
+ */
+ if (ke != null) {
+ type = OpCode.error;
+ txn = new ErrorTxn(Code.RUNTIMEINCONSISTENCY.intValue());
+ }
+
+ /* Prep the request and convert to a Txn */
+ else {
+ try {
+ pRequest2Txn(op.getType(), zxid, request, subrequest, false);
+ type = request.getHdr().getType();
Review comment:
*NULL_DEREFERENCE:* object returned by `request.getHdr()` could be null and is dereferenced at line 807.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java
##########
@@ -0,0 +1,1255 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.nio.ByteBuffer;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.security.sasl.SaslException;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.Environment;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.KeeperException.Code;
+import org.apache.zookeeper.KeeperException.SessionExpiredException;
+import org.apache.zookeeper.ZooDefs.OpCode;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.StatPersisted;
+import org.apache.zookeeper.jmx.MBeanRegistry;
+import org.apache.zookeeper.proto.AuthPacket;
+import org.apache.zookeeper.proto.ConnectRequest;
+import org.apache.zookeeper.proto.ConnectResponse;
+import org.apache.zookeeper.proto.GetSASLRequest;
+import org.apache.zookeeper.proto.ReplyHeader;
+import org.apache.zookeeper.proto.RequestHeader;
+import org.apache.zookeeper.proto.SetSASLResponse;
+import org.apache.zookeeper.server.DataTree.ProcessTxnResult;
+import org.apache.zookeeper.server.RequestProcessor.RequestProcessorException;
+import org.apache.zookeeper.server.ServerCnxn.CloseRequestException;
+import org.apache.zookeeper.server.SessionTracker.Session;
+import org.apache.zookeeper.server.SessionTracker.SessionExpirer;
+import org.apache.zookeeper.server.auth.AuthenticationProvider;
+import org.apache.zookeeper.server.auth.ProviderRegistry;
+import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
+import org.apache.zookeeper.server.quorum.ReadOnlyZooKeeperServer;
+import org.apache.zookeeper.txn.CreateSessionTxn;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This class implements a simple standalone ZooKeeperServer. It sets up the
+ * following chain of RequestProcessors to process requests:
+ * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
+ */
+public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
+ protected static final Logger LOG;
+
+ static {
+ LOG = LoggerFactory.getLogger(ZooKeeperServer.class);
+
+ Environment.logEnv("Server environment:", LOG);
+ }
+
+ protected ZooKeeperServerBean jmxServerBean;
+ protected DataTreeBean jmxDataTreeBean;
+
+ public static final int DEFAULT_TICK_TIME = 3000;
+ protected int tickTime = DEFAULT_TICK_TIME;
+ /** value of -1 indicates unset, use default */
+ protected int minSessionTimeout = -1;
+ /** value of -1 indicates unset, use default */
+ protected int maxSessionTimeout = -1;
+ protected SessionTracker sessionTracker;
+ private FileTxnSnapLog txnLogFactory = null;
+ private ZKDatabase zkDb;
+ private final AtomicLong hzxid = new AtomicLong(0);
+ public final static Exception ok = new Exception("No prob");
+ protected RequestProcessor firstProcessor;
+ protected volatile State state = State.INITIAL;
+
+ protected enum State {
+ INITIAL, RUNNING, SHUTDOWN, ERROR
+ }
+
+ /**
+ * This is the secret that we use to generate passwords. For the moment,
+ * it's more of a checksum that's used in reconnection, which carries no
+ * security weight, and is treated internally as if it carries no
+ * security weight.
+ */
+ static final private long superSecret = 0XB3415C00L;
+
+ private final AtomicInteger requestsInProcess = new AtomicInteger(0);
+ final Deque<ChangeRecord> outstandingChanges = new ArrayDeque<>();
+ // this data structure must be accessed under the outstandingChanges lock
+ final HashMap<String, ChangeRecord> outstandingChangesForPath =
+ new HashMap<String, ChangeRecord>();
+
+ protected ServerCnxnFactory serverCnxnFactory;
+ protected ServerCnxnFactory secureServerCnxnFactory;
+
+ private final ServerStats serverStats;
+ private final ZooKeeperServerListener listener;
+ private ZooKeeperServerShutdownHandler zkShutdownHandler;
+ private volatile int createSessionTrackerServerId = 1;
+
+ void removeCnxn(ServerCnxn cnxn) {
+ zkDb.removeCnxn(cnxn);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. Nothing is setup, use the setX
+ * methods to prepare the instance (eg datadir, datalogdir, ticktime,
+ * builder, etc...)
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer() {
+ serverStats = new ServerStats(this);
+ listener = new ZooKeeperServerListenerImpl(this);
+ }
+
+ /**
+ * Creates a ZooKeeperServer instance. It sets everything up, but doesn't
+ * actually start listening for clients until run() is invoked.
+ *
+ * @param dataDir the directory to put the data
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
+ int minSessionTimeout, int maxSessionTimeout, ZKDatabase zkDb) {
+ serverStats = new ServerStats(this);
+ this.txnLogFactory = txnLogFactory;
+ this.txnLogFactory.setServerStats(this.serverStats);
+ this.zkDb = zkDb;
+ this.tickTime = tickTime;
+ setMinSessionTimeout(minSessionTimeout);
+ setMaxSessionTimeout(maxSessionTimeout);
+ listener = new ZooKeeperServerListenerImpl(this);
+ LOG.info("Created server with tickTime " + tickTime
+ + " minSessionTimeout " + getMinSessionTimeout()
+ + " maxSessionTimeout " + getMaxSessionTimeout()
+ + " datadir " + txnLogFactory.getDataDir()
+ + " snapdir " + txnLogFactory.getSnapDir());
+ }
+
+ /**
+ * creates a zookeeperserver instance.
+ * @param txnLogFactory the file transaction snapshot logging class
+ * @param tickTime the ticktime for the server
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime)
+ throws IOException {
+ this(txnLogFactory, tickTime, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ public ServerStats serverStats() {
+ return serverStats;
+ }
+
+ public void dumpConf(PrintWriter pwriter) {
+ pwriter.print("clientPort=");
+ pwriter.println(getClientPort());
+ pwriter.print("secureClientPort=");
+ pwriter.println(getSecureClientPort());
+ pwriter.print("dataDir=");
+ pwriter.println(zkDb.snapLog.getSnapDir().getAbsolutePath());
+ pwriter.print("dataDirSize=");
+ pwriter.println(getDataDirSize());
+ pwriter.print("dataLogDir=");
+ pwriter.println(zkDb.snapLog.getDataDir().getAbsolutePath());
+ pwriter.print("dataLogSize=");
+ pwriter.println(getLogDirSize());
+ pwriter.print("tickTime=");
+ pwriter.println(getTickTime());
+ pwriter.print("maxClientCnxns=");
+ pwriter.println(getMaxClientCnxnsPerHost());
+ pwriter.print("minSessionTimeout=");
+ pwriter.println(getMinSessionTimeout());
+ pwriter.print("maxSessionTimeout=");
+ pwriter.println(getMaxSessionTimeout());
+
+ pwriter.print("serverId=");
+ pwriter.println(getServerId());
+ }
+
+ public ZooKeeperServerConf getConf() {
+ return new ZooKeeperServerConf
+ (getClientPort(),
+ zkDb.snapLog.getSnapDir().getAbsolutePath(),
+ zkDb.snapLog.getDataDir().getAbsolutePath(),
+ getTickTime(),
+ getMaxClientCnxnsPerHost(),
+ getMinSessionTimeout(),
+ getMaxSessionTimeout(),
+ getServerId());
+ }
+
+ /**
+ * This constructor is for backward compatibility with the existing unit
+ * test code.
+ * It defaults to FileLogProvider persistence provider.
+ */
+ public ZooKeeperServer(File snapDir, File logDir, int tickTime)
+ throws IOException {
+ this( new FileTxnSnapLog(snapDir, logDir),
+ tickTime);
+ }
+
+ /**
+ * Default constructor, relies on the config for its argument values
+ *
+ * @throws IOException
+ */
+ public ZooKeeperServer(FileTxnSnapLog txnLogFactory)
+ throws IOException
+ {
+ this(txnLogFactory, DEFAULT_TICK_TIME, -1, -1, new ZKDatabase(txnLogFactory));
+ }
+
+ /**
+ * get the zookeeper database for this server
+ * @return the zookeeper database for this server
+ */
+ public ZKDatabase getZKDatabase() {
+ return this.zkDb;
+ }
+
+ /**
+ * set the zkdatabase for this zookeeper server
+ * @param zkDb
+ */
+ public void setZKDatabase(ZKDatabase zkDb) {
+ this.zkDb = zkDb;
+ }
+
+ /**
+ * Restore sessions and data
+ */
+ public void loadData() throws IOException, InterruptedException {
+ /*
+ * When a new leader starts executing Leader#lead, it
+ * invokes this method. The database, however, has been
+ * initialized before running leader election so that
+ * the server could pick its zxid for its initial vote.
+ * It does it by invoking QuorumPeer#getLastLoggedZxid.
+ * Consequently, we don't need to initialize it once more
+ * and avoid the penalty of loading it a second time. Not
+ * reloading it is particularly important for applications
+ * that host a large database.
+ *
+ * The following if block checks whether the database has
+ * been initialized or not. Note that this method is
+ * invoked by at least one other method:
+ * ZooKeeperServer#startdata.
+ *
+ * See ZOOKEEPER-1642 for more detail.
+ */
+ if(zkDb.isInitialized()){
+ setZxid(zkDb.getDataTreeLastProcessedZxid());
+ }
+ else {
+ setZxid(zkDb.loadDataBase());
+ }
+
+ // Clean up dead sessions
+ LinkedList<Long> deadSessions = new LinkedList<Long>();
+ for (Long session : zkDb.getSessions()) {
+ if (zkDb.getSessionWithTimeOuts().get(session) == null) {
+ deadSessions.add(session);
+ }
+ }
+
+ for (long session : deadSessions) {
+ // XXX: Is lastProcessedZxid really the best thing to use?
+ killSession(session, zkDb.getDataTreeLastProcessedZxid());
+ }
+
+ // Make a clean snapshot
+ takeSnapshot();
+ }
+
+ public void takeSnapshot(){
+ try {
+ txnLogFactory.save(zkDb.getDataTree(), zkDb.getSessionWithTimeOuts());
+ } catch (IOException e) {
+ LOG.error("Severe unrecoverable error, exiting", e);
+ // This is a severe error that we cannot recover from,
+ // so we need to exit
+ System.exit(10);
+ }
+ }
+
+ @Override
+ public long getDataDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getDataDir();
+ return getDirSize(path);
+ }
+
+ @Override
+ public long getLogDirSize() {
+ if (zkDb == null) {
+ return 0L;
+ }
+ File path = zkDb.snapLog.getSnapDir();
+ return getDirSize(path);
+ }
+
+ private long getDirSize(File file) {
+ long size = 0L;
+ if (file.isDirectory()) {
+ File[] files = file.listFiles();
+ if (files != null) {
+ for (File f : files) {
+ size += getDirSize(f);
+ }
+ }
+ } else {
+ size = file.length();
+ }
+ return size;
+ }
+
+ public long getZxid() {
+ return hzxid.get();
+ }
+
+ public SessionTracker getSessionTracker() {
+ return sessionTracker;
+ }
+
+ long getNextZxid() {
+ return hzxid.incrementAndGet();
+ }
+
+ public void setZxid(long zxid) {
+ hzxid.set(zxid);
+ }
+
+ private void close(long sessionId) {
+ Request si = new Request(null, sessionId, 0, OpCode.closeSession, null, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+
+ public void closeSession(long sessionId) {
+ LOG.info("Closing session 0x" + Long.toHexString(sessionId));
+
+ // we do not want to wait for a session close. send it as soon as we
+ // detect it!
+ close(sessionId);
+ }
+
+ protected void killSession(long sessionId, long zxid) {
+ zkDb.killSession(sessionId, zxid);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "ZooKeeperServer --- killSession: 0x"
+ + Long.toHexString(sessionId));
+ }
+ if (sessionTracker != null) {
+ sessionTracker.removeSession(sessionId);
+ }
+ }
+
+ public void expire(Session session) {
+ long sessionId = session.getSessionId();
+ LOG.info("Expiring session 0x" + Long.toHexString(sessionId)
+ + ", timeout of " + session.getTimeout() + "ms exceeded");
+ close(sessionId);
+ }
+
+ public static class MissingSessionException extends IOException {
+ private static final long serialVersionUID = 7467414635467261007L;
+
+ public MissingSessionException(String msg) {
+ super(msg);
+ }
+ }
+
+ void touch(ServerCnxn cnxn) throws MissingSessionException {
+ if (cnxn == null) {
+ return;
+ }
+ long id = cnxn.getSessionId();
+ int to = cnxn.getSessionTimeout();
+ if (!sessionTracker.touchSession(id, to)) {
+ throw new MissingSessionException(
+ "No session with sessionid 0x" + Long.toHexString(id)
+ + " exists, probably expired and removed");
+ }
+ }
+
+ protected void registerJMX() {
+ // register with JMX
+ try {
+ jmxServerBean = new ZooKeeperServerBean(this);
+ MBeanRegistry.getInstance().register(jmxServerBean, null);
+
+ try {
+ jmxDataTreeBean = new DataTreeBean(zkDb.getDataTree());
+ MBeanRegistry.getInstance().register(jmxDataTreeBean, jmxServerBean);
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxDataTreeBean = null;
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ jmxServerBean = null;
+ }
+ }
+
+ public void startdata()
+ throws IOException, InterruptedException {
+ //check to see if zkDb is not null
+ if (zkDb == null) {
+ zkDb = new ZKDatabase(this.txnLogFactory);
+ }
+ if (!zkDb.isInitialized()) {
+ loadData();
+ }
+ }
+
+ public synchronized void startup() {
+ if (sessionTracker == null) {
+ createSessionTracker();
+ }
+ startSessionTracker();
+ setupRequestProcessors();
+
+ registerJMX();
+
+ setState(State.RUNNING);
+ notifyAll();
+ }
+
+ protected void setupRequestProcessors() {
+ RequestProcessor finalProcessor = new FinalRequestProcessor(this);
+ RequestProcessor syncProcessor = new SyncRequestProcessor(this,
+ finalProcessor);
+ ((SyncRequestProcessor)syncProcessor).start();
+ firstProcessor = new PrepRequestProcessor(this, syncProcessor);
+ ((PrepRequestProcessor)firstProcessor).start();
+ }
+
+ public ZooKeeperServerListener getZooKeeperServerListener() {
+ return listener;
+ }
+
+ /**
+ * Change the server ID used by {@link #createSessionTracker()}. Must be called prior to
+ * {@link #startup()} being called
+ *
+ * @param newId ID to use
+ */
+ public void setCreateSessionTrackerServerId(int newId) {
+ createSessionTrackerServerId = newId;
+ }
+
+ protected void createSessionTracker() {
+ sessionTracker = new SessionTrackerImpl(this, zkDb.getSessionWithTimeOuts(),
+ tickTime, createSessionTrackerServerId, getZooKeeperServerListener());
+ }
+
+ protected void startSessionTracker() {
+ ((SessionTrackerImpl)sessionTracker).start();
+ }
+
+ /**
+ * Sets the state of ZooKeeper server. After changing the state, it notifies
+ * the server state change to a registered shutdown handler, if any.
+ * <p>
+ * The following are the server state transitions:
+ * <li>During startup the server will be in the INITIAL state.</li>
+ * <li>After successfully starting, the server sets the state to RUNNING.
+ * </li>
+ * <li>The server transitions to the ERROR state if it hits an internal
+ * error. {@link ZooKeeperServerListenerImpl} notifies any critical resource
+ * error events, e.g., SyncRequestProcessor not being able to write a txn to
+ * disk.</li>
+ * <li>During shutdown the server sets the state to SHUTDOWN, which
+ * corresponds to the server not running.</li>
+ *
+ * @param state new server state.
+ */
+ protected void setState(State state) {
+ this.state = state;
+ // Notify server state changes to the registered shutdown handler, if any.
+ if (zkShutdownHandler != null) {
+ zkShutdownHandler.handle(state);
+ } else {
+ LOG.debug("ZKShutdownHandler is not registered, so ZooKeeper server "
+ + "won't take any action on ERROR or SHUTDOWN server state changes");
+ }
+ }
+
+ /**
+ * This can be used while shutting down the server to see whether the server
+ * is already shutdown or not.
+ *
+ * @return true if the server is running or server hits an error, false
+ * otherwise.
+ */
+ protected boolean canShutdown() {
+ return state == State.RUNNING || state == State.ERROR;
+ }
+
+ /**
+ * @return true if the server is running, false otherwise.
+ */
+ public boolean isRunning() {
+ return state == State.RUNNING;
+ }
+
+ public void shutdown() {
+ shutdown(false);
+ }
+
+ /**
+ * Shut down the server instance
+ * @param fullyShutDown true if another server using the same database will not replace this one in the same process
+ */
+ public synchronized void shutdown(boolean fullyShutDown) {
+ if (!canShutdown()) {
+ LOG.debug("ZooKeeper server is not running, so not proceeding to shutdown!");
+ return;
+ }
+ LOG.info("shutting down");
+
+ // new RuntimeException("Calling shutdown").printStackTrace();
+ setState(State.SHUTDOWN);
+ // Since sessionTracker and syncThreads poll we just have to
+ // set running to false and they will detect it during the poll
+ // interval.
+ if (sessionTracker != null) {
+ sessionTracker.shutdown();
+ }
+ if (firstProcessor != null) {
+ firstProcessor.shutdown();
+ }
+
+ if (zkDb != null) {
+ if (fullyShutDown) {
+ zkDb.clear();
+ } else {
+ // else there is no need to clear the database
+ // * When a new quorum is established we can still apply the diff
+ // on top of the same zkDb data
+ // * If we fetch a new snapshot from leader, the zkDb will be
+ // cleared anyway before loading the snapshot
+ try {
+ //This will fast forward the database to the latest recorded transactions
+ zkDb.fastForwardDataBase();
+ } catch (IOException e) {
+ LOG.error("Error updating DB", e);
+ zkDb.clear();
+ }
+ }
+ }
+
+ unregisterJMX();
+ }
+
+ protected void unregisterJMX() {
+ // unregister from JMX
+ try {
+ if (jmxDataTreeBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxDataTreeBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ try {
+ if (jmxServerBean != null) {
+ MBeanRegistry.getInstance().unregister(jmxServerBean);
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to unregister with JMX", e);
+ }
+ jmxServerBean = null;
+ jmxDataTreeBean = null;
+ }
+
+ public void incInProcess() {
+ requestsInProcess.incrementAndGet();
+ }
+
+ public void decInProcess() {
+ requestsInProcess.decrementAndGet();
+ }
+
+ public int getInProcess() {
+ return requestsInProcess.get();
+ }
+
+ /**
+ * This structure is used to facilitate information sharing between PrepRP
+ * and FinalRP.
+ */
+ static class ChangeRecord {
+ ChangeRecord(long zxid, String path, StatPersisted stat, int childCount,
+ List<ACL> acl) {
+ this.zxid = zxid;
+ this.path = path;
+ this.stat = stat;
+ this.childCount = childCount;
+ this.acl = acl;
+ }
+
+ long zxid;
+
+ String path;
+
+ StatPersisted stat; /* Make sure to create a new object when changing */
+
+ int childCount;
+
+ List<ACL> acl; /* Make sure to create a new object when changing */
+
+ ChangeRecord duplicate(long zxid) {
+ StatPersisted stat = new StatPersisted();
+ if (this.stat != null) {
+ DataTree.copyStatPersisted(this.stat, stat);
+ }
+ return new ChangeRecord(zxid, path, stat, childCount,
+ acl == null ? new ArrayList<ACL>() : new ArrayList<ACL>(acl));
+ }
+ }
+
+ byte[] generatePasswd(long id) {
+ Random r = new Random(id ^ superSecret);
+ byte p[] = new byte[16];
+ r.nextBytes(p);
+ return p;
+ }
+
+ protected boolean checkPasswd(long sessionId, byte[] passwd) {
+ return sessionId != 0
+ && Arrays.equals(passwd, generatePasswd(sessionId));
+ }
+
+ long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
+ if (passwd == null) {
+ // Possible since it's just deserialized from a packet on the wire.
+ passwd = new byte[0];
+ }
+ long sessionId = sessionTracker.createSession(timeout);
+ Random r = new Random(sessionId ^ superSecret);
+ r.nextBytes(passwd);
+ ByteBuffer to = ByteBuffer.allocate(4);
+ to.putInt(timeout);
+ cnxn.setSessionId(sessionId);
+ Request si = new Request(cnxn, sessionId, 0, OpCode.createSession, to, null);
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ return sessionId;
+ }
+
+ /**
+ * set the owner of this session as owner
+ * @param id the session id
+ * @param owner the owner of the session
+ * @throws SessionExpiredException
+ */
+ public void setOwner(long id, Object owner) throws SessionExpiredException {
+ sessionTracker.setOwner(id, owner);
+ }
+
+ protected void revalidateSession(ServerCnxn cnxn, long sessionId,
+ int sessionTimeout) throws IOException {
+ boolean rc = sessionTracker.touchSession(sessionId, sessionTimeout);
+ if (LOG.isTraceEnabled()) {
+ ZooTrace.logTraceMessage(LOG, ZooTrace.SESSION_TRACE_MASK,
+ "Session 0x" + Long.toHexString(sessionId) +
+ " is valid: " + rc);
+ }
+ finishSessionInit(cnxn, rc);
+ }
+
+ public void reopenSession(ServerCnxn cnxn, long sessionId, byte[] passwd,
+ int sessionTimeout) throws IOException {
+ if (checkPasswd(sessionId, passwd)) {
+ revalidateSession(cnxn, sessionId, sessionTimeout);
+ } else {
+ LOG.warn("Incorrect password from " + cnxn.getRemoteSocketAddress()
+ + " for session 0x" + Long.toHexString(sessionId));
+ finishSessionInit(cnxn, false);
+ }
+ }
+
+ public void finishSessionInit(ServerCnxn cnxn, boolean valid) {
+ // register with JMX
+ try {
+ if (valid) {
+ if (serverCnxnFactory != null && serverCnxnFactory.cnxns.contains(cnxn)) {
+ serverCnxnFactory.registerConnection(cnxn);
+ } else if (secureServerCnxnFactory != null && secureServerCnxnFactory.cnxns.contains(cnxn)) {
+ secureServerCnxnFactory.registerConnection(cnxn);
+ }
+ }
+ } catch (Exception e) {
+ LOG.warn("Failed to register with JMX", e);
+ }
+
+ try {
+ ConnectResponse rsp = new ConnectResponse(0, valid ? cnxn.getSessionTimeout()
+ : 0, valid ? cnxn.getSessionId() : 0, // send 0 if session is no
+ // longer valid
+ valid ? generatePasswd(cnxn.getSessionId()) : new byte[16]);
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ BinaryOutputArchive bos = BinaryOutputArchive.getArchive(baos);
+ bos.writeInt(-1, "len");
+ rsp.serialize(bos, "connect");
+ if (!cnxn.isOldClient) {
+ bos.writeBool(
+ this instanceof ReadOnlyZooKeeperServer, "readOnly");
+ }
+ baos.close();
+ ByteBuffer bb = ByteBuffer.wrap(baos.toByteArray());
+ bb.putInt(bb.remaining() - 4).rewind();
+ cnxn.sendBuffer(bb);
+
+ if (valid) {
+ LOG.debug("Established session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " with negotiated timeout " + cnxn.getSessionTimeout()
+ + " for client "
+ + cnxn.getRemoteSocketAddress());
+ cnxn.enableRecv();
+ } else {
+
+ LOG.info("Invalid session 0x"
+ + Long.toHexString(cnxn.getSessionId())
+ + " for client "
+ + cnxn.getRemoteSocketAddress()
+ + ", probably expired");
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ }
+
+ } catch (Exception e) {
+ LOG.warn("Exception while establishing session, closing", e);
+ cnxn.close();
+ }
+ }
+
+ public void closeSession(ServerCnxn cnxn, RequestHeader requestHeader) {
+ closeSession(cnxn.getSessionId());
+ }
+
+ public long getServerId() {
+ return 0;
+ }
+
+ /**
+ * If the underlying Zookeeper server support local session, this method
+ * will set a isLocalSession to true if a request is associated with
+ * a local session.
+ *
+ * @param si
+ */
+ protected void setLocalSessionFlag(Request si) {
+ }
+
+ public void submitRequest(Request si) {
+ if (firstProcessor == null) {
+ synchronized (this) {
+ try {
+ // Since all requests are passed to the request
+ // processor it should wait for setting up the request
+ // processor chain. The state will be updated to RUNNING
+ // after the setup.
+ while (state == State.INITIAL) {
+ wait(1000);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("Unexpected interruption", e);
+ }
+ if (firstProcessor == null || state != State.RUNNING) {
+ throw new RuntimeException("Not started");
+ }
+ }
+ }
+ try {
+ touch(si.cnxn);
+ boolean validpacket = Request.isValid(si.type);
+ if (validpacket) {
+ firstProcessor.processRequest(si);
+ if (si.cnxn != null) {
+ incInProcess();
+ }
+ } else {
+ LOG.warn("Received packet at server of unknown type " + si.type);
+ new UnimplementedRequestProcessor().processRequest(si);
+ }
+ } catch (MissingSessionException e) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Dropping request: " + e.getMessage());
+ }
+ } catch (RequestProcessorException e) {
+ LOG.error("Unable to process request:" + e.getMessage(), e);
+ }
+ }
+
+ public static int getSnapCount() {
+ String sc = System.getProperty("zookeeper.snapCount");
+ try {
+ int snapCount = Integer.parseInt(sc);
+
+ // snapCount must be 2 or more. See org.apache.zookeeper.server.SyncRequestProcessor
+ if( snapCount < 2 ) {
+ LOG.warn("SnapCount should be 2 or more. Now, snapCount is reset to 2");
+ snapCount = 2;
+ }
+ return snapCount;
+ } catch (Exception e) {
+ return 100000;
+ }
+ }
+
+ public int getGlobalOutstandingLimit() {
+ String sc = System.getProperty("zookeeper.globalOutstandingLimit");
+ int limit;
+ try {
+ limit = Integer.parseInt(sc);
+ } catch (Exception e) {
+ limit = 1000;
+ }
+ return limit;
+ }
+
+ public void setServerCnxnFactory(ServerCnxnFactory factory) {
+ serverCnxnFactory = factory;
+ }
+
+ public ServerCnxnFactory getServerCnxnFactory() {
+ return serverCnxnFactory;
+ }
+
+ public ServerCnxnFactory getSecureServerCnxnFactory() {
+ return secureServerCnxnFactory;
+ }
+
+ public void setSecureServerCnxnFactory(ServerCnxnFactory factory) {
+ secureServerCnxnFactory = factory;
+ }
+
+ /**
+ * return the last proceesed id from the
+ * datatree
+ */
+ public long getLastProcessedZxid() {
+ return zkDb.getDataTreeLastProcessedZxid();
+ }
+
+ /**
+ * return the outstanding requests
+ * in the queue, which havent been
+ * processed yet
+ */
+ public long getOutstandingRequests() {
+ return getInProcess();
+ }
+
+ /**
+ * return the total number of client connections that are alive
+ * to this server
+ */
+ public int getNumAliveConnections() {
+ int numAliveConnections = 0;
+
+ if (serverCnxnFactory != null) {
+ numAliveConnections += serverCnxnFactory.getNumAliveConnections();
+ }
+
+ if (secureServerCnxnFactory != null) {
+ numAliveConnections += secureServerCnxnFactory.getNumAliveConnections();
+ }
+
+ return numAliveConnections;
+ }
+
+ /**
+ * trunccate the log to get in sync with others
+ * if in a quorum
+ * @param zxid the zxid that it needs to get in sync
+ * with others
+ * @throws IOException
+ */
+ public void truncateLog(long zxid) throws IOException {
+ this.zkDb.truncateLog(zxid);
+ }
+
+ public int getTickTime() {
+ return tickTime;
+ }
+
+ public void setTickTime(int tickTime) {
+ LOG.info("tickTime set to " + tickTime);
+ this.tickTime = tickTime;
+ }
+
+ public int getMinSessionTimeout() {
+ return minSessionTimeout;
+ }
+
+ public void setMinSessionTimeout(int min) {
+ this.minSessionTimeout = min == -1 ? tickTime * 2 : min;
+ LOG.info("minSessionTimeout set to {}", this.minSessionTimeout);
+ }
+
+ public int getMaxSessionTimeout() {
+ return maxSessionTimeout;
+ }
+
+ public void setMaxSessionTimeout(int max) {
+ this.maxSessionTimeout = max == -1 ? tickTime * 20 : max;
+ LOG.info("maxSessionTimeout set to {}", this.maxSessionTimeout);
+ }
+
+ public int getClientPort() {
+ return serverCnxnFactory != null ? serverCnxnFactory.getLocalPort() : -1;
+ }
+
+ public int getSecureClientPort() {
+ return secureServerCnxnFactory != null ? secureServerCnxnFactory.getLocalPort() : -1;
+ }
+
+ /** Maximum number of connections allowed from particular host (ip) */
+ public int getMaxClientCnxnsPerHost() {
+ if (serverCnxnFactory != null) {
+ return serverCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ if (secureServerCnxnFactory != null) {
+ return secureServerCnxnFactory.getMaxClientCnxnsPerHost();
+ }
+ return -1;
+ }
+
+ public void setTxnLogFactory(FileTxnSnapLog txnLog) {
+ this.txnLogFactory = txnLog;
+ }
+
+ public FileTxnSnapLog getTxnLogFactory() {
+ return this.txnLogFactory;
+ }
+
+ /**
+ * Returns the elapsed sync of time of transaction log in milliseconds.
+ */
+ public long getTxnLogElapsedSyncTime() {
+ return txnLogFactory.getTxnLogElapsedSyncTime();
+ }
+
+ public String getState() {
+ return "standalone";
+ }
+
+ public void dumpEphemerals(PrintWriter pwriter) {
+ zkDb.dumpEphemerals(pwriter);
+ }
+
+ public Map<Long, Set<String>> getEphemerals() {
+ return zkDb.getEphemerals();
+ }
+
+ public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
+ BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
+ ConnectRequest connReq = new ConnectRequest();
+ connReq.deserialize(bia, "connect");
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Session establishment request from client "
+ + cnxn.getRemoteSocketAddress()
+ + " client's lastZxid is 0x"
+ + Long.toHexString(connReq.getLastZxidSeen()));
+ }
+ boolean readOnly = false;
+ try {
+ readOnly = bia.readBool("readOnly");
+ cnxn.isOldClient = false;
+ } catch (IOException e) {
+ // this is ok -- just a packet from an old client which
+ // doesn't contain readOnly field
+ LOG.warn("Connection request from old client "
+ + cnxn.getRemoteSocketAddress()
+ + "; will be dropped if server is in r-o mode");
+ }
+ if (!readOnly && this instanceof ReadOnlyZooKeeperServer) {
+ String msg = "Refusing session request for not-read-only client "
+ + cnxn.getRemoteSocketAddress();
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ if (connReq.getLastZxidSeen() > zkDb.dataTree.lastProcessedZxid) {
+ String msg = "Refusing session request for client "
+ + cnxn.getRemoteSocketAddress()
+ + " as it has seen zxid 0x"
+ + Long.toHexString(connReq.getLastZxidSeen())
+ + " our last zxid is 0x"
+ + Long.toHexString(getZKDatabase().getDataTreeLastProcessedZxid())
+ + " client must try another server";
+
+ LOG.info(msg);
+ throw new CloseRequestException(msg);
+ }
+ int sessionTimeout = connReq.getTimeOut();
+ byte passwd[] = connReq.getPasswd();
+ int minSessionTimeout = getMinSessionTimeout();
+ if (sessionTimeout < minSessionTimeout) {
+ sessionTimeout = minSessionTimeout;
+ }
+ int maxSessionTimeout = getMaxSessionTimeout();
+ if (sessionTimeout > maxSessionTimeout) {
+ sessionTimeout = maxSessionTimeout;
+ }
+ cnxn.setSessionTimeout(sessionTimeout);
+ // We don't want to receive any packets until we are sure that the
+ // session is setup
+ cnxn.disableRecv();
+ long sessionId = connReq.getSessionId();
+ if (sessionId == 0) {
+ long id = createSession(cnxn, passwd, sessionTimeout);
+ LOG.debug("Client attempting to establish new session:" +
+ " session = 0x{}, zxid = 0x{}, timeout = {}, address = {}",
+ Long.toHexString(id),
+ Long.toHexString(connReq.getLastZxidSeen()),
+ connReq.getTimeOut(),
+ cnxn.getRemoteSocketAddress());
+ } else {
+ long clientSessionId = connReq.getSessionId();
+ LOG.debug("Client attempting to renew session:" +
+ " session = 0x{}, zxid = 0x{}, timeout = {}, address = {}",
+ Long.toHexString(clientSessionId),
+ Long.toHexString(connReq.getLastZxidSeen()),
+ connReq.getTimeOut(),
+ cnxn.getRemoteSocketAddress());
+ if (serverCnxnFactory != null) {
+ serverCnxnFactory.closeSession(sessionId);
+ }
+ if (secureServerCnxnFactory != null) {
+ secureServerCnxnFactory.closeSession(sessionId);
+ }
+ cnxn.setSessionId(sessionId);
+ reopenSession(cnxn, sessionId, passwd, sessionTimeout);
+ }
+ }
+
+ public boolean shouldThrottle(long outStandingCount) {
+ if (getGlobalOutstandingLimit() < getInProcess()) {
+ return outStandingCount > 0;
+ }
+ return false;
+ }
+
+ public void processPacket(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
+ // We have the request, now process and setup for next
+ InputStream bais = new ByteBufferInputStream(incomingBuffer);
+ BinaryInputArchive bia = BinaryInputArchive.getArchive(bais);
+ RequestHeader h = new RequestHeader();
+ h.deserialize(bia, "header");
+ // Through the magic of byte buffers, txn will not be
+ // pointing
+ // to the start of the txn
+ incomingBuffer = incomingBuffer.slice();
+ if (h.getType() == OpCode.auth) {
+ LOG.info("got auth packet " + cnxn.getRemoteSocketAddress());
+ AuthPacket authPacket = new AuthPacket();
+ ByteBufferInputStream.byteBuffer2Record(incomingBuffer, authPacket);
+ String scheme = authPacket.getScheme();
+ AuthenticationProvider ap = ProviderRegistry.getProvider(scheme);
+ Code authReturn = KeeperException.Code.AUTHFAILED;
+ if(ap != null) {
+ try {
+ authReturn = ap.handleAuthentication(cnxn, authPacket.getAuth());
+ } catch(RuntimeException e) {
+ LOG.warn("Caught runtime exception from AuthenticationProvider: " + scheme + " due to " + e);
+ authReturn = KeeperException.Code.AUTHFAILED;
+ }
+ }
+ if (authReturn == KeeperException.Code.OK) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Authentication succeeded for scheme: " + scheme);
+ }
+ LOG.info("auth success " + cnxn.getRemoteSocketAddress());
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0,
+ KeeperException.Code.OK.intValue());
+ cnxn.sendResponse(rh, null, null);
+ } else {
+ if (ap == null) {
+ LOG.warn("No authentication provider for scheme: "
+ + scheme + " has "
+ + ProviderRegistry.listProviders());
+ } else {
+ LOG.warn("Authentication failed for scheme: " + scheme);
+ }
+ // send a response...
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0,
+ KeeperException.Code.AUTHFAILED.intValue());
+ cnxn.sendResponse(rh, null, null);
+ // ... and close connection
+ cnxn.sendBuffer(ServerCnxnFactory.closeConn);
+ cnxn.disableRecv();
+ }
+ return;
+ } else {
+ if (h.getType() == OpCode.sasl) {
+ Record rsp = processSasl(incomingBuffer,cnxn);
+ ReplyHeader rh = new ReplyHeader(h.getXid(), 0, KeeperException.Code.OK.intValue());
+ cnxn.sendResponse(rh,rsp, "response"); // not sure about 3rd arg..what is it?
+ return;
+ }
+ else {
+ Request si = new Request(cnxn, cnxn.getSessionId(), h.getXid(),
+ h.getType(), incomingBuffer, cnxn.getAuthInfo());
+ si.setOwner(ServerCnxn.me);
+ // Always treat packet from the client as a possible
+ // local request.
+ setLocalSessionFlag(si);
+ submitRequest(si);
+ }
+ }
+ cnxn.incrOutstandingRequests(h);
+ }
+
+ private Record processSasl(ByteBuffer incomingBuffer, ServerCnxn cnxn) throws IOException {
+ LOG.debug("Responding to client SASL token.");
+ GetSASLRequest clientTokenRecord = new GetSASLRequest();
+ ByteBufferInputStream.byteBuffer2Record(incomingBuffer,clientTokenRecord);
+ byte[] clientToken = clientTokenRecord.getToken();
+ LOG.debug("Size of client SASL token: " + clientToken.length);
Review comment:
*NULL_DEREFERENCE:* object `clientToken` last assigned on line 1158 could be null and is dereferenced at line 1159.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/persistence/FileTxnLog.java
##########
@@ -0,0 +1,746 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.persistence;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.Closeable;
+import java.io.EOFException;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.FilterInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.zip.Adler32;
+import java.util.zip.Checksum;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.BinaryOutputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.jute.OutputArchive;
+import org.apache.jute.Record;
+import org.apache.zookeeper.server.ServerStats;
+import org.apache.zookeeper.server.util.SerializeUtils;
+import org.apache.zookeeper.txn.TxnHeader;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This class implements the TxnLog interface. It provides api's
+ * to access the txnlogs and add entries to it.
+ * <p>
+ * The format of a Transactional log is as follows:
+ * <blockquote><pre>
+ * LogFile:
+ * FileHeader TxnList ZeroPad
+ *
+ * FileHeader: {
+ * magic 4bytes (ZKLG)
+ * version 4bytes
+ * dbid 8bytes
+ * }
+ *
+ * TxnList:
+ * Txn || Txn TxnList
+ *
+ * Txn:
+ * checksum Txnlen TxnHeader Record 0x42
+ *
+ * checksum: 8bytes Adler32 is currently used
+ * calculated across payload -- Txnlen, TxnHeader, Record and 0x42
+ *
+ * Txnlen:
+ * len 4bytes
+ *
+ * TxnHeader: {
+ * sessionid 8bytes
+ * cxid 4bytes
+ * zxid 8bytes
+ * time 8bytes
+ * type 4bytes
+ * }
+ *
+ * Record:
+ * See Jute definition file for details on the various record types
+ *
+ * ZeroPad:
+ * 0 padded to EOF (filled during preallocation stage)
+ * </pre></blockquote>
+ */
+public class FileTxnLog implements TxnLog, Closeable {
+
+ private static final Logger LOG;
+
+ public final static int TXNLOG_MAGIC =
+ ByteBuffer.wrap("ZKLG".getBytes()).getInt();
+
+ public final static int VERSION = 2;
+
+ public static final String LOG_FILE_PREFIX = "log";
+
+ static final String FSYNC_WARNING_THRESHOLD_MS_PROPERTY = "fsync.warningthresholdms";
+ static final String ZOOKEEPER_FSYNC_WARNING_THRESHOLD_MS_PROPERTY = "zookeeper." + FSYNC_WARNING_THRESHOLD_MS_PROPERTY;
+
+ /** Maximum time we allow for elapsed fsync before WARNing */
+ private final static long fsyncWarningThresholdMS;
+
+ static {
+ LOG = LoggerFactory.getLogger(FileTxnLog.class);
+
+ /** Local variable to read fsync.warningthresholdms into */
+ Long fsyncWarningThreshold;
+ if ((fsyncWarningThreshold = Long.getLong(ZOOKEEPER_FSYNC_WARNING_THRESHOLD_MS_PROPERTY)) == null)
+ fsyncWarningThreshold = Long.getLong(FSYNC_WARNING_THRESHOLD_MS_PROPERTY, 1000);
+ fsyncWarningThresholdMS = fsyncWarningThreshold;
+ }
+
+ long lastZxidSeen;
+ volatile BufferedOutputStream logStream = null;
+ volatile OutputArchive oa;
+ volatile FileOutputStream fos = null;
+
+ File logDir;
+ private final boolean forceSync = !System.getProperty("zookeeper.forceSync", "yes").equals("no");
+ long dbId;
+ private LinkedList<FileOutputStream> streamsToFlush =
+ new LinkedList<FileOutputStream>();
+ File logFileWrite = null;
+ private FilePadding filePadding = new FilePadding();
+
+ private ServerStats serverStats;
+
+ private volatile long syncElapsedMS = -1L;
+
+ /**
+ * constructor for FileTxnLog. Take the directory
+ * where the txnlogs are stored
+ * @param logDir the directory where the txnlogs are stored
+ */
+ public FileTxnLog(File logDir) {
+ this.logDir = logDir;
+ }
+
+ /**
+ * method to allow setting preallocate size
+ * of log file to pad the file.
+ * @param size the size to set to in bytes
+ */
+ public static void setPreallocSize(long size) {
+ FilePadding.setPreallocSize(size);
+ }
+
+ /**
+ * Setter for ServerStats to monitor fsync threshold exceed
+ * @param serverStats used to update fsyncThresholdExceedCount
+ */
+ @Override
+ public synchronized void setServerStats(ServerStats serverStats) {
+ this.serverStats = serverStats;
+ }
+
+ /**
+ * creates a checksum algorithm to be used
+ * @return the checksum used for this txnlog
+ */
+ protected Checksum makeChecksumAlgorithm(){
+ return new Adler32();
+ }
+
+ /**
+ * rollover the current log file to a new one.
+ * @throws IOException
+ */
+ public synchronized void rollLog() throws IOException {
+ if (logStream != null) {
+ this.logStream.flush();
+ this.logStream = null;
+ oa = null;
+ }
+ }
+
+ /**
+ * close all the open file handles
+ * @throws IOException
+ */
+ public synchronized void close() throws IOException {
+ if (logStream != null) {
+ logStream.close();
+ }
+ for (FileOutputStream log : streamsToFlush) {
+ log.close();
+ }
+ }
+
+ /**
+ * append an entry to the transaction log
+ * @param hdr the header of the transaction
+ * @param txn the transaction part of the entry
+ * returns true iff something appended, otw false
+ */
+ public synchronized boolean append(TxnHeader hdr, Record txn)
+ throws IOException
+ {
+ if (hdr == null) {
+ return false;
+ }
+ if (hdr.getZxid() <= lastZxidSeen) {
+ LOG.warn("Current zxid " + hdr.getZxid()
+ + " is <= " + lastZxidSeen + " for "
+ + hdr.getType());
+ } else {
+ lastZxidSeen = hdr.getZxid();
+ }
+ if (logStream==null) {
+ if(LOG.isInfoEnabled()){
+ LOG.info("Creating new log file: " + Util.makeLogName(hdr.getZxid()));
+ }
+
+ logFileWrite = new File(logDir, Util.makeLogName(hdr.getZxid()));
+ fos = new FileOutputStream(logFileWrite);
+ logStream=new BufferedOutputStream(fos);
+ oa = BinaryOutputArchive.getArchive(logStream);
+ FileHeader fhdr = new FileHeader(TXNLOG_MAGIC,VERSION, dbId);
+ fhdr.serialize(oa, "fileheader");
+ // Make sure that the magic number is written before padding.
+ logStream.flush();
+ filePadding.setCurrentSize(fos.getChannel().position());
+ streamsToFlush.add(fos);
+ }
+ filePadding.padFile(fos.getChannel());
+ byte[] buf = Util.marshallTxnEntry(hdr, txn);
+ if (buf == null || buf.length == 0) {
+ throw new IOException("Faulty serialization for header " +
+ "and txn");
+ }
+ Checksum crc = makeChecksumAlgorithm();
+ crc.update(buf, 0, buf.length);
+ oa.writeLong(crc.getValue(), "txnEntryCRC");
+ Util.writeTxnBytes(oa, buf);
+
+ return true;
+ }
+
+ /**
+ * Find the log file that starts at, or just before, the snapshot. Return
+ * this and all subsequent logs. Results are ordered by zxid of file,
+ * ascending order.
+ * @param logDirList array of files
+ * @param snapshotZxid return files at, or before this zxid
+ * @return
+ */
+ public static File[] getLogFiles(File[] logDirList,long snapshotZxid) {
+ List<File> files = Util.sortDataDir(logDirList, LOG_FILE_PREFIX, true);
+ long logZxid = 0;
+ // Find the log file that starts before or at the same time as the
+ // zxid of the snapshot
+ for (File f : files) {
+ long fzxid = Util.getZxidFromName(f.getName(), LOG_FILE_PREFIX);
+ if (fzxid > snapshotZxid) {
+ continue;
+ }
+ // the files
+ // are sorted with zxid's
+ if (fzxid > logZxid) {
+ logZxid = fzxid;
+ }
+ }
+ List<File> v=new ArrayList<File>(5);
+ for (File f : files) {
+ long fzxid = Util.getZxidFromName(f.getName(), LOG_FILE_PREFIX);
+ if (fzxid < logZxid) {
+ continue;
+ }
+ v.add(f);
+ }
+ return v.toArray(new File[0]);
+
+ }
+
+ /**
+ * get the last zxid that was logged in the transaction logs
+ * @return the last zxid logged in the transaction logs
+ */
+ public long getLastLoggedZxid() {
+ File[] files = getLogFiles(logDir.listFiles(), 0);
+ long maxLog=files.length>0?
+ Util.getZxidFromName(files[files.length-1].getName(),LOG_FILE_PREFIX):-1;
+
+ // if a log file is more recent we must scan it to find
+ // the highest zxid
+ long zxid = maxLog;
+ TxnIterator itr = null;
+ try {
+ FileTxnLog txn = new FileTxnLog(logDir);
+ itr = txn.read(maxLog);
+ while (true) {
+ if(!itr.next())
+ break;
+ TxnHeader hdr = itr.getHeader();
+ zxid = hdr.getZxid();
+ }
+ } catch (IOException e) {
+ LOG.warn("Unexpected exception", e);
+ } finally {
+ close(itr);
+ }
+ return zxid;
+ }
+
+ private void close(TxnIterator itr) {
+ if (itr != null) {
+ try {
+ itr.close();
+ } catch (IOException ioe) {
+ LOG.warn("Error closing file iterator", ioe);
+ }
+ }
+ }
+
+ /**
+ * commit the logs. make sure that everything hits the
+ * disk
+ */
+ public synchronized void commit() throws IOException {
+ if (logStream != null) {
+ logStream.flush();
+ }
+ for (FileOutputStream log : streamsToFlush) {
+ log.flush();
+ if (forceSync) {
+ long startSyncNS = System.nanoTime();
+
+ FileChannel channel = log.getChannel();
+ channel.force(false);
+
+ syncElapsedMS = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startSyncNS);
+ if (syncElapsedMS > fsyncWarningThresholdMS) {
+ if(serverStats != null) {
+ serverStats.incrementFsyncThresholdExceedCount();
+ }
+ LOG.warn("fsync-ing the write ahead log in "
+ + Thread.currentThread().getName()
+ + " took " + syncElapsedMS
+ + "ms which will adversely effect operation latency. "
+ + "File size is " + channel.size() + " bytes. "
+ + "See the ZooKeeper troubleshooting guide");
+ }
+ }
+ }
+ while (streamsToFlush.size() > 1) {
+ streamsToFlush.removeFirst().close();
+ }
+ }
+
+ /**
+ *
+ * @return elapsed sync time of transaction log in milliseconds
+ */
+ public long getTxnLogSyncElapsedTime() {
+ return syncElapsedMS;
+ }
+
+ /**
+ * start reading all the transactions from the given zxid
+ * @param zxid the zxid to start reading transactions from
+ * @return returns an iterator to iterate through the transaction
+ * logs
+ */
+ public TxnIterator read(long zxid) throws IOException {
+ return read(zxid, true);
+ }
+
+ /**
+ * start reading all the transactions from the given zxid.
+ *
+ * @param zxid the zxid to start reading transactions from
+ * @param fastForward true if the iterator should be fast forwarded to point
+ * to the txn of a given zxid, else the iterator will point to the
+ * starting txn of a txnlog that may contain txn of a given zxid
+ * @return returns an iterator to iterate through the transaction logs
+ */
+ public TxnIterator read(long zxid, boolean fastForward) throws IOException {
+ return new FileTxnIterator(logDir, zxid, fastForward);
+ }
+
+ /**
+ * truncate the current transaction logs
+ * @param zxid the zxid to truncate the logs to
+ * @return true if successful false if not
+ */
+ public boolean truncate(long zxid) throws IOException {
+ FileTxnIterator itr = null;
+ try {
+ itr = new FileTxnIterator(this.logDir, zxid);
+ PositionInputStream input = itr.inputStream;
+ if(input == null) {
+ throw new IOException("No log files found to truncate! This could " +
+ "happen if you still have snapshots from an old setup or " +
+ "log files were deleted accidentally or dataLogDir was changed in zoo.cfg.");
+ }
+ long pos = input.getPosition();
+ // now, truncate at the current position
+ RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
+ raf.setLength(pos);
+ raf.close();
+ while(itr.goToNextLog()) {
+ if (!itr.logFile.delete()) {
+ LOG.warn("Unable to truncate {}", itr.logFile);
+ }
+ }
+ } finally {
+ close(itr);
+ }
+ return true;
+ }
+
+ /**
+ * read the header of the transaction file
+ * @param file the transaction file to read
+ * @return header that was read from the file
+ * @throws IOException
+ */
+ private static FileHeader readHeader(File file) throws IOException {
+ InputStream is =null;
+ try {
+ is = new BufferedInputStream(new FileInputStream(file));
+ InputArchive ia=BinaryInputArchive.getArchive(is);
+ FileHeader hdr = new FileHeader();
+ hdr.deserialize(ia, "fileheader");
+ return hdr;
+ } finally {
+ try {
+ if (is != null) is.close();
+ } catch (IOException e) {
+ LOG.warn("Ignoring exception during close", e);
+ }
+ }
+ }
+
+ /**
+ * the dbid of this transaction database
+ * @return the dbid of this database
+ */
+ public long getDbId() throws IOException {
+ FileTxnIterator itr = new FileTxnIterator(logDir, 0);
+ FileHeader fh=readHeader(itr.logFile);
+ itr.close();
+ if(fh==null)
+ throw new IOException("Unsupported Format.");
+ return fh.getDbid();
+ }
+
+ /**
+ * the forceSync value. true if forceSync is enabled, false otherwise.
+ * @return the forceSync value
+ */
+ public boolean isForceSync() {
+ return forceSync;
+ }
+
+ /**
+ * a class that keeps track of the position
+ * in the input stream. The position points to offset
+ * that has been consumed by the applications. It can
+ * wrap buffered input streams to provide the right offset
+ * for the application.
+ */
+ static class PositionInputStream extends FilterInputStream {
+ long position;
+ protected PositionInputStream(InputStream in) {
+ super(in);
+ position = 0;
+ }
+
+ @Override
+ public int read() throws IOException {
+ int rc = super.read();
+ if (rc > -1) {
+ position++;
+ }
+ return rc;
+ }
+
+ public int read(byte[] b) throws IOException {
+ int rc = super.read(b);
+ if (rc > 0) {
+ position += rc;
+ }
+ return rc;
+ }
+
+ @Override
+ public int read(byte[] b, int off, int len) throws IOException {
+ int rc = super.read(b, off, len);
+ if (rc > 0) {
+ position += rc;
+ }
+ return rc;
+ }
+
+ @Override
+ public long skip(long n) throws IOException {
+ long rc = super.skip(n);
+ if (rc > 0) {
+ position += rc;
+ }
+ return rc;
+ }
+ public long getPosition() {
+ return position;
+ }
+
+ @Override
+ public boolean markSupported() {
+ return false;
+ }
+
+ @Override
+ public void mark(int readLimit) {
+ throw new UnsupportedOperationException("mark");
+ }
+
+ @Override
+ public void reset() {
+ throw new UnsupportedOperationException("reset");
+ }
+ }
+
+ /**
+ * this class implements the txnlog iterator interface
+ * which is used for reading the transaction logs
+ */
+ public static class FileTxnIterator implements TxnLog.TxnIterator {
+ File logDir;
+ long zxid;
+ TxnHeader hdr;
+ Record record;
+ File logFile;
+ InputArchive ia;
+ static final String CRC_ERROR="CRC check failed";
+
+ PositionInputStream inputStream=null;
+ //stored files is the list of files greater than
+ //the zxid we are looking for.
+ private ArrayList<File> storedFiles;
+
+ /**
+ * create an iterator over a transaction database directory
+ * @param logDir the transaction database directory
+ * @param zxid the zxid to start reading from
+ * @param fastForward true if the iterator should be fast forwarded to
+ * point to the txn of a given zxid, else the iterator will
+ * point to the starting txn of a txnlog that may contain txn of
+ * a given zxid
+ * @throws IOException
+ */
+ public FileTxnIterator(File logDir, long zxid, boolean fastForward)
+ throws IOException {
+ this.logDir = logDir;
+ this.zxid = zxid;
+ init();
+
+ if (fastForward && hdr != null) {
+ while (hdr.getZxid() < zxid) {
+ if (!next())
Review comment:
*NULL_DEREFERENCE:* object `FileTxnLog$FileTxnIterator.inputStream` last assigned on line 544 could be null and is dereferenced by call to `next()` at line 567.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java
##########
@@ -0,0 +1,823 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server.quorum;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.io.StringReader;
+import java.io.Writer;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Map.Entry;
+
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.apache.zookeeper.common.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.slf4j.MDC;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom.OutputStreamStatement;
+import org.apache.zookeeper.common.AtomicFileWritingIdiom.WriterStatement;
+import org.apache.zookeeper.common.PathUtils;
+import org.apache.zookeeper.server.ZooKeeperServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.LearnerType;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.auth.QuorumAuth;
+import org.apache.zookeeper.server.quorum.flexible.QuorumHierarchical;
+import org.apache.zookeeper.server.quorum.flexible.QuorumMaj;
+import org.apache.zookeeper.server.quorum.flexible.QuorumVerifier;
+import org.apache.zookeeper.server.util.VerifyingFileFactory;
+
+import static org.apache.zookeeper.common.NetUtils.formatInetAddr;
+
+@InterfaceAudience.Public
+public class QuorumPeerConfig {
+ private static final Logger LOG = LoggerFactory.getLogger(QuorumPeerConfig.class);
+ private static final int UNSET_SERVERID = -1;
+ public static final String nextDynamicConfigFileSuffix = ".dynamic.next";
+
+ private static boolean standaloneEnabled = true;
+ private static boolean reconfigEnabled = false;
+
+ protected InetSocketAddress clientPortAddress;
+ protected InetSocketAddress secureClientPortAddress;
+ protected boolean sslQuorum = false;
+ protected boolean shouldUsePortUnification = false;
+ protected boolean sslQuorumReloadCertFiles = false;
+ protected File dataDir;
+ protected File dataLogDir;
+ protected String dynamicConfigFileStr = null;
+ protected String configFileStr = null;
+ protected int tickTime = ZooKeeperServer.DEFAULT_TICK_TIME;
+ protected int maxClientCnxns = 60;
+ /** defaults to -1 if not set explicitly */
+ protected int minSessionTimeout = -1;
+ /** defaults to -1 if not set explicitly */
+ protected int maxSessionTimeout = -1;
+ protected boolean localSessionsEnabled = false;
+ protected boolean localSessionsUpgradingEnabled = false;
+
+ protected int initLimit;
+ protected int syncLimit;
+ protected int electionAlg = 3;
+ protected int electionPort = 2182;
+ protected boolean quorumListenOnAllIPs = false;
+
+ protected long serverId = UNSET_SERVERID;
+
+ protected QuorumVerifier quorumVerifier = null, lastSeenQuorumVerifier = null;
+ protected int snapRetainCount = 3;
+ protected int purgeInterval = 0;
+ protected boolean syncEnabled = true;
+
+ protected LearnerType peerType = LearnerType.PARTICIPANT;
+
+ /**
+ * Configurations for the quorumpeer-to-quorumpeer sasl authentication
+ */
+ protected boolean quorumServerRequireSasl = false;
+ protected boolean quorumLearnerRequireSasl = false;
+ protected boolean quorumEnableSasl = false;
+ protected String quorumServicePrincipal = QuorumAuth.QUORUM_KERBEROS_SERVICE_PRINCIPAL_DEFAULT_VALUE;
+ protected String quorumLearnerLoginContext = QuorumAuth.QUORUM_LEARNER_SASL_LOGIN_CONTEXT_DFAULT_VALUE;
+ protected String quorumServerLoginContext = QuorumAuth.QUORUM_SERVER_SASL_LOGIN_CONTEXT_DFAULT_VALUE;
+ protected int quorumCnxnThreadsSize;
+
+ /**
+ * Minimum snapshot retain count.
+ * @see org.apache.zookeeper.server.PurgeTxnLog#purge(File, File, int)
+ */
+ private final int MIN_SNAP_RETAIN_COUNT = 3;
+
+ @SuppressWarnings("serial")
+ public static class ConfigException extends Exception {
+ public ConfigException(String msg) {
+ super(msg);
+ }
+ public ConfigException(String msg, Exception e) {
+ super(msg, e);
+ }
+ }
+
+ /**
+ * Parse a ZooKeeper configuration file
+ * @param path the patch of the configuration file
+ * @throws ConfigException error processing configuration
+ */
+ public void parse(String path) throws ConfigException {
+ LOG.info("Reading configuration from: " + path);
+
+ try {
+ File configFile = (new VerifyingFileFactory.Builder(LOG)
+ .warnForRelativePath()
+ .failForNonExistingPath()
+ .build()).create(path);
+
+ Properties cfg = new Properties();
+ FileInputStream in = new FileInputStream(configFile);
+ try {
+ cfg.load(in);
+ configFileStr = path;
+ } finally {
+ in.close();
+ }
+ //解析配置文件
+ parseProperties(cfg);
+ } catch (IOException e) {
+ throw new ConfigException("Error processing " + path, e);
+ } catch (IllegalArgumentException e) {
+ throw new ConfigException("Error processing " + path, e);
+ }
+
+ if (dynamicConfigFileStr!=null) {
+ try {
+ Properties dynamicCfg = new Properties();
+ FileInputStream inConfig = new FileInputStream(dynamicConfigFileStr);
+ try {
+ dynamicCfg.load(inConfig);
+ if (dynamicCfg.getProperty("version") != null) {
+ throw new ConfigException("dynamic file shouldn't have version inside");
+ }
+
+ String version = getVersionFromFilename(dynamicConfigFileStr);
+ // If there isn't any version associated with the filename,
+ // the default version is 0.
+ if (version != null) {
+ dynamicCfg.setProperty("version", version);
+ }
+ } finally {
+ inConfig.close();
+ }
+ setupQuorumPeerConfig(dynamicCfg, false);
+
+ } catch (IOException e) {
+ throw new ConfigException("Error processing " + dynamicConfigFileStr, e);
+ } catch (IllegalArgumentException e) {
+ throw new ConfigException("Error processing " + dynamicConfigFileStr, e);
+ }
+ File nextDynamicConfigFile = new File(configFileStr + nextDynamicConfigFileSuffix);
+ if (nextDynamicConfigFile.exists()) {
+ try {
+ Properties dynamicConfigNextCfg = new Properties();
+ FileInputStream inConfigNext = new FileInputStream(nextDynamicConfigFile);
+ try {
+ dynamicConfigNextCfg.load(inConfigNext);
+ } finally {
+ inConfigNext.close();
+ }
+ boolean isHierarchical = false;
+ for (Entry<Object, Object> entry : dynamicConfigNextCfg.entrySet()) {
+ String key = entry.getKey().toString().trim();
+ if (key.startsWith("group") || key.startsWith("weight")) {
+ isHierarchical = true;
+ break;
+ }
+ }
+ lastSeenQuorumVerifier = createQuorumVerifier(dynamicConfigNextCfg, isHierarchical);
+ } catch (IOException e) {
+ LOG.warn("NextQuorumVerifier is initiated to null");
+ }
+ }
+ }
+ }
+
+ // This method gets the version from the end of dynamic file name.
+ // For example, "zoo.cfg.dynamic.0" returns initial version "0".
+ // "zoo.cfg.dynamic.1001" returns version of hex number "0x1001".
+ // If a dynamic file name doesn't have any version at the end of file,
+ // e.g. "zoo.cfg.dynamic", it returns null.
+ public static String getVersionFromFilename(String filename) {
+ int i = filename.lastIndexOf('.');
+ if(i < 0 || i >= filename.length())
+ return null;
+
+ String hexVersion = filename.substring(i + 1);
+ try {
+ long version = Long.parseLong(hexVersion, 16);
+ return Long.toHexString(version);
+ } catch (NumberFormatException e) {
+ return null;
+ }
+ }
+
+ /**
+ * Parse config from a Properties.
+ * @param zkProp Properties to parse from.
+ * @throws IOException
+ * @throws ConfigException
+ */
+ public void parseProperties(Properties zkProp)
+ throws IOException, ConfigException {
+ int clientPort = 0;
+ int secureClientPort = 0;
+ String clientPortAddress = null;
+ String secureClientPortAddress = null;
+ VerifyingFileFactory vff = new VerifyingFileFactory.Builder(LOG).warnForRelativePath().build();
+ for (Entry<Object, Object> entry : zkProp.entrySet()) {
+ String key = entry.getKey().toString().trim();
+ String value = entry.getValue().toString().trim();
+ if (key.equals("dataDir")) {
+ dataDir = vff.create(value);
+ } else if (key.equals("dataLogDir")) {
+ dataLogDir = vff.create(value);
+ } else if (key.equals("clientPort")) {
+ clientPort = Integer.parseInt(value);
+ } else if (key.equals("localSessionsEnabled")) {
+ localSessionsEnabled = Boolean.parseBoolean(value);
+ } else if (key.equals("localSessionsUpgradingEnabled")) {
+ localSessionsUpgradingEnabled = Boolean.parseBoolean(value);
+ } else if (key.equals("clientPortAddress")) {
+ clientPortAddress = value.trim();
+ } else if (key.equals("secureClientPort")) {
+ secureClientPort = Integer.parseInt(value);
+ } else if (key.equals("secureClientPortAddress")){
+ secureClientPortAddress = value.trim();
+ } else if (key.equals("tickTime")) {
+ tickTime = Integer.parseInt(value);
+ } else if (key.equals("maxClientCnxns")) {
+ maxClientCnxns = Integer.parseInt(value);
+ } else if (key.equals("minSessionTimeout")) {
+ minSessionTimeout = Integer.parseInt(value);
+ } else if (key.equals("maxSessionTimeout")) {
+ maxSessionTimeout = Integer.parseInt(value);
+ } else if (key.equals("initLimit")) {
+ initLimit = Integer.parseInt(value);
+ } else if (key.equals("syncLimit")) {
+ syncLimit = Integer.parseInt(value);
+ } else if (key.equals("electionAlg")) {
+ electionAlg = Integer.parseInt(value);
+ } else if (key.equals("quorumListenOnAllIPs")) {
+ quorumListenOnAllIPs = Boolean.parseBoolean(value);
+ } else if (key.equals("peerType")) {
+ if (value.toLowerCase().equals("observer")) {
+ peerType = LearnerType.OBSERVER;
+ } else if (value.toLowerCase().equals("participant")) {
+ peerType = LearnerType.PARTICIPANT;
+ } else
+ {
+ throw new ConfigException("Unrecognised peertype: " + value);
+ }
+ } else if (key.equals( "syncEnabled" )) {
+ syncEnabled = Boolean.parseBoolean(value);
+ } else if (key.equals("dynamicConfigFile")){
+ dynamicConfigFileStr = value;
+ } else if (key.equals("autopurge.snapRetainCount")) {
+ snapRetainCount = Integer.parseInt(value);
+ } else if (key.equals("autopurge.purgeInterval")) {
+ purgeInterval = Integer.parseInt(value);
+ } else if (key.equals("standaloneEnabled")) {
+ if (value.toLowerCase().equals("true")) {
+ setStandaloneEnabled(true);
+ } else if (value.toLowerCase().equals("false")) {
+ setStandaloneEnabled(false);
+ } else {
+ throw new ConfigException("Invalid option " + value + " for standalone mode. Choose 'true' or 'false.'");
+ }
+ } else if (key.equals("reconfigEnabled")) {
+ if (value.toLowerCase().equals("true")) {
+ setReconfigEnabled(true);
+ } else if (value.toLowerCase().equals("false")) {
+ setReconfigEnabled(false);
+ } else {
+ throw new ConfigException("Invalid option " + value + " for reconfigEnabled flag. Choose 'true' or 'false.'");
+ }
+ } else if (key.equals("sslQuorum")){
+ sslQuorum = Boolean.parseBoolean(value);
+ } else if (key.equals("portUnification")){
+ shouldUsePortUnification = Boolean.parseBoolean(value);
+ } else if (key.equals("sslQuorumReloadCertFiles")) {
+ sslQuorumReloadCertFiles = Boolean.parseBoolean(value);
+ } else if ((key.startsWith("server.") || key.startsWith("group") || key.startsWith("weight")) && zkProp.containsKey("dynamicConfigFile")) {
+ throw new ConfigException("parameter: " + key + " must be in a separate dynamic config file");
+ } else if (key.equals(QuorumAuth.QUORUM_SASL_AUTH_ENABLED)) {
+ quorumEnableSasl = Boolean.parseBoolean(value);
+ } else if (key.equals(QuorumAuth.QUORUM_SERVER_SASL_AUTH_REQUIRED)) {
+ quorumServerRequireSasl = Boolean.parseBoolean(value);
+ } else if (key.equals(QuorumAuth.QUORUM_LEARNER_SASL_AUTH_REQUIRED)) {
+ quorumLearnerRequireSasl = Boolean.parseBoolean(value);
+ } else if (key.equals(QuorumAuth.QUORUM_LEARNER_SASL_LOGIN_CONTEXT)) {
+ quorumLearnerLoginContext = value;
+ } else if (key.equals(QuorumAuth.QUORUM_SERVER_SASL_LOGIN_CONTEXT)) {
+ quorumServerLoginContext = value;
+ } else if (key.equals(QuorumAuth.QUORUM_KERBEROS_SERVICE_PRINCIPAL)) {
+ quorumServicePrincipal = value;
+ } else if (key.equals("quorum.cnxn.threads.size")) {
+ quorumCnxnThreadsSize = Integer.parseInt(value);
+ } else {
+ System.setProperty("zookeeper." + key, value);
+ }
+ }
+
+ if (!quorumEnableSasl && quorumServerRequireSasl) {
+ throw new IllegalArgumentException(
+ QuorumAuth.QUORUM_SASL_AUTH_ENABLED
+ + " is disabled, so cannot enable "
+ + QuorumAuth.QUORUM_SERVER_SASL_AUTH_REQUIRED);
+ }
+ if (!quorumEnableSasl && quorumLearnerRequireSasl) {
+ throw new IllegalArgumentException(
+ QuorumAuth.QUORUM_SASL_AUTH_ENABLED
+ + " is disabled, so cannot enable "
+ + QuorumAuth.QUORUM_LEARNER_SASL_AUTH_REQUIRED);
+ }
+ // If quorumpeer learner is not auth enabled then self won't be able to
+ // join quorum. So this condition is ensuring that the quorumpeer learner
+ // is also auth enabled while enabling quorum server require sasl.
+ if (!quorumLearnerRequireSasl && quorumServerRequireSasl) {
+ throw new IllegalArgumentException(
+ QuorumAuth.QUORUM_LEARNER_SASL_AUTH_REQUIRED
+ + " is disabled, so cannot enable "
+ + QuorumAuth.QUORUM_SERVER_SASL_AUTH_REQUIRED);
+ }
+
+ // Reset to MIN_SNAP_RETAIN_COUNT if invalid (less than 3)
+ // PurgeTxnLog.purge(File, File, int) will not allow to purge less
+ // than 3.
+ if (snapRetainCount < MIN_SNAP_RETAIN_COUNT) {
+ LOG.warn("Invalid autopurge.snapRetainCount: " + snapRetainCount
+ + ". Defaulting to " + MIN_SNAP_RETAIN_COUNT);
+ snapRetainCount = MIN_SNAP_RETAIN_COUNT;
+ }
+
+ if (dataDir == null) {
+ throw new IllegalArgumentException("dataDir is not set");
+ }
+ if (dataLogDir == null) {
+ dataLogDir = dataDir;
+ }
+
+ if (clientPort == 0) {
+ LOG.info("clientPort is not set");
+ if (clientPortAddress != null) {
+ throw new IllegalArgumentException("clientPortAddress is set but clientPort is not set");
+ }
+ } else if (clientPortAddress != null) {
+ this.clientPortAddress = new InetSocketAddress(
+ InetAddress.getByName(clientPortAddress), clientPort);
+ LOG.info("clientPortAddress is {}", formatInetAddr(this.clientPortAddress));
+ } else {
+ this.clientPortAddress = new InetSocketAddress(clientPort);
+ LOG.info("clientPortAddress is {}", formatInetAddr(this.clientPortAddress));
+ }
+
+ if (secureClientPort == 0) {
+ LOG.info("secureClientPort is not set");
+ if (secureClientPortAddress != null) {
+ throw new IllegalArgumentException("secureClientPortAddress is set but secureClientPort is not set");
+ }
+ } else if (secureClientPortAddress != null) {
+ this.secureClientPortAddress = new InetSocketAddress(
+ InetAddress.getByName(secureClientPortAddress), secureClientPort);
+ LOG.info("secureClientPortAddress is {}", formatInetAddr(this.secureClientPortAddress));
+ } else {
+ this.secureClientPortAddress = new InetSocketAddress(secureClientPort);
+ LOG.info("secureClientPortAddress is {}", formatInetAddr(this.secureClientPortAddress));
+ }
+ if (this.secureClientPortAddress != null) {
+ configureSSLAuth();
+ }
+
+ if (tickTime == 0) {
+ throw new IllegalArgumentException("tickTime is not set");
+ }
+
+ minSessionTimeout = minSessionTimeout == -1 ? tickTime * 2 : minSessionTimeout;
+ maxSessionTimeout = maxSessionTimeout == -1 ? tickTime * 20 : maxSessionTimeout;
+
+ if (minSessionTimeout > maxSessionTimeout) {
+ throw new IllegalArgumentException(
+ "minSessionTimeout must not be larger than maxSessionTimeout");
+ }
+
+ // backward compatibility - dynamic configuration in the same file as
+ // static configuration params see writeDynamicConfig()
+ if (dynamicConfigFileStr == null) {
+ setupQuorumPeerConfig(zkProp, true);
+ if (isDistributed() && isReconfigEnabled()) {
+ // we don't backup static config for standalone mode.
+ // we also don't backup if reconfig feature is disabled.
+ backupOldConfig();
+ }
+ }
+ }
+
+ /**
+ * Configure SSL authentication only if it is not configured.
+ *
+ * @throws ConfigException
+ * If authentication scheme is configured but authentication
+ * provider is not configured.
+ */
+ public static void configureSSLAuth() throws ConfigException {
+ try (ClientX509Util clientX509Util = new ClientX509Util()) {
+ String sslAuthProp = "zookeeper.authProvider." + System.getProperty(clientX509Util.getSslAuthProviderProperty(), "x509");
+ if (System.getProperty(sslAuthProp) == null) {
+ if ("zookeeper.authProvider.x509".equals(sslAuthProp)) {
+ System.setProperty("zookeeper.authProvider.x509",
+ "org.apache.zookeeper.server.auth.X509AuthenticationProvider");
+ } else {
+ throw new ConfigException("No auth provider configured for the SSL authentication scheme '"
+ + System.getProperty(clientX509Util.getSslAuthProviderProperty()) + "'.");
+ }
+ }
+ }
+ }
+
+ /**
+ * Backward compatibility -- It would backup static config file on bootup
+ * if users write dynamic configuration in "zoo.cfg".
+ */
+ private void backupOldConfig() throws IOException {
+ new AtomicFileWritingIdiom(new File(configFileStr + ".bak"), new OutputStreamStatement() {
+ @Override
+ public void write(OutputStream output) throws IOException {
+ InputStream input = null;
+ try {
+ input = new FileInputStream(new File(configFileStr));
+ byte[] buf = new byte[1024];
+ int bytesRead;
+ while ((bytesRead = input.read(buf)) > 0) {
+ output.write(buf, 0, bytesRead);
+ }
+ } finally {
+ if( input != null) {
+ input.close();
+ }
+ }
+ }
+ });
+ }
+
+ /**
+ * Writes dynamic configuration file
+ */
+ public static void writeDynamicConfig(final String dynamicConfigFilename,
+ final QuorumVerifier qv,
+ final boolean needKeepVersion)
+ throws IOException {
+
+ new AtomicFileWritingIdiom(new File(dynamicConfigFilename), new WriterStatement() {
+ @Override
+ public void write(Writer out) throws IOException {
+ Properties cfg = new Properties();
+ cfg.load( new StringReader(
+ qv.toString()));
+
+ List<String> servers = new ArrayList<String>();
+ for (Entry<Object, Object> entry : cfg.entrySet()) {
+ String key = entry.getKey().toString().trim();
+ if ( !needKeepVersion && key.startsWith("version"))
+ continue;
+
+ String value = entry.getValue().toString().trim();
+ servers.add(key
+ .concat("=")
+ .concat(value));
+ }
+
+ Collections.sort(servers);
+ out.write(StringUtils.joinStrings(servers, "\n"));
+ }
+ });
+ }
+
+ /**
+ * Edit static config file.
+ * If there are quorum information in static file, e.g. "server.X", "group",
+ * it will remove them.
+ * If it needs to erase client port information left by the old config,
+ * "eraseClientPortAddress" should be set true.
+ * It should also updates dynamic file pointer on reconfig.
+ */
+ public static void editStaticConfig(final String configFileStr,
+ final String dynamicFileStr,
+ final boolean eraseClientPortAddress)
+ throws IOException {
+ // Some tests may not have a static config file.
+ if (configFileStr == null)
+ return;
+
+ File configFile = (new VerifyingFileFactory.Builder(LOG)
+ .warnForRelativePath()
+ .failForNonExistingPath()
+ .build()).create(configFileStr);
+
+ final File dynamicFile = (new VerifyingFileFactory.Builder(LOG)
+ .warnForRelativePath()
+ .failForNonExistingPath()
+ .build()).create(dynamicFileStr);
+
+ final Properties cfg = new Properties();
+ FileInputStream in = new FileInputStream(configFile);
+ try {
+ cfg.load(in);
+ } finally {
+ in.close();
+ }
+
+ new AtomicFileWritingIdiom(new File(configFileStr), new WriterStatement() {
+ @Override
+ public void write(Writer out) throws IOException {
+ for (Entry<Object, Object> entry : cfg.entrySet()) {
+ String key = entry.getKey().toString().trim();
+
+ if (key.startsWith("server.")
+ || key.startsWith("group")
+ || key.startsWith("weight")
+ || key.startsWith("dynamicConfigFile")
+ || (eraseClientPortAddress
+ && (key.startsWith("clientPort")
+ || key.startsWith("clientPortAddress")))) {
+ // not writing them back to static file
+ continue;
+ }
+
+ String value = entry.getValue().toString().trim();
+ out.write(key.concat("=").concat(value).concat("\n"));
+ }
+
+ // updates the dynamic file pointer
+ String dynamicConfigFilePath = PathUtils.normalizeFileSystemPath(dynamicFile.getCanonicalPath());
+ out.write("dynamicConfigFile="
+ .concat(dynamicConfigFilePath)
+ .concat("\n"));
+ }
+ });
+ }
+
+
+ public static void deleteFile(String filename){
+ if (filename == null) return;
+ File f = new File(filename);
+ if (f.exists()) {
+ try{
+ f.delete();
+ } catch (Exception e) {
+ LOG.warn("deleting " + filename + " failed");
+ }
+ }
+ }
+
+
+ private static QuorumVerifier createQuorumVerifier(Properties dynamicConfigProp, boolean isHierarchical) throws ConfigException{
+ if(isHierarchical){
+ return new QuorumHierarchical(dynamicConfigProp);
+ } else {
+ /*
+ * The default QuorumVerifier is QuorumMaj
+ */
+ //LOG.info("Defaulting to majority quorums");
+ return new QuorumMaj(dynamicConfigProp);
+ }
+ }
+
+ void setupQuorumPeerConfig(Properties prop, boolean configBackwardCompatibilityMode)
+ throws IOException, ConfigException {
+ quorumVerifier = parseDynamicConfig(prop, electionAlg, true, configBackwardCompatibilityMode);
+ setupMyId();
+ setupClientPort();
+ setupPeerType();
+ checkValidity();
+ }
+
+ /**
+ * Parse dynamic configuration file and return
+ * quorumVerifier for new configuration.
+ * @param dynamicConfigProp Properties to parse from.
+ * @throws IOException
+ * @throws ConfigException
+ */
+ public static QuorumVerifier parseDynamicConfig(Properties dynamicConfigProp, int eAlg, boolean warnings,
+ boolean configBackwardCompatibilityMode) throws IOException, ConfigException {
+ boolean isHierarchical = false;
+ for (Entry<Object, Object> entry : dynamicConfigProp.entrySet()) {
+ String key = entry.getKey().toString().trim();
+ if (key.startsWith("group") || key.startsWith("weight")) {
+ isHierarchical = true;
+ } else if (!configBackwardCompatibilityMode && !key.startsWith("server.") && !key.equals("version")){
+ LOG.info(dynamicConfigProp.toString());
+ throw new ConfigException("Unrecognised parameter: " + key);
+ }
+ }
+
+ QuorumVerifier qv = createQuorumVerifier(dynamicConfigProp, isHierarchical);
+
+ int numParticipators = qv.getVotingMembers().size();
Review comment:
*NULL_DEREFERENCE:* object returned by `qv.getVotingMembers()` could be null and is dereferenced at line 632.
(at-me [in a reply](https://docs.muse.dev/docs/talk-to-muse/) with `help` or `ignore`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] maoling commented on pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
maoling commented on pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695#issuecomment-841764845
@EagerToLeave I close this PR, you can reopen it if I make a wrong judgment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] maoling closed pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
maoling closed pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] maoling commented on pull request #1695: Comment
Posted by GitBox <gi...@apache.org>.
maoling commented on pull request #1695:
URL: https://github.com/apache/zookeeper/pull/1695#issuecomment-838656711
@EagerToLeave Is this a misoperation???
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org