[GitHub] [zookeeper] ztzg commented on a change in pull request #1526: ZOOKEEPER-3874 Official API to start ZooKeeper server from Java

[GitBox <gi...@apache.org>]

ztzg commented on a change in pull request #1526:
URL: https://github.com/apache/zookeeper/pull/1526#discussion_r520010844



##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java
##########
@@ -0,0 +1,120 @@
+package org.apache.zookeeper.server.embedded;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.nio.file.Path;
+import java.util.Properties;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+/**
+ * This API allows you to start a ZooKeeper server node from Java code <p>
+ * The server will run inside the same process.<p>
+ * Tipical usecases are:
+ * <ul>
+ * <li>Running automated tests</li>
+ * <li>Launch ZooKeeper server with a Java based service management system</li>
+ * </ul>
+ * <p>
+ * Please take into consideration that in production usually it is better to not run the client
+ * together with the server in order to avoid race conditions, especially around how ephemeral nodes work.
+ */
+@InterfaceAudience.Public
+@InterfaceStability.Evolving
+public interface ZooKeeperServerEmbedded extends AutoCloseable {
+    /**
+     * Builder for ZooKeeperServerEmbedded.
+     */
+    class ZookKeeperServerEmbeddedBuilder {
+
+        private Path baseDir;
+        private Properties configuration;
+        private ExitHandler exitHandler = ExitHandler.EXIT;
+
+        /**
+         * Base directory of the server.
+         * The system will create a temporary configuration file inside this directory.
+         * Please remember that dynamic configuration files wil be saved into this directory by default.
+         * <p>
+         * If you do not set a 'dataDir' configuration entry the system will use a subdirectory if baseDir.

Review comment:
       Nit: "of baseDir."

##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServerMain.java
##########
@@ -237,4 +237,26 @@ ServerCnxnFactory getCnxnFactory() {
         return cnxnFactory;
     }
 
+    /**
+     * Shutdowns properly the service, this method is not a public API.
+     */
+    public void close() {
+        ServerCnxnFactory primaryCnxnFactory = this.cnxnFactory;
+        if (primaryCnxnFactory == null) {
+            // in case of pure TLS we can hook into secureCnxnFactory
+            primaryCnxnFactory = secureCnxnFactory;
+        }
+        if (primaryCnxnFactory == null || primaryCnxnFactory.getZooKeeperServer() == null) {
+            return;
+        }
+        ZooKeeperServerShutdownHandler zkShutdownHandler = primaryCnxnFactory.getZooKeeperServer().getZkShutdownHandler();
+        zkShutdownHandler.handle(ZooKeeperServer.State.SHUTDOWN);

Review comment:
       Just curious, and showing my ignorance here: why not:
   
   ```java
   primaryCnxnFactory.getZooKeeperServer().setState(ZooKeeperServer.State.SHUTDOWN);
   ```
   
   ?

##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java
##########
@@ -0,0 +1,120 @@
+package org.apache.zookeeper.server.embedded;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.nio.file.Path;
+import java.util.Properties;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+/**
+ * This API allows you to start a ZooKeeper server node from Java code <p>
+ * The server will run inside the same process.<p>
+ * Tipical usecases are:

Review comment:
       Nit: "Typical."

##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/server/embedded/ZookeeperServerClusterTest.java
##########
@@ -0,0 +1,124 @@
+package org.apache.zookeeper.server.embedded;
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Properties;
+import org.apache.zookeeper.PortAssignment;
+import org.apache.zookeeper.test.ClientBase;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class ZookeeperServerClusterTest {
+
+    @BeforeAll
+    public static void setUpEnvironment() {
+        System.setProperty("zookeeper.admin.enableServer", "false");
+        System.setProperty("zookeeper.4lw.commands.whitelist", "*");
+    }
+
+    @AfterAll
+    public static void cleanUpEnvironment() throws InterruptedException, IOException {
+        System.clearProperty("zookeeper.admin.enableServer");
+        System.clearProperty("zookeeper.4lw.commands.whitelist");
+        System.clearProperty("java.security.auth.login.config");

Review comment:
       Leftover?

##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/server/embedded/ZookeeperServerEmbeddedTest.java
##########
@@ -0,0 +1,69 @@
+package org.apache.zookeeper.server.embedded;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright
+ * ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the
+ * License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied. See the License for the specific language governing permissions and limitations under the License.
+ */
+
+import static org.junit.Assert.assertTrue;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.Properties;
+import org.apache.zookeeper.PortAssignment;
+import org.apache.zookeeper.test.ClientBase;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class ZookeeperServerEmbeddedTest {
+
+    @BeforeAll
+    public static void setUpEnvironment() {
+        System.setProperty("zookeeper.admin.enableServer", "false");
+        System.setProperty("zookeeper.4lw.commands.whitelist", "*");
+    }
+
+    @AfterAll
+    public static void cleanUpEnvironment() throws InterruptedException, IOException {
+        System.clearProperty("zookeeper.admin.enableServer");

Review comment:
       Missing `System.clearProperty("zookeeper.4lw.commands.whitelist")`?

##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/server/embedded/ZookeeperServeInfo.java
##########
@@ -0,0 +1,297 @@
+package org.apache.zookeeper.server.embedded;
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.lang.management.ManagementFactory;
+import java.lang.reflect.UndeclaredThrowableException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+import javax.management.InstanceNotFoundException;
+import javax.management.MBeanServer;
+import javax.management.MBeanServerInvocationHandler;
+import javax.management.ObjectInstance;
+import javax.management.ObjectName;
+import org.apache.zookeeper.common.StringUtils;
+import org.apache.zookeeper.server.ConnectionMXBean;
+import org.apache.zookeeper.server.ZooKeeperServerBean;
+import org.apache.zookeeper.server.quorum.LocalPeerMXBean;
+import org.apache.zookeeper.server.quorum.QuorumBean;
+import org.apache.zookeeper.server.quorum.QuorumMXBean;
+import org.apache.zookeeper.server.quorum.RemotePeerMXBean;
+
+public class ZookeeperServeInfo {
+
+    private static final MBeanServer localServer = ManagementFactory.getPlatformMBeanServer();
+
+    public static class PeerInfo {
+
+        private final String name;
+        private final String quorumAddress;
+        private final String state;
+        private final boolean leader;
+
+        public PeerInfo(String name, String quorumAddress, String state, boolean leader) {
+            this.name = name;
+            this.quorumAddress = quorumAddress;
+            this.state = state;
+            this.leader = leader;
+        }
+
+        public String getName() {
+            return name;
+        }
+
+        public String getQuorumAddress() {
+            return quorumAddress;
+        }
+
+        public String getState() {
+            return state;
+        }
+
+        public boolean isLeader() {
+            return leader;
+        }
+
+        @Override
+        public String toString() {
+            return "PeerInfo{" + "name=" + name + ", leader=" + leader + ", quorumAddress=" + quorumAddress
+                    + ", state=" + state + '}';
+        }
+    }
+
+    public static class ConnectionInfo {
+
+        private final String sourceip;
+        private final String sessionid;
+        private final String lastoperation;
+        private final String lastResponseTime;
+        private final String avgLatency;
+        private final String lastLatency;
+        private final String nodes;
+
+        public ConnectionInfo(String sourceip, String sessionid, String lastoperation, String lastResponseTime,
+                              String avgLatency, String lastLatency, String nodes) {
+            this.sourceip = sourceip;
+            this.sessionid = sessionid;
+            this.lastoperation = lastoperation;
+            this.lastResponseTime = lastResponseTime;
+            this.avgLatency = avgLatency;
+            this.lastLatency = lastLatency;
+            this.nodes = nodes;
+        }
+
+        public String getLastLatency() {
+            return lastLatency;
+        }
+
+        public String getSourceip() {
+            return sourceip;
+        }
+
+        public String getSessionid() {
+            return sessionid;
+        }
+
+        public String getLastoperation() {
+            return lastoperation;
+        }
+
+        public String getLastResponseTime() {
+            return lastResponseTime;
+        }
+
+        public String getAvgLatency() {
+            return avgLatency;
+        }
+
+        public String getNodes() {
+            return nodes;
+        }
+
+        @Override
+        public String toString() {
+            return "ConnectionInfo{" + "sourceip=" + sourceip + ", sessionid=" + sessionid + ", lastoperation="
+                    + lastoperation + ", lastResponseTime=" + lastResponseTime + ", avgLatency=" + avgLatency
+                    + ", nodes=" + nodes + '}';
+        }
+    }
+
+    public static class ServerInfo {
+
+        private final List<ConnectionInfo> connections = new ArrayList<>();
+        private boolean isleader;
+        private boolean standaloneMode;
+        public List<PeerInfo> peers = new ArrayList<>();
+
+        public boolean isStandaloneMode() {
+            return standaloneMode;
+        }
+
+        public List<ConnectionInfo> getConnections() {
+            return connections;
+        }
+
+        public boolean isIsleader() {
+            return isleader;
+        }

Review comment:
       I suppose there is a good reason for this not to be called `isLeader`?  (Introspection or something?  But I did not see any in this context.)
   

##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java
##########
@@ -0,0 +1,120 @@
+package org.apache.zookeeper.server.embedded;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.nio.file.Path;
+import java.util.Properties;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+/**
+ * This API allows you to start a ZooKeeper server node from Java code <p>
+ * The server will run inside the same process.<p>
+ * Tipical usecases are:
+ * <ul>
+ * <li>Running automated tests</li>
+ * <li>Launch ZooKeeper server with a Java based service management system</li>
+ * </ul>
+ * <p>
+ * Please take into consideration that in production usually it is better to not run the client
+ * together with the server in order to avoid race conditions, especially around how ephemeral nodes work.
+ */
+@InterfaceAudience.Public
+@InterfaceStability.Evolving
+public interface ZooKeeperServerEmbedded extends AutoCloseable {
+    /**
+     * Builder for ZooKeeperServerEmbedded.
+     */
+    class ZookKeeperServerEmbeddedBuilder {
+
+        private Path baseDir;
+        private Properties configuration;
+        private ExitHandler exitHandler = ExitHandler.EXIT;
+
+        /**
+         * Base directory of the server.
+         * The system will create a temporary configuration file inside this directory.
+         * Please remember that dynamic configuration files wil be saved into this directory by default.
+         * <p>
+         * If you do not set a 'dataDir' configuration entry the system will use a subdirectory if baseDir.
+         * @param baseDir
+         * @return the builder
+         */
+        public ZookKeeperServerEmbeddedBuilder baseDir(Path baseDir) {
+            this.baseDir = baseDir;
+            return this;
+        }
+
+        /**
+         * Set the contents of the main configuration as it would be in zk_server.conf file.
+         * @param configuration the configuration
+         * @return the builder
+         */
+        public ZookKeeperServerEmbeddedBuilder configuration(Properties configuration) {
+            this.configuration = configuration;
+            return this;
+        }
+
+        /**
+         * Set the behaviour in case of hard system errors, see {@link ExitHandler}.
+         * @param exitHandler the handler
+         * @return the builder
+         */
+        public ZookKeeperServerEmbeddedBuilder exitHandler(ExitHandler exitHandler) {
+            this.exitHandler = exitHandler;
+            return this;
+        }
+
+        /**
+         * Validate the configuration and create the server, without starting it.
+         * @return the new server
+         * @throws Exception
+         * @see #start()
+         */
+        public ZooKeeperServerEmbedded build() throws Exception {
+            if (baseDir == null) {
+                throw new IllegalArgumentException();
+            }
+            if (configuration == null) {
+                throw new IllegalArgumentException();
+            }
+            if (exitHandler == null) {
+                throw new IllegalArgumentException();
+            }

Review comment:
       Two small things bother me (very slightly) here:
   
   1. `IllegalArgumentException` in an argument-less method;
   2. No clear indication of which parameter was `null` in error logs.
   
   Joshua Bloch [seems to recommend](https://softwareengineering.stackexchange.com/a/241320) throwing `IllegalStateException`.  But perhaps just using `Objects.requireNonNull` in the buildee's constructor would be simpler/clearer?
   
   
   

##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbeddedImpl.java
##########
@@ -0,0 +1,164 @@
+package org.apache.zookeeper.server.embedded;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+import java.io.OutputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Map;
+import java.util.Properties;
+import org.apache.zookeeper.server.DatadirCleanupManager;
+import org.apache.zookeeper.server.ExitCode;
+import org.apache.zookeeper.server.ServerConfig;
+import org.apache.zookeeper.server.ZooKeeperServerMain;
+import org.apache.zookeeper.server.quorum.QuorumPeer;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig;
+import org.apache.zookeeper.server.quorum.QuorumPeerMain;
+import org.apache.zookeeper.util.ServiceUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright
+ * ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the
+ * License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied. See the License for the specific language governing permissions and limitations under the License.
+ */

Review comment:
       Nit: This shouldn't be a Javadoc comment.  Also: shouldn't such license banners be "canonicalized" to  match `ZooKeeper.java`, and be at the top?

##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/server/embedded/ZookeeperServerSslEmbeddedTest.java
##########
@@ -0,0 +1,117 @@
+package org.apache.zookeeper.server.embedded;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright
+ * ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the
+ * License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied. See the License for the specific language governing permissions and limitations under the License.
+ */
+
+import static org.junit.Assert.assertTrue;
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.Properties;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import org.apache.zookeeper.PortAssignment;
+import org.apache.zookeeper.WatchedEvent;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.test.ClientBase;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class ZookeeperServerSslEmbeddedTest {
+
+    @BeforeAll
+    public static void setUpEnvironment() {
+        System.setProperty("zookeeper.admin.enableServer", "false");
+        System.setProperty("zookeeper.4lw.commands.whitelist", "*");
+    }
+
+    @AfterAll
+    public static void cleanUpEnvironment() throws InterruptedException, IOException {
+        System.clearProperty("zookeeper.admin.enableServer");
+        System.clearProperty("zookeeper.4lw.commands.whitelist");
+        System.clearProperty("java.security.auth.login.config");
+        System.clearProperty("zookeeper.client.secure");

Review comment:
       Missing `System.clearProperty("zookeeper.clientCnxnSocket")`?

##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbeddedImpl.java
##########
@@ -0,0 +1,164 @@
+package org.apache.zookeeper.server.embedded;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+import java.io.OutputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Map;
+import java.util.Properties;
+import org.apache.zookeeper.server.DatadirCleanupManager;
+import org.apache.zookeeper.server.ExitCode;
+import org.apache.zookeeper.server.ServerConfig;
+import org.apache.zookeeper.server.ZooKeeperServerMain;
+import org.apache.zookeeper.server.quorum.QuorumPeer;
+import org.apache.zookeeper.server.quorum.QuorumPeerConfig;
+import org.apache.zookeeper.server.quorum.QuorumPeerMain;
+import org.apache.zookeeper.util.ServiceUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright
+ * ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the
+ * License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied. See the License for the specific language governing permissions and limitations under the License.
+ */
+/**
+ * Implementation
+ */
+@SuppressFBWarnings("DM_EXIT")

Review comment:
       Is this annotation a leftover from a previous iteration? It doesn't seem to be needed.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org