[GitHub] [fluo-muchos] keith-turner commented on issue #316: Add support for ZooKeeper 3.5

[GitBox <gi...@apache.org>]

keith-turner commented on issue #316: Add support for ZooKeeper 3.5
URL: https://github.com/apache/fluo-muchos/pull/316#issuecomment-581618750
 
 
   I took a look at what is in the zookeeper 3.4 and 3.5 dirs and found found the following. The contents of ZK/lib differs a lot between 3.4 and 3.5. 
   
   ```bash
   $ ls zookeeper-3.4.14/**/*.jar
   zookeeper-3.4.14/dist-maven/zookeeper-3.4.14.jar          zookeeper-3.4.14/lib/audience-annotations-0.5.0.jar  zookeeper-3.4.14/lib/slf4j-api-1.7.25.jar
   zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-javadoc.jar  zookeeper-3.4.14/lib/jline-0.9.94.jar                zookeeper-3.4.14/lib/slf4j-log4j12-1.7.25.jar
   zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-sources.jar  zookeeper-3.4.14/lib/log4j-1.2.17.jar
   zookeeper-3.4.14/dist-maven/zookeeper-3.4.14-tests.jar    zookeeper-3.4.14/lib/netty-3.10.6.Final.jar
   $ ls apache-zookeeper-3.5.6-bin/**/*.jar
   apache-zookeeper-3.5.6-bin/lib/audience-annotations-0.5.0.jar       apache-zookeeper-3.5.6-bin/lib/log4j-1.2.17.jar
   apache-zookeeper-3.5.6-bin/lib/commons-cli-1.2.jar                  apache-zookeeper-3.5.6-bin/lib/netty-buffer-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jackson-annotations-2.9.10.jar       apache-zookeeper-3.5.6-bin/lib/netty-codec-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jackson-core-2.9.10.jar              apache-zookeeper-3.5.6-bin/lib/netty-common-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jackson-databind-2.9.10.jar          apache-zookeeper-3.5.6-bin/lib/netty-handler-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/javax.servlet-api-3.1.0.jar          apache-zookeeper-3.5.6-bin/lib/netty-resolver-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-http-9.4.17.v20190418.jar      apache-zookeeper-3.5.6-bin/lib/netty-transport-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-io-9.4.17.v20190418.jar        apache-zookeeper-3.5.6-bin/lib/netty-transport-native-epoll-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-security-9.4.17.v20190418.jar  apache-zookeeper-3.5.6-bin/lib/netty-transport-native-unix-common-4.1.42.Final.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-server-9.4.17.v20190418.jar    apache-zookeeper-3.5.6-bin/lib/slf4j-api-1.7.25.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-servlet-9.4.17.v20190418.jar   apache-zookeeper-3.5.6-bin/lib/slf4j-log4j12-1.7.25.jar
   apache-zookeeper-3.5.6-bin/lib/jetty-util-9.4.17.v20190418.jar      apache-zookeeper-3.5.6-bin/lib/zookeeper-3.5.6.jar
   apache-zookeeper-3.5.6-bin/lib/jline-2.11.jar                       apache-zookeeper-3.5.6-bin/lib/zookeeper-jute-3.5.6.jar
   apache-zookeeper-3.5.6-bin/lib/json-simple-1.1.1.jar
   ```
   
   It seems both 3.4 and 3.5 have a lib dir, but we are not currently adding ZK/lib in accumulo-env.sh.  So I suspect the ZK 3.4 dependencies are currently supplied elsewhere on the classpath.  @karthick-rn suggested possibly always adding ZK/lib to the classpath, does anyone know if this works?
   
   >  I prefer the latter, simply because it won't change the CLASSPATH & any previous Zookeeper versions would still work.
   
   Are there good reasons to support ZK 3.4 and 3.5?  If not we always have the option of making ZK 3.5 the minimum supported version of ZK for muchos (we can add a sanity check during setup).  If this makes maintaining muchos simpler, it may be a good way to go.
   
   > We may also have to copy these 2 zookeeper*.jar files to hadoop common lib & hadoop hdfs lib to make sure the same version of Zookeeper is being picked across the cluster.
   
   This may not be an issue for Accumulo. I think the order in which items are listed on the classpath is important for deciding which classes are used (see specification order section at the end of [this doc](https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html)).  So if two ZK versions are listed on the classpath, the one listed first will be chosen.  So if accumulo-env.sh puts ZK_HOME first on the classpath, then maybe it will always use that version even if Hadoop has it. However, not sure if this holds for java 11.
   
   >  I am also wondering if we should refine the addition of ${HADOOP_HOME}/share/hadoop/common/lib/* to the classpath and only include the specific jars required for ADLS Gen2.
   
   One tricky thing with that is when you change the version of Hadoop, the versions of deps will change and their transitive deps may also change.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services