[GitHub] [accumulo] karthick-rn edited a comment on issue #1578: Accumulo master hangs after TLS on ZK

[GitBox <gi...@apache.org>]

karthick-rn edited a comment on issue #1578: Accumulo master hangs after TLS on ZK
URL: https://github.com/apache/accumulo/issues/1578#issuecomment-609907620
 
 
   > Also, if you can explain the specific steps you took to configure TLS on ZK, so we can reproduce it, that could be helpful to test in different environments.
   
   **Steps to configure TLS on ZK:**
   a) Generate certificates & keystores:
   1) Run the below commands on each hosts & generate a '.crt' file per host
   ```
   keytool -genkeypair -alias $(hostname -f) -keyalg RSA -keysize 2048 -dname "cn=$(hostname -f)" -keypass changeit -keystore keystore.jks -storepass changeit
   keytool -exportcert -alias $(hostname -f) -keystore keystore.jks -file $(hostname -f).crt -rfc -storepass changeit
   ```
   2) Copy the '*.crt' file generated on each hosts to host1 and generate the truststore.jks as shown below 
   ```
   for i in `ls *.crt`; do
       name=$(echo $i | sed 's/\.crt//g')
       keytool -importcert -alias $name -file $name.crt -keystore truststore.jks -storepass changeit -noprompt
   done
   ```
   3) Copy "truststore.jks" to all the hosts
   ```
   for i in `cat host_list`; do 
       scp truststore.jks $i:/path/to/truststore/; 
   done
   ```
   where `host_list` is a file that contains fqdn of all hosts
   
   4) Verify the contents of the truststore.jks  & ensure it contains all the hosts in the cluster
   `keytool -list -v -keystore truststore.jks`
   
   b) Configurations:
   
   1) Update the server & quorum configs on $ZOOKEEPER_HOME/conf/zoo.cfg
   ```
   # Server configuration
   secureClientPort=2281
   serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
   
   # Quorum configuration
   sslQuorum=true
   ssl.quorum.keyStore.location=/path/to/keystore.jks
   ssl.quorum.keyStore.password=changeit
   ssl.quorum.trustStore.location=/path/to/truststore.jks
   ssl.quorum.trustStore.password=changeit
   
   # the port at which the clients will connect
   #clientPort=2181 (Comment or remove the insecure client port)
   ```
   
   2) Update the client & server configs on $ZOOKEEPER_HOME/bin/zkEnv.sh
   ```
   SERVER_JVMFLAGS="-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory \
     -Dzookeeper.ssl.keyStore.location=/path/to/keystore.jks \
     -Dzookeeper.ssl.keyStore.password=changeit \
     -Dzookeeper.ssl.trustStore.location=/path/to/truststore.jks \
     -Dzookeeper.ssl.trustStore.password=changeit"
   
   CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty \
     -Dzookeeper.client.secure=true \
     -Dzookeeper.ssl.keyStore.location=/path/to/keystore.jks \
     -Dzookeeper.ssl.keyStore.password=changeit \
     -Dzookeeper.ssl.trustStore.location=/path/to/truststore.jks \
     -Dzookeeper.ssl.trustStore.password=changeit"
   ```
   c) Testing
   1) Start Zookeeper service on the hosts running ZK
   `$ZOOKEEPER_HOME/bin/zkServer.sh start`
   
   2) The following messages in the ZK log confirms the ensemble is running on TLS
   ```
   INFO  [main:QuorumPeer@1779] - Using TLS encrypted quorum communication
   INFO  [main:QuorumPeer@1787] - Port unification disabled
   INFO  [QuorumPeerListener:QuorumCnxManager$Listener@894] - Creating TLS-only quorum server socket
   ```
   
   Accumulo (client connection)
   
   The following changes are required to $ACCUMULO_HOME/conf/accumulo-env.sh
   <pre>
   CLASSPATH="${conf}:${lib}/*:${HADOOP_CONF_DIR}:${ZOOKEEPER_HOME}/*:<b>${ZOOKEEPER_HOME}/lib/"
   
   CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty \
     -Dzookeeper.client.secure=true \
     -Dzookeeper.ssl.keyStore.location=$ZOOKEEPER_HOME/conf/ssl/keystore.jks \
     -Dzookeeper.ssl.keyStore.password=hadoop \
     -Dzookeeper.ssl.trustStore.location=$ZOOKEEPER_HOME/conf/ssl/truststore.jks \
     -Dzookeeper.ssl.trustStore.password=hadoop"
   </b>
   JAVA_OPTS=("${JAVA_OPTS[@]}"
     "-Daccumulo.log.dir=${ACCUMULO_LOG_DIR}"
     "-Daccumulo.application=${cmd}${ACCUMULO_SERVICE_INSTANCE}_$(hostname)"
     "-Daccumulo.metrics.service.instance=${ACCUMULO_SERVICE_INSTANCE}" <b>$CLIENT_JVMFLAGS </b>)
   </pre>
   
   Also, replace the ZK port from `2181` to `2281` on `accumulo.properties` & `accumulo-client.properties`
   
   
   **Reference:**
   https://zookeeper.apache.org/doc/r3.5.7/zookeeperAdmin.html#Quorum+TLS
   https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services