You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Sreejesh Radhakrishnan <sr...@db.com.INVALID> on 2022/05/03 13:50:40 UTC
ZK crashing wit NPE ( WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2] java.lang.NullPointerException
Classification: Public
Hi
Sorry not sure if this is the right email. But still trying my luck,
I am getting an NPE as pointed on the subject line, I am using Strimzi KAFKA on GKE (1.21.9-gke.1002) using n1-standard-4 infra.
Zookeeper is throwing below error, is that something which you guys have seen or have idea of what causing it?
get ZK log
Detected Zookeeper ID 2
Preparing truststore
Adding /opt/kafka/cluster-ca-certs/ca.crt to truststore /tmp/zookeeper/cluster.truststore.p12 with alias ca
Certificate was added to keystore
Preparing truststore is complete
Looking for the right CA
Found the right CA: /opt/kafka/cluster-ca-certs/ca.crt
Preparing keystore for client and quorum listeners
Preparing keystore for client and quorum listeners is complete
Starting Zookeeper with configuration:
# The directory where the snapshot is stored.
dataDir=/var/lib/zookeeper/data
# Other options
4lw.commands.whitelist=*
standaloneEnabled=false
reconfigEnabled=true
clientPort=12181
clientPortAddress=127.0.0.1
# TLS options
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.clientAuth=need
ssl.quorum.clientAuth=need
secureClientPort=2181
sslQuorum=true
ssl.trustStore.location=/tmp/zookeeper/cluster.truststore.p12
ssl.trustStore.***
ssl.trustStore.type=PKCS12
ssl.quorum.trustStore.location=/tmp/zookeeper/cluster.truststore.p12
ssl.quorum.trustStore.***
ssl.quorum.trustStore.type=PKCS12
ssl.keyStore.location=/tmp/zookeeper/cluster.keystore.p12
ssl.keyStore.***
ssl.keyStore.type=PKCS12
ssl.quorum.keyStore.location=/tmp/zookeeper/cluster.keystore.p12
ssl.quorum.keyStore.***
ssl.quorum.keyStore.type=PKCS12
# Provided configuration
tickTime=2000
initLimit=5
syncLimit=2
autopurge.purgeInterval=1
# Zookeeper nodes configuration
server.1=uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
server.2=uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
server.3=uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
+ exec /usr/bin/tini -w -e 143 -- /opt/kafka/bin/zookeeper-server-start.sh /tmp/zookeeper.properties
2022-05-03 09:15:52,181 INFO Reading configuration from: /tmp/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
2022-05-03 09:15:52,193 INFO clientPortAddress is 127.0.0.1:12181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
2022-05-03 09:15:52,193 INFO secureClientPortAddress is 0.0.0.0:2181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
2022-05-03 09:15:52,197 INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [main]
2022-05-03 09:15:52,198 INFO observerMasterPort is not set (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
2022-05-03 09:15:52,198 INFO metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
2022-05-03 09:15:52,233 INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager) [main]
2022-05-03 09:15:52,233 INFO autopurge.purgeInterval set to 1 (org.apache.zookeeper.server.DatadirCleanupManager) [main]
2022-05-03 09:15:52,241 INFO Log4j 1.2 jmx support found and enabled. (org.apache.zookeeper.jmx.ManagedUtil) [main]
2022-05-03 09:15:52,235 INFO Purge task started. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask]
2022-05-03 09:15:52,266 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [PurgeTask]
2022-05-03 09:15:52,268 INFO Starting quorum peer, myid=2 (org.apache.zookeeper.server.quorum.QuorumPeerMain) [main]
2022-05-03 09:15:52,291 INFO zookeeper.snapshot.compression.method = CHECKED (org.apache.zookeeper.server.persistence.SnapStream) [PurgeTask]
2022-05-03 09:15:52,298 INFO Purge task completed. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask]
2022-05-03 09:15:52,306 INFO ServerMetrics initialized with provider org.apache.zookeeper.metrics.impl.DefaultMetricsProvider@5276e6b0<ma...@5276e6b0> (org.apache.zookeeper.server.ServerMetrics) [main]
2022-05-03 09:15:52,404 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,404 INFO zookeeper.netty.advancedFlowControl.enabled = false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,404 INFO handshakeThrottlingEnabled = false, zookeeper.netty.server.outstandingHandshake.limit = -1 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,464 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main]
2022-05-03 09:15:52,466 WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [main]
2022-05-03 09:15:52,466 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,466 INFO zookeeper.netty.advancedFlowControl.enabled = false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,466 INFO handshakeThrottlingEnabled = false, zookeeper.netty.server.outstandingHandshake.limit = -1 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,467 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main]
2022-05-03 09:15:52,467 WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [main]
2022-05-03 09:15:52,476 INFO zookeeper.quorumCnxnTimeoutMs=-1 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,507 INFO Logging initialized @932ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log) [main]
2022-05-03 09:15:52,633 WARN o.e.j.s.ServletContextHandler@1a75e76a{/,null,STOPPED}<mailto:o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,STOPPED%7d> contextPath ends with /* (org.eclipse.jetty.server.handler.ContextHandler) [main]
2022-05-03 09:15:52,633 WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler) [main]
2022-05-03 09:15:52,666 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [main]
2022-05-03 09:15:52,666 INFO Local sessions disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO Local session upgrading disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO tickTime set to 2000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO minSessionTimeout set to 4000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO maxSessionTimeout set to 40000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO initLimit set to 5 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO syncLimit set to 2 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,667 INFO connectToLearnerMasterLimit set to 0 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,686 INFO (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,688 INFO ______ _ (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO |___ / | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO / / ___ ___ | | __ ___ ___ _ __ ___ _ __ (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO / / / _ \ / _ \ | |/ / / _ \ / _ \ | '_ \ / _ \ | '__| (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO / /__ | (_) | | (_) | | < | __/ | __/ | |_) | | __/ | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO /_____| \___/ \___/ |_|\_\ \___| \___| | .__/ \___| |_| (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO |_| (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,689 INFO (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:zookeeper.version=3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:host.name=uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc.cluster.local (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.version=11.0.14 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.vendor=Red Hat, Inc. (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.home=/usr/lib/jvm/java-11-openjdk-11.0.14.0.9-2.el8_5.x86_64 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.class.path=/opt/kafka/bin/../libs/accessors-smart-2.4.7.jar:/opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/annotations-13.0.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/automaton-1.11-8.jar:/opt/kafka/bin/../libs/checker-qual-3.5.0.jar:/opt/kafka/bin/../libs/commons-cli-1.4.jar:/opt/kafka/bin/../libs/commons-lang-2.6.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/connect-api-3.1.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-3.1.0.jar:/opt/kafka/bin/../libs/connect-file-3.1.0.jar:/opt/kafka/bin/../libs/connect-json-3.1.0.jar:/opt/kafka/bin/../libs/connect-mirror-3.1.0.jar:/opt/kafka/bin/../libs/connect-mirror-client-3.1.0.jar:/opt/kafka/bin/../libs/connect-runtime-3.1.0.jar:/opt/kafka/bin/../libs/connect-transforms-3.1.0.jar:/opt/kafka/bin/../libs/cruise-control-metrics-reporter-2.5.82.jar:/opt/kafka/bin/../libs/error_prone_annotations-2.3.4.jar:/opt/kafka/bin/../libs/failureaccess-1.0.1.jar:/opt/kafka/bin/../libs/generex-1.0.2.jar:/opt/kafka/bin/../libs/gson-2.8.6.jar:/opt/kafka/bin/../libs/guava-30.1-jre.jar:/opt/kafka/bin/../libs/hk2-api-2.6.1.jar:/opt/kafka/bin/../libs/hk2-locator-2.6.1.jar:/opt/kafka/bin/../libs/hk2-utils-2.6.1.jar:/opt/kafka/bin/../libs/j2objc-annotations-1.3.jar:/opt/kafka/bin/../libs/jackson-annotations-2.12.3.jar:/opt/kafka/bin/../libs/jackson-core-2.12.3.jar:/opt/kafka/bin/../libs/jackson-databind-2.12.3.jar:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.12.3.jar:/opt/kafka/bin/../libs/jackson-dataformat-yaml-2.12.3.jar:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.12.3.jar:/opt/kafka/bin/../libs/jackson-datatype-jsr310-2.12.3.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.12.3.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.12.3.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.12.3.jar:/opt/kafka/bin/../libs/jackson-module-scala_2.13-2.12.3.jar:/opt/kafka/bin/../libs/jaeger-client-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-core-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-thrift-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-tracerresolver-1.6.0.jar:/opt/kafka/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/kafka/bin/../libs/jakarta.inject-2.6.1.jar:/opt/kafka/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/kafka/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/kafka/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka/bin/../libs/javassist-3.27.0-GA.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.34.jar:/opt/kafka/bin/../libs/jersey-common-2.34.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.34.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/kafka/bin/../libs/jersey-hk2-2.34.jar:/opt/kafka/bin/../libs/jersey-server-2.34.jar:/opt/kafka/bin/../libs/jetty-client-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-continuation-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-http-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-io-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-security-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-server-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-servlet-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-servlets-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-util-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-util-ajax-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jline-3.12.1.jar:/opt/kafka/bin/../libs/jmx_prometheus_javaagent-0.16.1.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/jose4j-0.7.8.jar:/opt/kafka/bin/../libs/json-path-2.6.0.jar:/opt/kafka/bin/../libs/json-smart-2.4.7.jar:/opt/kafka/bin/../libs/jsonevent-layout-1.7.jar:/opt/kafka/bin/../libs/jsr305-3.0.2.jar:/opt/kafka/bin/../libs/kafka-agent-0.28.0.jar:/opt/kafka/bin/../libs/kafka-clients-3.1.0.jar:/opt/kafka/bin/../libs/kafka-env-var-config-provider-1.0.0.jar:/opt/kafka/bin/../libs/kafka-kubernetes-config-provider-1.0.0.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-3.1.0.jar:/opt/kafka/bin/../libs/kafka-metadata-3.1.0.jar:/opt/kafka/bin/../libs/kafka-oauth-client-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-common-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-keycloak-authorizer-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-server-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-server-plain-0.10.0.jar:/opt/kafka/bin/../libs/kafka-quotas-plugin-0.2.0.jar:/opt/kafka/bin/../libs/kafka-raft-3.1.0.jar:/opt/kafka/bin/../libs/kafka-server-common-3.1.0.jar:/opt/kafka/bin/../libs/kafka-shell-3.1.0.jar:/opt/kafka/bin/../libs/kafka-storage-3.1.0.jar:/opt/kafka/bin/../libs/kafka-storage-api-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-examples-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-scala_2.13-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-3.1.0.jar:/opt/kafka/bin/../libs/kafka-tools-3.1.0.jar:/opt/kafka/bin/../libs/kafka_2.13-3.1.0.jar:/opt/kafka/bin/../libs/kotlin-stdlib-1.4.10.jar:/opt/kafka/bin/../libs/kotlin-stdlib-common-1.4.0.jar:/opt/kafka/bin/../libs/kubernetes-client-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-apps-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-batch-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-certificates-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-common-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-core-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-extensions-5.12.0.jar:/opt/kafka/bin/../libs/libthrift-0.14.1.jar:/opt/kafka/bin/../libs/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/logging-interceptor-3.12.12.jar:/opt/kafka/bin/../libs/lz4-java-1.8.0.jar:/opt/kafka/bin/../libs/maven-artifact-3.8.1.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/metrics-core-4.1.12.1.jar:/opt/kafka/bin/../libs/mirror-maker-2-extensions-1.2.0.jar:/opt/kafka/bin/../libs/mirror-maker-agent-0.28.0.jar:/opt/kafka/bin/../libs/netty-buffer-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-codec-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-common-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-handler-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-resolver-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-epoll-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-unix-common-4.1.68.Final.jar:/opt/kafka/bin/../libs/nimbus-jose-jwt-9.10.jar:/opt/kafka/bin/../libs/okhttp-4.9.0.jar:/opt/kafka/bin/../libs/okio-2.8.0.jar:/opt/kafka/bin/../libs/opa-authorizer-1.4.0.jar:/opt/kafka/bin/../libs/opentracing-api-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-kafka-client-0.1.15.jar:/opt/kafka/bin/../libs/opentracing-noop-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-tracerresolver-0.1.8.jar:/opt/kafka/bin/../libs/opentracing-util-0.33.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/kafka/bin/../libs/paranamer-2.8.jar:/opt/kafka/bin/../libs/plexus-utils-3.2.1.jar:/opt/kafka/bin/../libs/reflections-0.9.12.jar:/opt/kafka/bin/../libs/rocksdbjni-6.22.1.1.jar:/opt/kafka/bin/../libs/scala-collection-compat_2.13-2.4.4.jar:/opt/kafka/bin/../libs/scala-java8-compat_2.13-1.0.0.jar:/opt/kafka/bin/../libs/scala-library-2.13.6.jar:/opt/kafka/bin/../libs/scala-logging_2.13-3.9.3.jar:/opt/kafka/bin/../libs/scala-reflect-2.13.6.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.30.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.30.jar:/opt/kafka/bin/../libs/snakeyaml-1.27.jar:/opt/kafka/bin/../libs/snappy-java-1.1.8.4.jar:/opt/kafka/bin/../libs/tracing-agent-0.28.0.jar:/opt/kafka/bin/../libs/trogdor-3.1.0.jar:/opt/kafka/bin/../libs/zjsonpatch-0.3.0.jar:/opt/kafka/bin/../libs/zookeeper-3.6.3.jar:/opt/kafka/bin/../libs/zookeeper-jute-3.6.3.jar:/opt/kafka/bin/../libs/zstd-jni-1.5.0-4.jar (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:os.version=5.4.170+ (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,692 INFO Server environment:user.name=kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO Server environment:user.home=/home/kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO Server environment:user.dir=/opt/kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO Server environment:os.memory.free=242MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO Server environment:os.memory.max=3118MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO Server environment:os.memory.total=260MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.enableEagerACLCheck = false (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.skipACL=="yes", ACL checks will be skipped (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.digest.enabled = true (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.closeSessionTxn.enabled = true (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.flushDelay=0 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.maxWriteQueuePollTime=0 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.maxBatchSize=1000 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,693 INFO zookeeper.intBufferStartingSizeBytes = 1024 (org.apache.zookeeper.server.ZooKeeperServer) [main]
2022-05-03 09:15:52,699 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
2022-05-03 09:15:52,700 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
2022-05-03 09:15:52,701 INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [main]
2022-05-03 09:15:52,701 INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase) [main]
2022-05-03 09:15:52,712 INFO Using TLS encrypted quorum communication (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,712 INFO Port unification disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO multiAddress.enabled set to false (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO multiAddress.reachabilityCheckEnabled set to true (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO multiAddress.reachabilityCheckTimeoutMs set to 1000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO QuorumPeer communication is not secured! (SASL auth disabled) (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO quorum.cnxn.threads.size set to 20 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,713 INFO Reading snapshot /var/lib/zookeeper/data/version-2/snapshot.0 (org.apache.zookeeper.server.persistence.FileSnap) [main]
2022-05-03 09:15:52,718 INFO The digest value is empty in snapshot (org.apache.zookeeper.server.DataTree) [main]
2022-05-03 09:15:52,721 INFO Snapshot loaded in 8 ms, highest zxid is 0x0, digest is 1371985504 (org.apache.zookeeper.server.ZKDatabase) [main]
2022-05-03 09:15:52,722 INFO binding to port /127.0.0.1:12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,812 INFO bound to port 12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,812 INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,814 INFO bound to port 2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
2022-05-03 09:15:52,818 INFO jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.14+9-LTS (org.eclipse.jetty.server.Server) [main]
2022-05-03 09:15:52,852 INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [main]
2022-05-03 09:15:52,852 INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [main]
2022-05-03 09:15:52,854 INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [main]
2022-05-03 09:15:52,857 WARN ServletContext@o.e.j.s.ServletContextHandler@1a75e76a{/,null,STARTING}<mailto:ServletContext@o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,STARTING%7d> has uncovered http methods for path: /* (org.eclipse.jetty.security.SecurityHandler) [main]
2022-05-03 09:15:52,866 INFO Started o.e.j.s.ServletContextHandler@1a75e76a{/,null,AVAILABLE}<mailto:o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,AVAILABLE%7d> (org.eclipse.jetty.server.handler.ContextHandler) [main]
2022-05-03 09:15:52,873 INFO Started ServerConnector@60bd273d{HTTP/1.1<mailto:ServerConnector@60bd273d%7bHTTP/1.1>, (http/1.1)}{0.0.0.0:8080} (org.eclipse.jetty.server.AbstractConnector) [main]
2022-05-03 09:15:52,873 INFO Started @1299ms (org.eclipse.jetty.server.Server) [main]
2022-05-03 09:15:52,874 INFO Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer) [main]
2022-05-03 09:15:52,875 INFO Using 4000ms as the quorum cnxn socket timeout (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
2022-05-03 09:15:52,884 INFO Election port bind maximum retries is 3 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [main]
2022-05-03 09:15:52,893 INFO zookeeper.fastleader.minNotificationInterval=200 (org.apache.zookeeper.server.quorum.FastLeaderElection) [main]
2022-05-03 09:15:52,894 INFO zookeeper.fastleader.maxNotificationInterval=60000 (org.apache.zookeeper.server.quorum.FastLeaderElection) [main]
2022-05-03 09:15:52,894 INFO Creating TLS-only quorum server socket (org.apache.zookeeper.server.quorum.QuorumCnxManager) [ListenerHandler-uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888]
2022-05-03 09:15:52,898 INFO 2 is accepting connections now, my election bind port: uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [ListenerHandler-uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888]
2022-05-03 09:15:52,901 INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider) [main]
2022-05-03 09:15:52,922 INFO LOOKING (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:52,938 INFO New election. My id = 2, proposed zxid=0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:52,951 INFO Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2, n.round:0x1, n.peerEpoch:0x0, n.zxid:0x0, message format version:0x2, n.config version:0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=2]]
2022-05-03 09:15:53,164 INFO Notification time out: 400 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:53,565 INFO Notification time out: 800 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:54,366 INFO Notification time out: 1600 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:55,966 INFO Notification time out: 3200 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:15:58,440 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:15:58,440 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-2]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:15:59,168 INFO Notification time out: 6400 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:16:04,173 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:04,174 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:05,570 INFO Notification time out: 12800 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:16:09,563 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:09,616 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:10,576 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:10,576 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:18,371 INFO Notification time out: 25600 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
2022-05-03 09:16:19,531 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:19,583 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:23,377 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:23,377 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:29,538 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-05-03 09:16:29,596 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
java.lang.NullPointerException
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
get Operator log
Kind Regards,
Sree
____________________________________________________
[https://brandportal.intranet.db.com/img/modules/logo.gif]
Sreejesh Radhakrishnan
Deutsche Bank AG, Filiale London
CB & IB Tech
10 Upper Bank Street, Canary Wharf, E14 5JJ London, United Kingdom
Mobile +44 7918-824-905
Email sreejesh.radhakrishnan@db.com<ma...@db.com>
[https://brandportal.intranet.db.com/img/modules/positive_impact_ENG.gif]<https://www.db.com/company/en/positiveimpact.htm>
---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
Re: ZK crashing wit NPE ( WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2] java.lang.NullPointerException
Posted by Flavio Junqueira <fp...@apache.org>.
Hello Sreejesh,
My read is that servers are not being able to connect to each other for leader election.
-Flavio
> On 3 May 2022, at 15:50, Sreejesh Radhakrishnan <sr...@db.com.INVALID> wrote:
>
> Classification: Public
>
> Hi
>
> Sorry not sure if this is the right email. But still trying my luck,
>
> I am getting an NPE as pointed on the subject line, I am using Strimzi KAFKA on GKE (1.21.9-gke.1002) using n1-standard-4 infra.
>
> Zookeeper is throwing below error, is that something which you guys have seen or have idea of what causing it?
>
> get ZK log
>
> Detected Zookeeper ID 2
> Preparing truststore
> Adding /opt/kafka/cluster-ca-certs/ca.crt to truststore /tmp/zookeeper/cluster.truststore.p12 with alias ca
> Certificate was added to keystore
> Preparing truststore is complete
> Looking for the right CA
> Found the right CA: /opt/kafka/cluster-ca-certs/ca.crt
> Preparing keystore for client and quorum listeners
> Preparing keystore for client and quorum listeners is complete
> Starting Zookeeper with configuration:
> # The directory where the snapshot is stored.
> dataDir=/var/lib/zookeeper/data
>
> # Other options
> 4lw.commands.whitelist=*
> standaloneEnabled=false
> reconfigEnabled=true
> clientPort=12181
> clientPortAddress=127.0.0.1
>
> # TLS options
> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
> ssl.clientAuth=need
> ssl.quorum.clientAuth=need
> secureClientPort=2181
> sslQuorum=true
>
> ssl.trustStore.location=/tmp/zookeeper/cluster.truststore.p12
> ssl.trustStore.***
> ssl.trustStore.type=PKCS12
> ssl.quorum.trustStore.location=/tmp/zookeeper/cluster.truststore.p12
> ssl.quorum.trustStore.***
> ssl.quorum.trustStore.type=PKCS12
>
> ssl.keyStore.location=/tmp/zookeeper/cluster.keystore.p12
> ssl.keyStore.***
> ssl.keyStore.type=PKCS12
> ssl.quorum.keyStore.location=/tmp/zookeeper/cluster.keystore.p12
> ssl.quorum.keyStore.***
> ssl.quorum.keyStore.type=PKCS12
>
> # Provided configuration
> tickTime=2000
> initLimit=5
> syncLimit=2
> autopurge.purgeInterval=1
>
>
> # Zookeeper nodes configuration
> server.1=uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
> server.2=uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
> server.3=uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc:2888:3888:participant;127.0.0.1:12181
>
> + exec /usr/bin/tini -w -e 143 -- /opt/kafka/bin/zookeeper-server-start.sh /tmp/zookeeper.properties
> 2022-05-03 09:15:52,181 INFO Reading configuration from: /tmp/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
> 2022-05-03 09:15:52,193 INFO clientPortAddress is 127.0.0.1:12181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
> 2022-05-03 09:15:52,193 INFO secureClientPortAddress is 0.0.0.0:2181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
> 2022-05-03 09:15:52,197 INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [main]
> 2022-05-03 09:15:52,198 INFO observerMasterPort is not set (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
> 2022-05-03 09:15:52,198 INFO metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [main]
> 2022-05-03 09:15:52,233 INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager) [main]
> 2022-05-03 09:15:52,233 INFO autopurge.purgeInterval set to 1 (org.apache.zookeeper.server.DatadirCleanupManager) [main]
> 2022-05-03 09:15:52,241 INFO Log4j 1.2 jmx support found and enabled. (org.apache.zookeeper.jmx.ManagedUtil) [main]
> 2022-05-03 09:15:52,235 INFO Purge task started. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask]
> 2022-05-03 09:15:52,266 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [PurgeTask]
> 2022-05-03 09:15:52,268 INFO Starting quorum peer, myid=2 (org.apache.zookeeper.server.quorum.QuorumPeerMain) [main]
> 2022-05-03 09:15:52,291 INFO zookeeper.snapshot.compression.method = CHECKED (org.apache.zookeeper.server.persistence.SnapStream) [PurgeTask]
> 2022-05-03 09:15:52,298 INFO Purge task completed. (org.apache.zookeeper.server.DatadirCleanupManager) [PurgeTask]
> 2022-05-03 09:15:52,306 INFO ServerMetrics initialized with provider org.apache.zookeeper.metrics.impl.DefaultMetricsProvider@5276e6b0 <ma...@5276e6b0> (org.apache.zookeeper.server.ServerMetrics) [main]
> 2022-05-03 09:15:52,404 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,404 INFO zookeeper.netty.advancedFlowControl.enabled = false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,404 INFO handshakeThrottlingEnabled = false, zookeeper.netty.server.outstandingHandshake.limit = -1 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,464 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main]
> 2022-05-03 09:15:52,466 WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [main]
> 2022-05-03 09:15:52,466 INFO zookeeper.client.portUnification=false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,466 INFO zookeeper.netty.advancedFlowControl.enabled = false (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,466 INFO handshakeThrottlingEnabled = false, zookeeper.netty.server.outstandingHandshake.limit = -1 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,467 INFO Using org.apache.zookeeper.server.NettyServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [main]
> 2022-05-03 09:15:52,467 WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [main]
> 2022-05-03 09:15:52,476 INFO zookeeper.quorumCnxnTimeoutMs=-1 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,507 INFO Logging initialized @932ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log) [main]
> 2022-05-03 09:15:52,633 WARN o.e.j.s.ServletContextHandler@1a75e76a{/,null,STOPPED} <mailto:o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,STOPPED%7d> contextPath ends with /* (org.eclipse.jetty.server.handler.ContextHandler) [main]
> 2022-05-03 09:15:52,633 WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler) [main]
> 2022-05-03 09:15:52,666 INFO zookeeper.snapshot.trust.empty : false (org.apache.zookeeper.server.persistence.FileTxnSnapLog) [main]
> 2022-05-03 09:15:52,666 INFO Local sessions disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO Local session upgrading disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO tickTime set to 2000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO minSessionTimeout set to 4000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO maxSessionTimeout set to 40000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO initLimit set to 5 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO syncLimit set to 2 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,667 INFO connectToLearnerMasterLimit set to 0 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,686 INFO (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,688 INFO ______ _ (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO |___ / | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO / / ___ ___ | | __ ___ ___ _ __ ___ _ __ (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO / / / _ \ / _ \ | |/ / / _ \ / _ \ | '_ \ / _ \ | '__| (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO / /__ | (_) | | (_) | | < | __/ | __/ | |_) | | __/ | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO /_____| \___/ \___/ |_|\_\ \___| \___| | .__/ \___| |_| (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO | | (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO |_| (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,689 INFO (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:zookeeper.version=3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:host.name=uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc.cluster.local (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.version=11.0.14 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.vendor=Red Hat, Inc. (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.home=/usr/lib/jvm/java-11-openjdk-11.0.14.0.9-2.el8_5.x86_64 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.class.path=/opt/kafka/bin/../libs/accessors-smart-2.4.7.jar:/opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/annotations-13.0.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/automaton-1.11-8.jar:/opt/kafka/bin/../libs/checker-qual-3.5.0.jar:/opt/kafka/bin/../libs/commons-cli-1.4.jar:/opt/kafka/bin/../libs/commons-lang-2.6.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/connect-api-3.1.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-3.1.0.jar:/opt/kafka/bin/../libs/connect-file-3.1.0.jar:/opt/kafka/bin/../libs/connect-json-3.1.0.jar:/opt/kafka/bin/../libs/connect-mirror-3.1.0.jar:/opt/kafka/bin/../libs/connect-mirror-client-3.1.0.jar:/opt/kafka/bin/../libs/connect-runtime-3.1.0.jar:/opt/kafka/bin/../libs/connect-transforms-3.1.0.jar:/opt/kafka/bin/../libs/cruise-control-metrics-reporter-2.5.82.jar:/opt/kafka/bin/../libs/error_prone_annotations-2.3.4.jar:/opt/kafka/bin/../libs/failureaccess-1.0.1.jar:/opt/kafka/bin/../libs/generex-1.0.2.jar:/opt/kafka/bin/../libs/gson-2.8.6.jar:/opt/kafka/bin/../libs/guava-30.1-jre.jar:/opt/kafka/bin/../libs/hk2-api-2.6.1.jar:/opt/kafka/bin/../libs/hk2-locator-2.6.1.jar:/opt/kafka/bin/../libs/hk2-utils-2.6.1.jar:/opt/kafka/bin/../libs/j2objc-annotations-1.3.jar:/opt/kafka/bin/../libs/jackson-annotations-2.12.3.jar:/opt/kafka/bin/../libs/jackson-core-2.12.3.jar:/opt/kafka/bin/../libs/jackson-databind-2.12.3.jar:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.12.3.jar:/opt/kafka/bin/../libs/jackson-dataformat-yaml-2.12.3.jar:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.12.3.jar:/opt/kafka/bin/../libs/jackson-datatype-jsr310-2.12.3.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.12.3.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.12.3.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.12.3.jar:/opt/kafka/bin/../libs/jackson-module-scala_2.13-2.12.3.jar:/opt/kafka/bin/../libs/jaeger-client-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-core-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-thrift-1.6.0.jar:/opt/kafka/bin/../libs/jaeger-tracerresolver-1.6.0.jar:/opt/kafka/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/kafka/bin/../libs/jakarta.inject-2.6.1.jar:/opt/kafka/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/kafka/bin/../libs/jakarta.ws.rs <http://jakarta.ws.rs/>-api-2.1.6.jar:/opt/kafka/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka/bin/../libs/javassist-3.27.0-GA.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs <http://javax.ws.rs/>-api-2.1.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.34.jar:/opt/kafka/bin/../libs/jersey-common-2.34.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.34.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/kafka/bin/../libs/jersey-hk2-2.34.jar:/opt/kafka/bin/../libs/jersey-server-2.34.jar:/opt/kafka/bin/../libs/jetty-client-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-continuation-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-http-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-io-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-security-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-server-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-servlet-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-servlets-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-util-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jetty-util-ajax-9.4.43.v20210629.jar:/opt/kafka/bin/../libs/jline-3.12.1.jar:/opt/kafka/bin/../libs/jmx_prometheus_javaagent-0.16.1.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/jose4j-0.7.8.jar:/opt/kafka/bin/../libs/json-path-2.6.0.jar:/opt/kafka/bin/../libs/json-smart-2.4.7.jar:/opt/kafka/bin/../libs/jsonevent-layout-1.7.jar:/opt/kafka/bin/../libs/jsr305-3.0.2.jar:/opt/kafka/bin/../libs/kafka-agent-0.28.0.jar:/opt/kafka/bin/../libs/kafka-clients-3.1.0.jar:/opt/kafka/bin/../libs/kafka-env-var-config-provider-1.0.0.jar:/opt/kafka/bin/../libs/kafka-kubernetes-config-provider-1.0.0.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-3.1.0.jar:/opt/kafka/bin/../libs/kafka-metadata-3.1.0.jar:/opt/kafka/bin/../libs/kafka-oauth-client-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-common-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-keycloak-authorizer-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-server-0.10.0.jar:/opt/kafka/bin/../libs/kafka-oauth-server-plain-0.10.0.jar:/opt/kafka/bin/../libs/kafka-quotas-plugin-0.2.0.jar:/opt/kafka/bin/../libs/kafka-raft-3.1.0.jar:/opt/kafka/bin/../libs/kafka-server-common-3.1.0.jar:/opt/kafka/bin/../libs/kafka-shell-3.1.0.jar:/opt/kafka/bin/../libs/kafka-storage-3.1.0.jar:/opt/kafka/bin/../libs/kafka-storage-api-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-examples-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-scala_2.13-3.1.0.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-3.1.0.jar:/opt/kafka/bin/../libs/kafka-tools-3.1.0.jar:/opt/kafka/bin/../libs/kafka_2.13-3.1.0.jar:/opt/kafka/bin/../libs/kotlin-stdlib-1.4.10.jar:/opt/kafka/bin/../libs/kotlin-stdlib-common-1.4.0.jar:/opt/kafka/bin/../libs/kubernetes-client-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-apps-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-batch-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-certificates-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-common-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-core-5.12.0.jar:/opt/kafka/bin/../libs/kubernetes-model-extensions-5.12.0.jar:/opt/kafka/bin/../libs/libthrift-0.14.1.jar:/opt/kafka/bin/../libs/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/logging-interceptor-3.12.12.jar:/opt/kafka/bin/../libs/lz4-java-1.8.0.jar:/opt/kafka/bin/../libs/maven-artifact-3.8.1.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/metrics-core-4.1.12.1.jar:/opt/kafka/bin/../libs/mirror-maker-2-extensions-1.2.0.jar:/opt/kafka/bin/../libs/mirror-maker-agent-0.28.0.jar:/opt/kafka/bin/../libs/netty-buffer-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-codec-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-common-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-handler-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-resolver-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-epoll-4.1.68.Final.jar:/opt/kafka/bin/../libs/netty-transport-native-unix-common-4.1.68.Final.jar:/opt/kafka/bin/../libs/nimbus-jose-jwt-9.10.jar:/opt/kafka/bin/../libs/okhttp-4.9.0.jar:/opt/kafka/bin/../libs/okio-2.8.0.jar:/opt/kafka/bin/../libs/opa-authorizer-1.4.0.jar:/opt/kafka/bin/../libs/opentracing-api-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-kafka-client-0.1.15.jar:/opt/kafka/bin/../libs/opentracing-noop-0.33.0.jar:/opt/kafka/bin/../libs/opentracing-tracerresolver-0.1.8.jar:/opt/kafka/bin/../libs/opentracing-util-0.33.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/kafka/bin/../libs/paranamer-2.8.jar:/opt/kafka/bin/../libs/plexus-utils-3.2.1.jar:/opt/kafka/bin/../libs/reflections-0.9.12.jar:/opt/kafka/bin/../libs/rocksdbjni-6.22.1.1.jar:/opt/kafka/bin/../libs/scala-collection-compat_2.13-2.4.4.jar:/opt/kafka/bin/../libs/scala-java8-compat_2.13-1.0.0.jar:/opt/kafka/bin/../libs/scala-library-2.13.6.jar:/opt/kafka/bin/../libs/scala-logging_2.13-3.9.3.jar:/opt/kafka/bin/../libs/scala-reflect-2.13.6.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.30.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.30.jar:/opt/kafka/bin/../libs/snakeyaml-1.27.jar:/opt/kafka/bin/../libs/snappy-java-1.1.8.4.jar:/opt/kafka/bin/../libs/tracing-agent-0.28.0.jar:/opt/kafka/bin/../libs/trogdor-3.1.0.jar:/opt/kafka/bin/../libs/zjsonpatch-0.3.0.jar:/opt/kafka/bin/../libs/zookeeper-3.6.3.jar:/opt/kafka/bin/../libs/zookeeper-jute-3.6.3.jar:/opt/kafka/bin/../libs/zstd-jni-1.5.0-4.jar (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.io.tmpdir=/tmp (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:java.compiler=<NA> (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:os.name=Linux (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:os.arch=amd64 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:os.version=5.4.170+ (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,692 INFO Server environment:user.name=kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO Server environment:user.home=/home/kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO Server environment:user.dir=/opt/kafka (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO Server environment:os.memory.free=242MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO Server environment:os.memory.max=3118MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO Server environment:os.memory.total=260MB (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.enableEagerACLCheck = false (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.skipACL=="yes", ACL checks will be skipped (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.digest.enabled = true (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.closeSessionTxn.enabled = true (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.flushDelay=0 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.maxWriteQueuePollTime=0 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.maxBatchSize=1000 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,693 INFO zookeeper.intBufferStartingSizeBytes = 1024 (org.apache.zookeeper.server.ZooKeeperServer) [main]
> 2022-05-03 09:15:52,699 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
> 2022-05-03 09:15:52,700 INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [main]
> 2022-05-03 09:15:52,701 INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [main]
> 2022-05-03 09:15:52,701 INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase) [main]
> 2022-05-03 09:15:52,712 INFO Using TLS encrypted quorum communication (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,712 INFO Port unification disabled (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO multiAddress.enabled set to false (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO multiAddress.reachabilityCheckEnabled set to true (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO multiAddress.reachabilityCheckTimeoutMs set to 1000 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO QuorumPeer communication is not secured! (SASL auth disabled) (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO quorum.cnxn.threads.size set to 20 (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,713 INFO Reading snapshot /var/lib/zookeeper/data/version-2/snapshot.0 (org.apache.zookeeper.server.persistence.FileSnap) [main]
> 2022-05-03 09:15:52,718 INFO The digest value is empty in snapshot (org.apache.zookeeper.server.DataTree) [main]
> 2022-05-03 09:15:52,721 INFO Snapshot loaded in 8 ms, highest zxid is 0x0, digest is 1371985504 (org.apache.zookeeper.server.ZKDatabase) [main]
> 2022-05-03 09:15:52,722 INFO binding to port /127.0.0.1:12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,812 INFO bound to port 12181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,812 INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,814 INFO bound to port 2181 (org.apache.zookeeper.server.NettyServerCnxnFactory) [main]
> 2022-05-03 09:15:52,818 INFO jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.14+9-LTS (org.eclipse.jetty.server.Server) [main]
> 2022-05-03 09:15:52,852 INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [main]
> 2022-05-03 09:15:52,852 INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [main]
> 2022-05-03 09:15:52,854 INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [main]
> 2022-05-03 09:15:52,857 WARN ServletContext@o.e.j.s.ServletContextHandler@1a75e76a{/,null,STARTING} <mailto:ServletContext@o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,STARTING%7d> has uncovered http methods for path: /* (org.eclipse.jetty.security.SecurityHandler) [main]
> 2022-05-03 09:15:52,866 INFO Started o.e.j.s.ServletContextHandler@1a75e76a{/,null,AVAILABLE} <mailto:o.e.j.s.ServletContextHandler@1a75e76a%7b/,null,AVAILABLE%7d> (org.eclipse.jetty.server.handler.ContextHandler) [main]
> 2022-05-03 09:15:52,873 INFO Started ServerConnector@60bd273d{HTTP/1.1 <mailto:ServerConnector@60bd273d%7bHTTP/1.1>, (http/1.1)}{0.0.0.0:8080} (org.eclipse.jetty.server.AbstractConnector) [main]
> 2022-05-03 09:15:52,873 INFO Started @1299ms (org.eclipse.jetty.server.Server) [main]
> 2022-05-03 09:15:52,874 INFO Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer) [main]
> 2022-05-03 09:15:52,875 INFO Using 4000ms as the quorum cnxn socket timeout (org.apache.zookeeper.server.quorum.QuorumPeer) [main]
> 2022-05-03 09:15:52,884 INFO Election port bind maximum retries is 3 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [main]
> 2022-05-03 09:15:52,893 INFO zookeeper.fastleader.minNotificationInterval=200 (org.apache.zookeeper.server.quorum.FastLeaderElection) [main]
> 2022-05-03 09:15:52,894 INFO zookeeper.fastleader.maxNotificationInterval=60000 (org.apache.zookeeper.server.quorum.FastLeaderElection) [main]
> 2022-05-03 09:15:52,894 INFO Creating TLS-only quorum server socket (org.apache.zookeeper.server.quorum.QuorumCnxManager) [ListenerHandler-uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888]
> 2022-05-03 09:15:52,898 INFO 2 is accepting connections now, my election bind port: uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [ListenerHandler-uwq-cluster-zookeeper-1.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.38:3888]
> 2022-05-03 09:15:52,901 INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider) [main]
> 2022-05-03 09:15:52,922 INFO LOOKING (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:52,938 INFO New election. My id = 2, proposed zxid=0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:52,951 INFO Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2, n.round:0x1, n.peerEpoch:0x0, n.zxid:0x0, message format version:0x2, n.config version:0x0 (org.apache.zookeeper.server.quorum.FastLeaderElection) [WorkerReceiver[myid=2]]
> 2022-05-03 09:15:53,164 INFO Notification time out: 400 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:53,565 INFO Notification time out: 800 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:54,366 INFO Notification time out: 1600 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:55,966 INFO Notification time out: 3200 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:15:58,440 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:15:58,440 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-2]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:15:59,168 INFO Notification time out: 6400 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:16:04,173 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:04,174 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:05,570 INFO Notification time out: 12800 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:16:09,563 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:09,616 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:10,576 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:10,576 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:18,371 INFO Notification time out: 25600 (org.apache.zookeeper.server.quorum.FastLeaderElection) [QuorumPeer[myid=2](plain=127.0.0.1:12181)(secure=0.0.0.0:2181)]
> 2022-05-03 09:16:19,531 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:19,583 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:23,377 WARN Cannot open channel to 1 at election address uwq-cluster-zookeeper-0.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.68:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-3]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:23,377 WARN Cannot open channel to 3 at election address uwq-cluster-zookeeper-2.uwq-cluster-zookeeper-nodes.uwq-kafka.svc/240.1.185.170:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager) [QuorumConnectionThread-[myid=2]-1]
> java.net.SocketTimeoutException: connect timed out
> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.base/java.net.Socket.connect(Socket.java:609)
> at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:300)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
> at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:29,538 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-1]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2022-05-03 09:16:29,596 WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory) [nioEventLoopGroup-4-2]
> java.lang.NullPointerException
> at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelActive(NettyServerCnxnFactory.java:260)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
> at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:522)
> at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
> at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
> at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> get Operator log
>
>
> Kind Regards,
>
> Sree
>
> ____________________________________________________
>
>
>
> Sreejesh Radhakrishnan
>
> Deutsche Bank AG, Filiale London
> CB & IB Tech
> 10 Upper Bank Street, Canary Wharf, E14 5JJ London, United Kingdom
> Mobile +44 7918-824-905
> Email sreejesh.radhakrishnan@db.com <ma...@db.com>
>
> <https://www.db.com/company/en/positiveimpact.htm>
>
>
>
> ---
> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
>
> Please refer to https://www.db.com/disclosures <https://www.db.com/disclosures> for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm <http://www.db.com/unitedkingdom/content/privacy.htm> for information about privacy.