You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@zookeeper.apache.org by "Sai kiran (Jira)" <ji...@apache.org> on 2022/05/06 11:36:00 UTC
[jira] [Created] (ZOOKEEPER-4536) Zookeeper quorum formation fails when TLS is enabled in k8s env
Sai kiran created ZOOKEEPER-4536:
------------------------------------
Summary: Zookeeper quorum formation fails when TLS is enabled in k8s env
Key: ZOOKEEPER-4536
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4536
Project: ZooKeeper
Issue Type: Bug
Components: leaderElection, quorum
Affects Versions: 3.7.0
Environment: Kubernetes 1.21.1
Reporter: Sai kiran
We have three(3) node zookeeper cluster running as a pod on Kubernetes cluster, zookeeper quorum formation fails with TLS handshake error, as the server name in the https request does not match with any of the SANs in the certificate configured for zookeeper server. Server name in the request is of the form "x-x-x-x.kubernetes.default.svc.cluster.local" (where x-x-x-x is the IP address of the POD), and I am unable to understand the reason behind pre-pending FQDN with a IP address.
Please find below the extract of the error logs from the zookeeper POD
{code:java}
2022-04-12T12:48:03.551+0200 [myid:] - ERROR [ListenerHandler-0.0.0.0/0.0.0.0:3888:ZKTrustManager@161] - Failed to verify host address: 192.168.140.200
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <192.168.140.200> doesn't match any of the subject alternative names: [eric-data-coordinator-zk, eric-data-coordinator-zk.eda-esmalir, eric-data-coordinator-zk.eda-esmalir.svc, eric-data-coordinator-zk.eda-esmalir.svc.cluster.local, *.eric-data-coordinator-zk-ensemble-service.eda-esmalir.svc.cluster.local, certified-scrape-target]
at org.apache.zookeeper.common.ZKHostnameVerifier.matchIPAddress(ZKHostnameVerifier.java:197) ~[zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.common.ZKHostnameVerifier.verify(ZKHostnameVerifier.java:165) ~[zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.common.ZKTrustManager.performHostVerification(ZKTrustManager.java:151) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.common.ZKTrustManager.checkClientTrusted(ZKTrustManager.java:79) [zookeeper-3.7.0.jar:3.7.0]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:688) [?:?]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:411) [?:?]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:375) [?:?]
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) [?:?]
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443) [?:?]
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421) [?:?]
at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) [?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) [?:?]
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1426) [?:?]
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1336) [?:?]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450) [?:?]
at sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:841) [?:?]
at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:366) [?:?]
at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.detectMode(UnifiedServerSocket.java:269) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.getSocket(UnifiedServerSocket.java:298) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.access$400(UnifiedServerSocket.java:172) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedInputStream.getRealInputStream(UnifiedServerSocket.java:699) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedInputStream.read(UnifiedServerSocket.java:693) [zookeeper-3.7.0.jar:3.7.0]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:252) [?:?]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:292) [?:?]
at java.io.BufferedInputStream.read(BufferedInputStream.java:351) [?:?]
at java.io.DataInputStream.readFully(DataInputStream.java:200) [?:?]
at java.io.DataInputStream.readLong(DataInputStream.java:421) [?:?]
at org.apache.zookeeper.server.quorum.QuorumCnxManager.handleConnection(QuorumCnxManager.java:602) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:555) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener$ListenerHandler.acceptConnections(QuorumCnxManager.java:1080) [zookeeper-3.7.0.jar:3.7.0]
at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener$ListenerHandler.run(QuorumCnxManager.java:1034) [zookeeper-3.7.0.jar:3.7.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)