[jira] [Created] (KAFKA-7376) After Kafka upgrade to v2.0.0 , Controller unable to communicate with brokers on SASL_SSL

["Sridhar (JIRA)" <ji...@apache.org>]

Sridhar created KAFKA-7376:
------------------------------

             Summary: After Kafka upgrade to v2.0.0 , Controller unable to communicate with brokers on SASL_SSL
                 Key: KAFKA-7376
                 URL: https://issues.apache.org/jira/browse/KAFKA-7376
             Project: Kafka
          Issue Type: Bug
          Components: controller
    Affects Versions: 2.0.0
            Reporter: Sridhar


Hi ,

We upgraded our Kafka cluster (3x nodes running on AWS cloud) to 2.0.0 version and enabled security with SASL_SSL (plain) encryption for Inter-broker and Client connection . 

But there are lot of errors in the controller log for the inter-broker communication .I have the followed exactly same steps as mentioned in the document and set all kafka brokers fqdn hostname in the SAN (SubjectAlternativeName) of my server certificate (selfsigned) .

[http://kafka.apache.org/documentation.html#security|http://example.com/]

 

openssl s_client -connect kafka-reghub-d2-3:9093
CONNECTED(00000003)
depth=1

Noticed someone else also facing the similar problem .

[https://github.com/confluentinc/common/issues/158]

 

 
{noformat}
Server Configuration : 
listeners=PLAINTEXT://kafka-3:9092,SASL_SSL://kafka-3:9093
advertised.listeners=PLAINTEXT://kafka-3:9092,SASL_SSL://kafka-3:9093
#Security
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=false
security.inter.broker.protocol=SASL_SSL
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
super.users=User:admin
ssl.client.auth=required
ssl.endpoint.identification.algorithm=
ssl.truststore.location=/etc/kafka/ssl/kafka.server.truststore.jks
ssl.truststore.password=
ssl.keystore.location=/etc/kafka/ssl/kafka.server.keystore.jks
ssl.keystore.password=
ssl.key.password=

#Zookeeper
zookeeper.connect=zk-reghub-d2-1:2181,zk-reghub-d2-2:2181,zk-reghub-d2-3:2181
zookeeper.connection.timeout.ms=6000
{noformat}
 

 
{code:java}
 
[2018-09-04 12:02:57,289] WARN [RequestSendThread controllerId=2] Controller 2's connection to broker kafka-3:9093 (id: 3 rack: eu-central-1c) was unsuccessful (kafka.controller.RequestSendThread)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1529)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:439)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:304)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:258)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:134)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:487)
at org.apache.kafka.common.network.Selector.poll(Selector.java:425)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:510)
at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:73)
at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:279)
at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:233)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:330)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1614)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:992)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:989)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1467)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:393)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:473)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:331)
... 9 more
Caused by: java.security.cert.CertificateException: No name matching kafka-3 found
at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:231)
at sun.security.util.HostnameChecker.match(HostnameChecker.java:96)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:455)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1601)
... 18 more
{code}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)