You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Lee Hyunjung <vi...@gmail.com> on 2016/02/14 10:47:07 UTC
SSL Mirror Maker
Hi,
I've set up broker ssl successfully. Here is the detail.
*[broker]*
1. run shell script (generate ssl key, certificate and CA and sign the
certificate)
2. here is server.properties on brokers.
listeners=PLAINTEXT://:9092,SSL://:9093
ssl.keystore.location=/opt/kafka/keys/server.keystore.jks
ssl.keystore.password=test1234
ssl.key.password=test1234
ssl.truststore.location=/opt/kafka/keys/server.truststore.jks
ssl.truststore.password=test1234
ssl.client.auth=required
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
After that I've checked server.log and it has below info.
with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL
-> EndPoint(192.168.64.1,9093,SSL)
And I also run below command and can see the proper server's certificate.
openssl s_client -debug -connect localhost:9093 -tls1
But When I run mirror maker process on the other mirror maker machine,
I got error.
*[Mirror Maker]*
1. I've run the same shell script which I've run for broker. (generate
ssl key, certificate and CA and sign the certificate)
2. Here is my mirror maker consumer configuration.
bootstrap.servers=brokerhost:9093
group.id=kafkaMirror
security.protocol=SSL
ssl.truststore.location=/opt/kafka/keys/client.truststore.jks
ssl.truststore.password=test1234
ssl.enabled.protocols=TLSv1
ssl.keystore.location=/opt/kafka/keys/client.keystore.jks
ssl.keystore.password=test1234
ssl.key.password=test1234
After that I tried below command. Below 3 command got same error.
bin/kafka-mirror-maker.sh --new.consumer --consumer.config
config/ssl_consumer.properties --producer.config config/producer.properties
--num.streams 10 --whitelist=test
bin/kafka-console-producer.sh --broker-list brokerhost:9093 --topic test
--producer.config config/ssl_client.properties
bin/kafka-console-consumer.sh --bootstrap-server brokerhost:9093 --topic
test --new-consumer --consumer.config config/ssl_client.properties
DEBUG Connection with {broker host}/{ip} disconnected
(org.apache.kafka.common.network.Selector)
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431)
at
sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
at
sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at
org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:377)
at
org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:242)
at
org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:68)
at org.apache.kafka.common.network.Selector.poll(Selector.java:281)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1478)
at
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:212)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:919)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:916)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369)
at
org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:335)
at
org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:413)
at
org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:269)
... 6 more
Caused by: sun.security.validator.ValidatorException: PKIX path building
failed: sun.security.provider.certpath.SunCertPathBuilderException: unable
to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at
sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at
sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at
sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
at
sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1465)
... 15 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
at
sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:145)
at
sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
t sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
How can I fix this error?
Thanks.
Re: SSL Mirror Maker
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Lee,
Is the CA used to sign the client certificates in the server truststore and
the CA used to sign the server certificates in the client truststore? See
the following blog post for a working example (including a Vagrant setup):
http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption
Hope it helps.
Ismael
On 14 Feb 2016 16:27, "Lee Hyunjung" <vi...@gmail.com> wrote:
> Hi,
>
> I've set up broker ssl successfully. Here is the detail.
>
> *[broker]*
> 1. run shell script (generate ssl key, certificate and CA and sign the
> certificate)
> 2. here is server.properties on brokers.
>
> listeners=PLAINTEXT://:9092,SSL://:9093
> ssl.keystore.location=/opt/kafka/keys/server.keystore.jks
> ssl.keystore.password=test1234
> ssl.key.password=test1234
> ssl.truststore.location=/opt/kafka/keys/server.truststore.jks
> ssl.truststore.password=test1234
> ssl.client.auth=required
> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
>
> After that I've checked server.log and it has below info.
>
> with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL
> -> EndPoint(192.168.64.1,9093,SSL)
>
> And I also run below command and can see the proper server's certificate.
> openssl s_client -debug -connect localhost:9093 -tls1
>
>
>
> But When I run mirror maker process on the other mirror maker machine,
> I got error.
>
> *[Mirror Maker]*
> 1. I've run the same shell script which I've run for broker. (generate
> ssl key, certificate and CA and sign the certificate)
> 2. Here is my mirror maker consumer configuration.
> bootstrap.servers=brokerhost:9093
> group.id=kafkaMirror
> security.protocol=SSL
> ssl.truststore.location=/opt/kafka/keys/client.truststore.jks
> ssl.truststore.password=test1234
> ssl.enabled.protocols=TLSv1
> ssl.keystore.location=/opt/kafka/keys/client.keystore.jks
> ssl.keystore.password=test1234
> ssl.key.password=test1234
>
> After that I tried below command. Below 3 command got same error.
>
> bin/kafka-mirror-maker.sh --new.consumer --consumer.config
> config/ssl_consumer.properties --producer.config config/producer.properties
> --num.streams 10 --whitelist=test
>
> bin/kafka-console-producer.sh --broker-list brokerhost:9093 --topic test
> --producer.config config/ssl_client.properties
>
> bin/kafka-console-consumer.sh --bootstrap-server brokerhost:9093 --topic
> test --new-consumer --consumer.config config/ssl_client.properties
>
>
> DEBUG Connection with {broker host}/{ip} disconnected
> (org.apache.kafka.common.network.Selector)
> javax.net.ssl.SSLHandshakeException: General SSLEngine problem
> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431)
> at
> sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
> at
> sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
> at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
> at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:377)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:242)
> at
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:68)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:281)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270)
> at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
> at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
> at
>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1478)
> at
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:212)
> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:919)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:916)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:335)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:413)
> at
>
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:269)
> ... 6 more
> Caused by: sun.security.validator.ValidatorException: PKIX path building
> failed: sun.security.provider.certpath.SunCertPathBuilderException: unable
> to find valid certification path to requested target
> at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
> at
> sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
> at sun.security.validator.Validator.validate(Validator.java:260)
> at
>
> sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
> at
>
> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
> at
>
> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
> at
>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1465)
> ... 15 more
> Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
> unable to find valid certification path to requested target
> at
>
> sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:145)
> at
>
> sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
> at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
> t sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
>
> How can I fix this error?
>
> Thanks.
>