You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Calvin Chen <pi...@hotmail.com> on 2021/04/30 16:26:57 UTC
Kafka SSL
Hi all
I'm working on Kafka(kafka_2.13-2.7.0)cluster with SSL enabled, and I need help on Kafka broker config(I got error of connection failed) and client SSL config(I got error of SSL handshake failed).
I setup Kafka and client SSL config by taking reference of
Apache Kafka<https://kafka.apache.org/documentation/#security_ssl>
Apache Kafka TLS encryption & authentication - Azure HDInsight | Microsoft Docs<https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-ssl-encryption-authentication>
And I can verify my Kafka cluster SSL with below command:
openssl s_client -debug -connect sc2-kafka-dev-001_node-1:9093 -tls1_2
some output is:
Server certificate
-----BEGIN CERTIFICATE-----
MIID1TCCAb0CFGy5db0MHYKTnZZAQpnHsR3ywrsqMA0GCSqGSIb3DQEBCwUAMBwx
GjAYBgNVBAMMEUthZmthLVNlY3VyaXR5LUNBMB4XDTIxMDQzMDE0NDEzMVoXDTIy
MDQzMDE0NDEzMVowMjEwMC4GA1UEAwwnc2MyLWthZmthLWRldi0wMDFfbm9kZS0x
LmVuZy52bXdhcmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
wuL14qBmI++Ii/lxLU32TlGd0VlDX29JXjyqEUoaXDjYBroY5+FDhawladB3YU3/
IY2fQ9PHoPLVntBnMMf29m8buVFKXsRT0mOjkyVuUUZcp0L9mLMKnKE1Rn+EJM93
Ys0A8/YJgp3LYu0cbLbqw9TUdFkyesaV5zqAXse14npi0eqXk5pk5ss2ePfqa6bN
m2zM1eZrJjjp1vFx0oL8N6z2z6+AS67unyj9x2SjyXQgigbnz36VM99EUeMeQLuz
weuZN97sKKW4ub+ya0R6lbS5pum+iQ4ukA9TeiXllqwoFZTEZistsbec5OvgVgC0
41I6rtlGdqkAPEyU8xtfnwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAdTBndO51t
IK40oYHf2dWHE4WPvZfDoQpAVwhLptsbQD4RVdpPUxagbh4F4zAFwIZgCpwU0YBz
sq71p45x/3NjX40eIWsC0WgQoCQsCWimXQSMOltopNEhrSICd7mD1H/C1uftNXU1
uAGRGUC8wgX1ULdHLg0Szvz519ia+uZqOKyzsMBDZnmtesli3lTmXjjO5E5aPLaU
ztLeZrhHzR7ib9ZtIidl4hviPKbdLBPkeBqk7b821RbCK1Ny8eSOBYY3wePqTGU3
LbLEEeFgNBr9wEsmEcr237QW4UrYX5TjxeoykQj72u9tAb8mTrAY8QXUo9f826hQ
kTcSe504t6hMmX6oP9R3wUHqpIAZ3woqOV/I2KwCt2L3thUXyJK7F9XTSZQq89DT
E4SQlEthR+Mq/eIqyunq403MnQuxRGpfkiOLzBO1vUYDbnWjaC3oouTW9Y1rhF0L
t+DqaMXSTLyhcLZ8xUMcpgfROMArjufTsQ5KWqUYCTUffsrRVFzlyg02OjzgYJ5a
XR/lp64V3Ul1/8EM7QujDgdq9KTRu4FxuOk+8AFMOz4UJ1iqFONBKz6UTYmKjECw
aEp8k8WjuyHeuO5+d9qav+xYSQbHhZ5QSILKlyDSDkLWTjgNyvCMKzabtTW1HfQJ
p4DsCTjGse76yHJNAnH0jdGBVvi8ONdhuA==
-----END CERTIFICATE-----
subject=CN = sc2-kafka-dev-001_node-1.eng.vmware.com
issuer=CN = Kafka-Security-CA
So when I see above output, does it means my SSL setup for Kafka broker is ok?
However, I didn't get below keyword in server.log, as mentioned from Kafka webpage, I should see below in server.log.
with addresses: PLAINTEXT -> EndPoint({{fqdn}},9092,PLAINTEXT),SSL -> EndPoint({{fqdn}},9093,SSL)
My two server.log output are:
[2021-04-30 09:05:08,954] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)
While another one is:
[2021-04-30 09:05:30,183] WARN [Controller id=2, targetBrokerId=1] Connection to node 1 (sc2-kafka-dev-001_node-1.eng.vmware.com/10.185.50.10:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2021-04-30 09:05:30,311] WARN [Controller id=2, targetBrokerId=3] Connection to node 3 (sc2-kafka-dev-001_node-3.eng.vmware.com/10.185.50.12:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
It looks like the Kafka cluster with SSL enabled has some problem on setup connection across brokers. BTW, I haven't apply for the DNS record for my brokers, I setup domain name in /etc/hosts, and it shall be ok for the test?
Also, when I test Kafka command line with SSL config, I see auth error, but I didn't config auth, I just config ssl encryption:
[worker@sc2-kafka-dev-001_node-1 client]$ /opt/kafka/kafka_2.13-2.7.0/bin/kafka-console-producer.sh --broker-list sc2-kafka-dev-001_node-1:9093 --topic topic1 --producer.config ./client-ssl.properties
>[2021-04-30 09:11:19,574] ERROR [Producer clientId=console-producer] Connection to node -1 (sc2-kafka-dev-001_node-1/10.185.50.10:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2021-04-30 09:11:19,575] WARN [Producer clientId=console-producer] Bootstrap broker sc2-kafka-dev-001_node-1:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
Here is my part of Kafka broker config:
listeners=PLAINTEXT://sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL://sc2-kafka-dev-001_node-2.eng.vmware.com:9093
advertised.listeners=PLAINTEXT://sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL://sc2-kafka-dev-001_node-2.eng.vmware.com:9093
ssl.endpoint.identification.algorithm=
security.inter.broker.protocol=SSL
ssl.keystore.location=/data/ssl1/kafka.server.keystore.jks
ssl.keystore.password=MyServerPassword123
ssl.key.password=MyServerPassword123
ssl.truststore.location=/data/ssl1/kafka.server.truststore.jks
ssl.truststore.password=MyServerPassword123
ssl.enabled.protocols=TLSv1.2
ssl.truststore.type=JKS
ssl.keystore.type=JKS
ssl.secure.random.implementation=SHA1PRNG
Here is my client config:
security.protocol=SSL
ssl.truststore.location=/data/client/kafka.client.truststore.jks
ssl.truststore.password=MyClientPassword123
ssl.enabled.protocols=TLSv1.2
THANKS
Re: Kafka SSL
Posted by Ran Lupovich <ra...@gmail.com>.
https://docs.confluent.io/platform/current/kafka/authentication_ssl.html
Check this out
בתאריך יום ו׳, 30 באפר׳ 2021, 20:06, מאת Ran Lupovich <
ranlupovich@gmail.com>:
> Hi seems you setup in port 9093 only ssl as a method of authentication and
> method of transfer encryption, so it means in the client configuration you
> would need the keystore configured as well, you could choose other mean of
> authentication such as PLAIN_SSL or so own, hope thats helps, keep us
> updated, good luck
>
> בתאריך יום ו׳, 30 באפר׳ 2021, 19:27, מאת Calvin Chen <
> pingc.sh@hotmail.com>:
>
>> Hi all
>>
>> I'm working on Kafka(kafka_2.13-2.7.0)cluster with SSL enabled, and I
>> need help on Kafka broker config(I got error of connection failed) and
>> client SSL config(I got error of SSL handshake failed).
>>
>>
>> I setup Kafka and client SSL config by taking reference of
>> Apache Kafka<https://kafka.apache.org/documentation/#security_ssl>
>> Apache Kafka TLS encryption & authentication - Azure HDInsight |
>> Microsoft Docs<
>> https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-ssl-encryption-authentication
>> >
>>
>> And I can verify my Kafka cluster SSL with below command:
>>
>> openssl s_client -debug -connect sc2-kafka-dev-001_node-1:9093 -tls1_2
>>
>> some output is:
>>
>> Server certificate
>> -----BEGIN CERTIFICATE-----
>> MIID1TCCAb0CFGy5db0MHYKTnZZAQpnHsR3ywrsqMA0GCSqGSIb3DQEBCwUAMBwx
>> GjAYBgNVBAMMEUthZmthLVNlY3VyaXR5LUNBMB4XDTIxMDQzMDE0NDEzMVoXDTIy
>> MDQzMDE0NDEzMVowMjEwMC4GA1UEAwwnc2MyLWthZmthLWRldi0wMDFfbm9kZS0x
>> LmVuZy52bXdhcmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
>> wuL14qBmI++Ii/lxLU32TlGd0VlDX29JXjyqEUoaXDjYBroY5+FDhawladB3YU3/
>> IY2fQ9PHoPLVntBnMMf29m8buVFKXsRT0mOjkyVuUUZcp0L9mLMKnKE1Rn+EJM93
>> Ys0A8/YJgp3LYu0cbLbqw9TUdFkyesaV5zqAXse14npi0eqXk5pk5ss2ePfqa6bN
>> m2zM1eZrJjjp1vFx0oL8N6z2z6+AS67unyj9x2SjyXQgigbnz36VM99EUeMeQLuz
>> weuZN97sKKW4ub+ya0R6lbS5pum+iQ4ukA9TeiXllqwoFZTEZistsbec5OvgVgC0
>> 41I6rtlGdqkAPEyU8xtfnwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAdTBndO51t
>> IK40oYHf2dWHE4WPvZfDoQpAVwhLptsbQD4RVdpPUxagbh4F4zAFwIZgCpwU0YBz
>> sq71p45x/3NjX40eIWsC0WgQoCQsCWimXQSMOltopNEhrSICd7mD1H/C1uftNXU1
>> uAGRGUC8wgX1ULdHLg0Szvz519ia+uZqOKyzsMBDZnmtesli3lTmXjjO5E5aPLaU
>> ztLeZrhHzR7ib9ZtIidl4hviPKbdLBPkeBqk7b821RbCK1Ny8eSOBYY3wePqTGU3
>> LbLEEeFgNBr9wEsmEcr237QW4UrYX5TjxeoykQj72u9tAb8mTrAY8QXUo9f826hQ
>> kTcSe504t6hMmX6oP9R3wUHqpIAZ3woqOV/I2KwCt2L3thUXyJK7F9XTSZQq89DT
>> E4SQlEthR+Mq/eIqyunq403MnQuxRGpfkiOLzBO1vUYDbnWjaC3oouTW9Y1rhF0L
>> t+DqaMXSTLyhcLZ8xUMcpgfROMArjufTsQ5KWqUYCTUffsrRVFzlyg02OjzgYJ5a
>> XR/lp64V3Ul1/8EM7QujDgdq9KTRu4FxuOk+8AFMOz4UJ1iqFONBKz6UTYmKjECw
>> aEp8k8WjuyHeuO5+d9qav+xYSQbHhZ5QSILKlyDSDkLWTjgNyvCMKzabtTW1HfQJ
>> p4DsCTjGse76yHJNAnH0jdGBVvi8ONdhuA==
>> -----END CERTIFICATE-----
>> subject=CN = sc2-kafka-dev-001_node-1.eng.vmware.com
>>
>> issuer=CN = Kafka-Security-CA
>>
>>
>> So when I see above output, does it means my SSL setup for Kafka broker
>> is ok?
>>
>>
>> However, I didn't get below keyword in server.log, as mentioned from
>> Kafka webpage, I should see below in server.log.
>>
>>
>> with addresses: PLAINTEXT -> EndPoint({{fqdn}},9092,PLAINTEXT),SSL ->
>> EndPoint({{fqdn}},9093,SSL)
>>
>> My two server.log output are:
>>
>> [2021-04-30 09:05:08,954] INFO [KafkaServer id=1] started
>> (kafka.server.KafkaServer)
>>
>> While another one is:
>>
>> [2021-04-30 09:05:30,183] WARN [Controller id=2, targetBrokerId=1]
>> Connection to node 1 (
>> sc2-kafka-dev-001_node-1.eng.vmware.com/10.185.50.10:9093) could not be
>> established. Broker may not be available.
>> (org.apache.kafka.clients.NetworkClient)
>> [2021-04-30 09:05:30,311] WARN [Controller id=2, targetBrokerId=3]
>> Connection to node 3 (
>> sc2-kafka-dev-001_node-3.eng.vmware.com/10.185.50.12:9093) could not be
>> established. Broker may not be available.
>> (org.apache.kafka.clients.NetworkClient)
>>
>> It looks like the Kafka cluster with SSL enabled has some problem on
>> setup connection across brokers. BTW, I haven't apply for the DNS record
>> for my brokers, I setup domain name in /etc/hosts, and it shall be ok for
>> the test?
>>
>>
>> Also, when I test Kafka command line with SSL config, I see auth error,
>> but I didn't config auth, I just config ssl encryption:
>>
>> [worker@sc2-kafka-dev-001_node-1 client]$
>> /opt/kafka/kafka_2.13-2.7.0/bin/kafka-console-producer.sh --broker-list
>> sc2-kafka-dev-001_node-1:9093 --topic topic1 --producer.config
>> ./client-ssl.properties
>> >[2021-04-30 09:11:19,574] ERROR [Producer clientId=console-producer]
>> Connection to node -1 (sc2-kafka-dev-001_node-1/10.185.50.10:9093)
>> failed authentication due to: SSL handshake failed
>> (org.apache.kafka.clients.NetworkClient)
>> [2021-04-30 09:11:19,575] WARN [Producer clientId=console-producer]
>> Bootstrap broker sc2-kafka-dev-001_node-1:9093 (id: -1 rack: null)
>> disconnected (org.apache.kafka.clients.NetworkClient)
>>
>>
>> Here is my part of Kafka broker config:
>>
>> listeners=PLAINTEXT://sc2-kafka-dev-001_node-2.eng.vmware.com:9092,
>> SSL://sc2-kafka-dev-001_node-2.eng.vmware.com:9093
>> advertised.listeners=PLAINTEXT://
>> sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL://
>> sc2-kafka-dev-001_node-2.eng.vmware.com:9093
>>
>> ssl.endpoint.identification.algorithm=
>> security.inter.broker.protocol=SSL
>>
>> ssl.keystore.location=/data/ssl1/kafka.server.keystore.jks
>> ssl.keystore.password=MyServerPassword123
>> ssl.key.password=MyServerPassword123
>> ssl.truststore.location=/data/ssl1/kafka.server.truststore.jks
>> ssl.truststore.password=MyServerPassword123
>> ssl.enabled.protocols=TLSv1.2
>> ssl.truststore.type=JKS
>> ssl.keystore.type=JKS
>> ssl.secure.random.implementation=SHA1PRNG
>>
>>
>> Here is my client config:
>>
>> security.protocol=SSL
>> ssl.truststore.location=/data/client/kafka.client.truststore.jks
>> ssl.truststore.password=MyClientPassword123
>> ssl.enabled.protocols=TLSv1.2
>>
>>
>>
>> THANKS
>>
>
Re: Kafka SSL
Posted by Ran Lupovich <ra...@gmail.com>.
Hi seems you setup in port 9093 only ssl as a method of authentication and
method of transfer encryption, so it means in the client configuration you
would need the keystore configured as well, you could choose other mean of
authentication such as PLAIN_SSL or so own, hope thats helps, keep us
updated, good luck
בתאריך יום ו׳, 30 באפר׳ 2021, 19:27, מאת Calvin Chen <pingc.sh@hotmail.com
>:
> Hi all
>
> I'm working on Kafka(kafka_2.13-2.7.0)cluster with SSL enabled, and I need
> help on Kafka broker config(I got error of connection failed) and client
> SSL config(I got error of SSL handshake failed).
>
>
> I setup Kafka and client SSL config by taking reference of
> Apache Kafka<https://kafka.apache.org/documentation/#security_ssl>
> Apache Kafka TLS encryption & authentication - Azure HDInsight | Microsoft
> Docs<
> https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-ssl-encryption-authentication
> >
>
> And I can verify my Kafka cluster SSL with below command:
>
> openssl s_client -debug -connect sc2-kafka-dev-001_node-1:9093 -tls1_2
>
> some output is:
>
> Server certificate
> -----BEGIN CERTIFICATE-----
> MIID1TCCAb0CFGy5db0MHYKTnZZAQpnHsR3ywrsqMA0GCSqGSIb3DQEBCwUAMBwx
> GjAYBgNVBAMMEUthZmthLVNlY3VyaXR5LUNBMB4XDTIxMDQzMDE0NDEzMVoXDTIy
> MDQzMDE0NDEzMVowMjEwMC4GA1UEAwwnc2MyLWthZmthLWRldi0wMDFfbm9kZS0x
> LmVuZy52bXdhcmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
> wuL14qBmI++Ii/lxLU32TlGd0VlDX29JXjyqEUoaXDjYBroY5+FDhawladB3YU3/
> IY2fQ9PHoPLVntBnMMf29m8buVFKXsRT0mOjkyVuUUZcp0L9mLMKnKE1Rn+EJM93
> Ys0A8/YJgp3LYu0cbLbqw9TUdFkyesaV5zqAXse14npi0eqXk5pk5ss2ePfqa6bN
> m2zM1eZrJjjp1vFx0oL8N6z2z6+AS67unyj9x2SjyXQgigbnz36VM99EUeMeQLuz
> weuZN97sKKW4ub+ya0R6lbS5pum+iQ4ukA9TeiXllqwoFZTEZistsbec5OvgVgC0
> 41I6rtlGdqkAPEyU8xtfnwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAdTBndO51t
> IK40oYHf2dWHE4WPvZfDoQpAVwhLptsbQD4RVdpPUxagbh4F4zAFwIZgCpwU0YBz
> sq71p45x/3NjX40eIWsC0WgQoCQsCWimXQSMOltopNEhrSICd7mD1H/C1uftNXU1
> uAGRGUC8wgX1ULdHLg0Szvz519ia+uZqOKyzsMBDZnmtesli3lTmXjjO5E5aPLaU
> ztLeZrhHzR7ib9ZtIidl4hviPKbdLBPkeBqk7b821RbCK1Ny8eSOBYY3wePqTGU3
> LbLEEeFgNBr9wEsmEcr237QW4UrYX5TjxeoykQj72u9tAb8mTrAY8QXUo9f826hQ
> kTcSe504t6hMmX6oP9R3wUHqpIAZ3woqOV/I2KwCt2L3thUXyJK7F9XTSZQq89DT
> E4SQlEthR+Mq/eIqyunq403MnQuxRGpfkiOLzBO1vUYDbnWjaC3oouTW9Y1rhF0L
> t+DqaMXSTLyhcLZ8xUMcpgfROMArjufTsQ5KWqUYCTUffsrRVFzlyg02OjzgYJ5a
> XR/lp64V3Ul1/8EM7QujDgdq9KTRu4FxuOk+8AFMOz4UJ1iqFONBKz6UTYmKjECw
> aEp8k8WjuyHeuO5+d9qav+xYSQbHhZ5QSILKlyDSDkLWTjgNyvCMKzabtTW1HfQJ
> p4DsCTjGse76yHJNAnH0jdGBVvi8ONdhuA==
> -----END CERTIFICATE-----
> subject=CN = sc2-kafka-dev-001_node-1.eng.vmware.com
>
> issuer=CN = Kafka-Security-CA
>
>
> So when I see above output, does it means my SSL setup for Kafka broker is
> ok?
>
>
> However, I didn't get below keyword in server.log, as mentioned from Kafka
> webpage, I should see below in server.log.
>
>
> with addresses: PLAINTEXT -> EndPoint({{fqdn}},9092,PLAINTEXT),SSL ->
> EndPoint({{fqdn}},9093,SSL)
>
> My two server.log output are:
>
> [2021-04-30 09:05:08,954] INFO [KafkaServer id=1] started
> (kafka.server.KafkaServer)
>
> While another one is:
>
> [2021-04-30 09:05:30,183] WARN [Controller id=2, targetBrokerId=1]
> Connection to node 1 (
> sc2-kafka-dev-001_node-1.eng.vmware.com/10.185.50.10:9093) could not be
> established. Broker may not be available.
> (org.apache.kafka.clients.NetworkClient)
> [2021-04-30 09:05:30,311] WARN [Controller id=2, targetBrokerId=3]
> Connection to node 3 (
> sc2-kafka-dev-001_node-3.eng.vmware.com/10.185.50.12:9093) could not be
> established. Broker may not be available.
> (org.apache.kafka.clients.NetworkClient)
>
> It looks like the Kafka cluster with SSL enabled has some problem on setup
> connection across brokers. BTW, I haven't apply for the DNS record for my
> brokers, I setup domain name in /etc/hosts, and it shall be ok for the test?
>
>
> Also, when I test Kafka command line with SSL config, I see auth error,
> but I didn't config auth, I just config ssl encryption:
>
> [worker@sc2-kafka-dev-001_node-1 client]$
> /opt/kafka/kafka_2.13-2.7.0/bin/kafka-console-producer.sh --broker-list
> sc2-kafka-dev-001_node-1:9093 --topic topic1 --producer.config
> ./client-ssl.properties
> >[2021-04-30 09:11:19,574] ERROR [Producer clientId=console-producer]
> Connection to node -1 (sc2-kafka-dev-001_node-1/10.185.50.10:9093) failed
> authentication due to: SSL handshake failed
> (org.apache.kafka.clients.NetworkClient)
> [2021-04-30 09:11:19,575] WARN [Producer clientId=console-producer]
> Bootstrap broker sc2-kafka-dev-001_node-1:9093 (id: -1 rack: null)
> disconnected (org.apache.kafka.clients.NetworkClient)
>
>
> Here is my part of Kafka broker config:
>
> listeners=PLAINTEXT://sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL://
> sc2-kafka-dev-001_node-2.eng.vmware.com:9093
> advertised.listeners=PLAINTEXT://
> sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL://
> sc2-kafka-dev-001_node-2.eng.vmware.com:9093
>
> ssl.endpoint.identification.algorithm=
> security.inter.broker.protocol=SSL
>
> ssl.keystore.location=/data/ssl1/kafka.server.keystore.jks
> ssl.keystore.password=MyServerPassword123
> ssl.key.password=MyServerPassword123
> ssl.truststore.location=/data/ssl1/kafka.server.truststore.jks
> ssl.truststore.password=MyServerPassword123
> ssl.enabled.protocols=TLSv1.2
> ssl.truststore.type=JKS
> ssl.keystore.type=JKS
> ssl.secure.random.implementation=SHA1PRNG
>
>
> Here is my client config:
>
> security.protocol=SSL
> ssl.truststore.location=/data/client/kafka.client.truststore.jks
> ssl.truststore.password=MyClientPassword123
> ssl.enabled.protocols=TLSv1.2
>
>
>
> THANKS
>