You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Awadhesh Gupta <aw...@gmail.com> on 2017/10/03 17:44:18 UTC
Re: SSL setup in Kafka 2.10.0.10.2.1 for keystore and truststore files
Hi,
I validated the client chain in server log after enabling the SSL log and
it was showing entries of both the certificate in chain.
I imported server csr (ca-cert file generated from command penssl req -new
-x509 -keyout ca-key -out ca-cert -days $VALIDITY) to Client trust store
and client csr to Server trust store and then found no error in
Server/Client SSL communication. I could see the publisher can produce the
messages and consumer can consume the messages without any error.
I am not sure if keytool command generated self signed certificates needs
to be imported to both client and server application everytime?
Is this also valid for Verisign or other standard CA generated certificate?
Regarding host name validation, does FQDN with hostname always present in
CN (common name) of the certificate? What if I want to use some free form
text in CSR for CN field to make it for multiple host?
Thanks
Awadhesh
On Fri, Sep 29, 2017 at 5:59 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> This normally means that the truststore in your producer doesn't contain a)
> the public key of your broker or b) the public keys of the CA which signed
> the broker key. With this error it didn't even get to the verification of
> the client certificate yet. Looking at the blog post it looks like there is
> something wrong with your kafka.client.truststore.jks. What you can try is
> to run these two commands and compare the output - whether they talk about
> the same certificates. On on the host where you run the client:
> keytool -list -v -keystore kafka.client.truststore.jks
> And this one on the broker:
> keytool -list -v -keystore kafka.server.keystore.jks
>
> You can also compare the certificates in the SSL debug log. Section
> starting with "adding as trusted cert:" lists what is in your client
> truststore. Section called "*** Certificate chain" shows the certificates
> which are used by the broker.
>
> When using SSL between different hosts you normally should not need
> anything special, since the hostname validation
> (ssl.endpoint.identification.algorithm is AFAIK disabled by default). If
> you enable the hostname verification you will need that the hostname (CN or
> alternative DNS names from the broker key) needs to match the hostname
> which you use to connect to. But this is not your case - the error would be
> different.
>
> Jakub
>
> On Fri, Sep 29, 2017 at 1:05 PM, Awadhesh Gupta <aw...@gmail.com>
> wrote:
>
> > Thanks M Manna.
> >
> > I followed the steps to recreate the keystore & truststore for SSL setup
> on
> > both Client&Server machine and it is working fine if I run the client
> and
> > broker on same Linux host.
> >
> > Problem starts when I publish the messages from Kafka Client deployed on
> > different Linux machine.
> >
> > I enabled SSL log in kafka-run-class.sh to see the handshake traces.
> >
> > I am getting following error in Producer log for Kafka broker
> > certificates - Does client application should have access of Server
> > certificates as well?
> > Exception traces:
> >
> > kafka-producer-network-thread | console-producer, fatal error: 46:
> General
> > SSLEngine problem
> > Caused by: sun.security.validator.ValidatorException: PKIX path building
> > failed: sun.security.provider.certpath.SunCertPathBuilderException:
> unable
> > to find valid certification path to requested target
> >
> > kafka-producer-network-thread | console-producer, SEND TLSv1.2, Alert:
> > fatal, description= certificate_unknown
> >
> > Want to understand if we need to consider any specific configuration for
> > Publisher if it it is sending messages to Kafka broker deployed on
> another
> > host. Please note that I had already created client certificate with
> steps
> > as mentioned in Confluent 101
> > <https://www.confluent.io/blog/apache-kafka-security-
> > authorization-authentication-encryption/>
> > page.
> >
> > I have also imported signed client certificates to JDK provided
> certificate
> > file ($JAVA_HOME/jre\lib/security/cacerts) but no luck.
> >
> > Thanks
> > Awadhesh
> >
> > On Thu, Sep 28, 2017 at 2:02 PM, M. Manna <ma...@gmail.com> wrote:
> >
> > > Hi Awadhesh,
> > >
> > > This seems like your certificate import order (intermediate - root) is
> > > jumbled up. Could you kindly follow the instructions on confluent.io
> > where
> > > Ismael Juma has provided a nice set of steps to follow for SSL setup.
> > >
> > > https://www.confluent.io/blog/apache-kafka-security-
> > > authorization-authentication-encryption/
> > >
> > > Kindest Regards,
> > >
> > > On 28 September 2017 at 09:10, Awadhesh Gupta <
> awadhesh.india@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I am trying to setup Kafka SSL using certificates on my windows
> machine
> > > > using reference of security_overview section of Kafka documents. I
> have
> > > > created server.keystore.jks, client.keystore.jks and respective trust
> > > store
> > > > file and signed it using keytool command. I followed complete steps
> as
> > > > mentioned in "Encryption and Authentication using SSL" section.
> > > >
> > > > I also configured these files is server.properties file and started
> > both
> > > > zookeeper and broker.
> > > >
> > > > Here I configured broker listeners as
> > > >
> > > > listeners=SSL://0.0.0.0:9093
> > > >
> > > >
> > > > When I test the setup of truststore and keystore using below command
> > > >
> > > > opens s_client -debug -connect localhost:9093 -tls1
> > > >
> > > >
> > > > I am getting correct subject and issuer in response but at the same
> > time
> > > I
> > > > am getting below exception in kafka-broker console
> > > >
> > > > javax.net.ssl.SSLHandshakeException: null cert chain
> > > > at sun.security.ssl.Handshaker.checkthrown(Handshaker.java:
> > 1478)
> > > >
> > > > Further, all the message post using Kafka publisher with clients
> > > > certificate ( created with above steps) on port 9093 is rejected by
> > > broker.
> > > >
> > > > Want to understand if some steps are missing to create certificate
> > chain.
> > > >
> > > >
> > > > Thanks in advance
> > > > Awadhesh
> > > >
> > >
> >
>
Re: SSL setup in Kafka 2.10.0.10.2.1 for keystore and truststore files
Posted by Jakub Scholz <ja...@scholz.cz>.
> Regarding host name validation, does FQDN with hostname always present in
CN (common name) of the certificate? What if I want to use some free form
text in CSR for CN field to make it for multiple host?
You have two options. Either you can use wildcard certificates as suggested
by Martin. Or you can add more hostnames into the Subject Alternative
Names, These will be also checked during the hostname verification. Unlike
the wildcard certificate these can be also for completely different
domains. But if you need signed certificate from CA it is up to you to
check with your CA whether they sign it or not.
> I am not sure if keytool command generated self signed certificates needs
to be imported to both client and server application everytime?
> Is this also valid for Verisign or other standard CA generated
certificate?
If you use self-signed certificate, you can verify its identity only using
the public key. So you always have to copy the public keys around and load
them into the counter part truststores. With certificates signed by public
CA such as Verisign you don't need to do this. You just need to make sure
that the application you are using has the Verisign keys whcih don't change
often.
Jakub
On Tue, Oct 3, 2017 at 7:44 PM, Awadhesh Gupta <aw...@gmail.com>
wrote:
> Hi,
>
> I validated the client chain in server log after enabling the SSL log and
> it was showing entries of both the certificate in chain.
>
> I imported server csr (ca-cert file generated from command penssl req
> -new -x509 -keyout ca-key -out ca-cert -days $VALIDITY) to Client trust
> store and client csr to Server trust store and then found no error in
> Server/Client SSL communication. I could see the publisher can produce the
> messages and consumer can consume the messages without any error.
>
> I am not sure if keytool command generated self signed certificates needs
> to be imported to both client and server application everytime?
> Is this also valid for Verisign or other standard CA generated certificate?
>
> Regarding host name validation, does FQDN with hostname always present in
> CN (common name) of the certificate? What if I want to use some free form
> text in CSR for CN field to make it for multiple host?
>
>
> Thanks
> Awadhesh
>
> On Fri, Sep 29, 2017 at 5:59 PM, Jakub Scholz <ja...@scholz.cz> wrote:
>
>> This normally means that the truststore in your producer doesn't contain
>> a)
>> the public key of your broker or b) the public keys of the CA which signed
>> the broker key. With this error it didn't even get to the verification of
>> the client certificate yet. Looking at the blog post it looks like there
>> is
>> something wrong with your kafka.client.truststore.jks. What you can try is
>> to run these two commands and compare the output - whether they talk about
>> the same certificates. On on the host where you run the client:
>> keytool -list -v -keystore kafka.client.truststore.jks
>> And this one on the broker:
>> keytool -list -v -keystore kafka.server.keystore.jks
>>
>> You can also compare the certificates in the SSL debug log. Section
>> starting with "adding as trusted cert:" lists what is in your client
>> truststore. Section called "*** Certificate chain" shows the certificates
>> which are used by the broker.
>>
>> When using SSL between different hosts you normally should not need
>> anything special, since the hostname validation
>> (ssl.endpoint.identification.algorithm is AFAIK disabled by default). If
>> you enable the hostname verification you will need that the hostname (CN
>> or
>> alternative DNS names from the broker key) needs to match the hostname
>> which you use to connect to. But this is not your case - the error would
>> be
>> different.
>>
>> Jakub
>>
>> On Fri, Sep 29, 2017 at 1:05 PM, Awadhesh Gupta <awadhesh.india@gmail.com
>> >
>> wrote:
>>
>> > Thanks M Manna.
>> >
>> > I followed the steps to recreate the keystore & truststore for SSL
>> setup on
>> > both Client&Server machine and it is working fine if I run the client
>> and
>> > broker on same Linux host.
>> >
>> > Problem starts when I publish the messages from Kafka Client deployed on
>> > different Linux machine.
>> >
>> > I enabled SSL log in kafka-run-class.sh to see the handshake traces.
>> >
>> > I am getting following error in Producer log for Kafka broker
>> > certificates - Does client application should have access of Server
>> > certificates as well?
>> > Exception traces:
>> >
>> > kafka-producer-network-thread | console-producer, fatal error: 46:
>> General
>> > SSLEngine problem
>> > Caused by: sun.security.validator.ValidatorException: PKIX path
>> building
>> > failed: sun.security.provider.certpath.SunCertPathBuilderException:
>> unable
>> > to find valid certification path to requested target
>> >
>> > kafka-producer-network-thread | console-producer, SEND TLSv1.2, Alert:
>> > fatal, description= certificate_unknown
>> >
>> > Want to understand if we need to consider any specific configuration for
>> > Publisher if it it is sending messages to Kafka broker deployed on
>> another
>> > host. Please note that I had already created client certificate with
>> steps
>> > as mentioned in Confluent 101
>> > <https://www.confluent.io/blog/apache-kafka-security-
>> > authorization-authentication-encryption/>
>> > page.
>> >
>> > I have also imported signed client certificates to JDK provided
>> certificate
>> > file ($JAVA_HOME/jre\lib/security/cacerts) but no luck.
>> >
>> > Thanks
>> > Awadhesh
>> >
>> > On Thu, Sep 28, 2017 at 2:02 PM, M. Manna <ma...@gmail.com> wrote:
>> >
>> > > Hi Awadhesh,
>> > >
>> > > This seems like your certificate import order (intermediate - root) is
>> > > jumbled up. Could you kindly follow the instructions on confluent.io
>> > where
>> > > Ismael Juma has provided a nice set of steps to follow for SSL setup.
>> > >
>> > > https://www.confluent.io/blog/apache-kafka-security-
>> > > authorization-authentication-encryption/
>> > >
>> > > Kindest Regards,
>> > >
>> > > On 28 September 2017 at 09:10, Awadhesh Gupta <
>> awadhesh.india@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > I am trying to setup Kafka SSL using certificates on my windows
>> machine
>> > > > using reference of security_overview section of Kafka documents. I
>> have
>> > > > created server.keystore.jks, client.keystore.jks and respective
>> trust
>> > > store
>> > > > file and signed it using keytool command. I followed complete steps
>> as
>> > > > mentioned in "Encryption and Authentication using SSL" section.
>> > > >
>> > > > I also configured these files is server.properties file and started
>> > both
>> > > > zookeeper and broker.
>> > > >
>> > > > Here I configured broker listeners as
>> > > >
>> > > > listeners=SSL://0.0.0.0:9093
>> > > >
>> > > >
>> > > > When I test the setup of truststore and keystore using below command
>> > > >
>> > > > opens s_client -debug -connect localhost:9093 -tls1
>> > > >
>> > > >
>> > > > I am getting correct subject and issuer in response but at the same
>> > time
>> > > I
>> > > > am getting below exception in kafka-broker console
>> > > >
>> > > > javax.net.ssl.SSLHandshakeException: null cert chain
>> > > > at sun.security.ssl.Handshaker.checkthrown(Handshaker.java:
>> > 1478)
>> > > >
>> > > > Further, all the message post using Kafka publisher with clients
>> > > > certificate ( created with above steps) on port 9093 is rejected by
>> > > broker.
>> > > >
>> > > > Want to understand if some steps are missing to create certificate
>> > chain.
>> > > >
>> > > >
>> > > > Thanks in advance
>> > > > Awadhesh
>> > > >
>> > >
>> >
>>
>
>
Re: SSL setup in Kafka 2.10.0.10.2.1 for keystore and truststore
files
Posted by Martin Gainty <mg...@hotmail.com>.
________________________________
From: Awadhesh Gupta <aw...@gmail.com>
Sent: Tuesday, October 3, 2017 1:44 PM
To: users@kafka.apache.org; jakub@scholz.cz
Subject: Re: SSL setup in Kafka 2.10.0.10.2.1 for keystore and truststore files
Hi,
I validated the client chain in server log after enabling the SSL log and
it was showing entries of both the certificate in chain.
I imported server csr (ca-cert file generated from command penssl req -new
-x509 -keyout ca-key -out ca-cert -days $VALIDITY) to Client trust store
and client csr to Server trust store and then found no error in
Server/Client SSL communication. I could see the publisher can produce the
messages and consumer can consume the messages without any error.
I am not sure if keytool command generated self signed certificates needs
to be imported to both client and server application everytime?
Is this also valid for Verisign or other standard CA generated certificate?
Regarding host name validation, does FQDN with hostname always present in
CN (common name) of the certificate? What if I want to use some free form
text in CSR for CN field to make it for multiple host?
MG>DigiCert certificate supports multiple subDomains with wildcard for CN
MG>https://www.digicert.com/faq-general.htm#wildcard
MG>remember its your CA Provider that utimately determines which certificate passes validation or not
<https://www.digicert.com/faq-general.htm#wildcard>
SSL Digital Certificate Security :: DigiCert General FAQ<https://www.digicert.com/faq-general.htm#wildcard>
www.digicert.com
Digital SSL Certificate Questions? DigiCert has the answers!
Thanks
Awadhesh
On Fri, Sep 29, 2017 at 5:59 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> This normally means that the truststore in your producer doesn't contain a)
> the public key of your broker or b) the public keys of the CA which signed
> the broker key. With this error it didn't even get to the verification of
> the client certificate yet. Looking at the blog post it looks like there is
> something wrong with your kafka.client.truststore.jks. What you can try is
> to run these two commands and compare the output - whether they talk about
> the same certificates. On on the host where you run the client:
> keytool -list -v -keystore kafka.client.truststore.jks
> And this one on the broker:
> keytool -list -v -keystore kafka.server.keystore.jks
>
> You can also compare the certificates in the SSL debug log. Section
> starting with "adding as trusted cert:" lists what is in your client
> truststore. Section called "*** Certificate chain" shows the certificates
> which are used by the broker.
>
> When using SSL between different hosts you normally should not need
> anything special, since the hostname validation
> (ssl.endpoint.identification.algorithm is AFAIK disabled by default). If
> you enable the hostname verification you will need that the hostname (CN or
> alternative DNS names from the broker key) needs to match the hostname
> which you use to connect to. But this is not your case - the error would be
> different.
>
> Jakub
>
> On Fri, Sep 29, 2017 at 1:05 PM, Awadhesh Gupta <aw...@gmail.com>
> wrote:
>
> > Thanks M Manna.
> >
> > I followed the steps to recreate the keystore & truststore for SSL setup
> on
> > both Client&Server machine and it is working fine if I run the client
> and
> > broker on same Linux host.
> >
> > Problem starts when I publish the messages from Kafka Client deployed on
> > different Linux machine.
> >
> > I enabled SSL log in kafka-run-class.sh to see the handshake traces.
> >
> > I am getting following error in Producer log for Kafka broker
> > certificates - Does client application should have access of Server
> > certificates as well?
> > Exception traces:
> >
> > kafka-producer-network-thread | console-producer, fatal error: 46:
> General
> > SSLEngine problem
> > Caused by: sun.security.validator.ValidatorException: PKIX path building
> > failed: sun.security.provider.certpath.SunCertPathBuilderException:
> unable
> > to find valid certification path to requested target
> >
> > kafka-producer-network-thread | console-producer, SEND TLSv1.2, Alert:
> > fatal, description= certificate_unknown
> >
> > Want to understand if we need to consider any specific configuration for
> > Publisher if it it is sending messages to Kafka broker deployed on
> another
> > host. Please note that I had already created client certificate with
> steps
> > as mentioned in Confluent 101
> > <https://www.confluent.io/blog/apache-kafka-security-
> > authorization-authentication-encryption/>
> > page.
> >
> > I have also imported signed client certificates to JDK provided
> certificate
> > file ($JAVA_HOME/jre\lib/security/cacerts) but no luck.
> >
> > Thanks
> > Awadhesh
> >
> > On Thu, Sep 28, 2017 at 2:02 PM, M. Manna <ma...@gmail.com> wrote:
> >
> > > Hi Awadhesh,
> > >
> > > This seems like your certificate import order (intermediate - root) is
> > > jumbled up. Could you kindly follow the instructions on confluent.io
> > where
> > > Ismael Juma has provided a nice set of steps to follow for SSL setup.
> > >
> > > https://www.confluent.io/blog/apache-kafka-security-
> > > authorization-authentication-encryption/
> > >
> > > Kindest Regards,
> > >
> > > On 28 September 2017 at 09:10, Awadhesh Gupta <
> awadhesh.india@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I am trying to setup Kafka SSL using certificates on my windows
> machine
> > > > using reference of security_overview section of Kafka documents. I
> have
> > > > created server.keystore.jks, client.keystore.jks and respective trust
> > > store
> > > > file and signed it using keytool command. I followed complete steps
> as
> > > > mentioned in "Encryption and Authentication using SSL" section.
> > > >
> > > > I also configured these files is server.properties file and started
> > both
> > > > zookeeper and broker.
> > > >
> > > > Here I configured broker listeners as
> > > >
> > > > listeners=SSL://0.0.0.0:9093
> > > >
> > > >
> > > > When I test the setup of truststore and keystore using below command
> > > >
> > > > opens s_client -debug -connect localhost:9093 -tls1
> > > >
> > > >
> > > > I am getting correct subject and issuer in response but at the same
> > time
> > > I
> > > > am getting below exception in kafka-broker console
> > > >
> > > > javax.net.ssl.SSLHandshakeException: null cert chain
> > > > at sun.security.ssl.Handshaker.checkthrown(Handshaker.java:
> > 1478)
> > > >
> > > > Further, all the message post using Kafka publisher with clients
> > > > certificate ( created with above steps) on port 9093 is rejected by
> > > broker.
> > > >
> > > > Want to understand if some steps are missing to create certificate
> > chain.
> > > >
> > > >
> > > > Thanks in advance
> > > > Awadhesh
> > > >
> > >
> >
>