You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Dhirendra Singh <dh...@gmail.com> on 2021/06/04 04:40:17 UTC
kafka 2 way ssl authentication
I am trying to setup 2 way ssl authentication. My requirement is broker
should authenticate only specific clients.
My organization has a CA which issue all certificates in pkcs12 format.
steps i followed are as follows.
1. get a certificate for the broker and configured it in the broker keystore
ssl.keystore.location=/home/kafka/certificate.p12
ssl.keystore.password=xxxxx
ssl.client.auth=required
2. get a certificate for the client and configured it in the client keystore
ssl.keystore.location=/home/kafka/certificate.p12
ssl.keystore.password=xxxxx
3. extracted the public certificate from the client certificate using
keytool command
keytool -export -file cert -keystore certificate.p12 -alias "12345"
-storetype pkcs12 -storepass xxxxx
4. imported the certificate into broker truststore. broker truststore
contains only the client 12345 certificate.
keytool -keystore truststore.p12 -import -file cert -alias 12345
-storetype pkcs12 -storepass xxxxx -noprompt
5. configured the truststore in the broker.
ssl.truststore.location=/home/kafka/truststore.p12
ssl.truststore.password=xxxxx
6. configured the truststore in client. client truststore contains CA
certificates.
ssl.truststore.location=/etc/pki/java/cacerts
ssl.truststore.password=xxxxx
When i run the broker and client i expect the broker to authenticate the
client and establish ssl connection. but instead following error is thrown.
[2021-06-03 23:32:06,864] ERROR [AdminClient clientId=adminclient-1]
Connection to node -1 (abc.com/10.129.140.212:9093) failed authentication
due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2021-06-03 23:32:06,866] WARN [AdminClient clientId=adminclient-1]
Metadata update failed due to authentication error
(org.apache.kafka.clients.admin.internals.AdminMetadataManager)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
failed
Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
message: server_hello
I tried various things but nothing seems to work. when i replace the broker
truststore with /etc/pki/java/cacerts truststore file which contains only
the CA certificate
then it works fine. but it will authenticate any client which has
certificate issued by the CA.
what could be the issue ?
Re: kafka 2 way ssl authentication
Posted by Ran Lupovich <ra...@gmail.com>.
Share your new configs and logs
בתאריך יום ו׳, 4 ביוני 2021, 12:06, מאת Dhirendra Singh <
dhirendraks@gmail.com>:
> I tried the keytool command suggested by you. still getting the same error.
>
> On Fri, Jun 4, 2021 at 10:50 AM Ran Lupovich <ra...@gmail.com>
> wrote:
>
> > The default format is jks,
> >
> >
> > use keytool to create a Java KeyStore (JKS) with the certificate and key
> > for use by Kafka. You'll be prompted to create a new password for the
> > resulting file as well as enter the password for the PKCS12 file from the
> > previous step. Hang onto the new JKS password for use in configuration
> > below.
> >
> > $ keytool -importkeystore -srckeystore server.p12 -destkeystore
> > kafka.server.keystore.jks -srcstoretype pkcs12 -alias
> > myserver.internal.net
> >
> > Note: It's safe to ignore the following warning from keytool.
> >
> > The JKS keystore uses a proprietary format. It is recommended to
> > migrate to PKCS12 which is an industry standard format using "keytool
> > -importkeystore -srckeystore server.p12 -destkeystore
> > kafka.server.keystore.jks -srcstoretype pkcs12"
> >
> >
> > בתאריך יום ו׳, 4 ביוני 2021, 07:40, מאת Dhirendra Singh <
> > dhirendraks@gmail.com>:
> >
> > > I am trying to setup 2 way ssl authentication. My requirement is broker
> > > should authenticate only specific clients.
> > > My organization has a CA which issue all certificates in pkcs12 format.
> > > steps i followed are as follows.
> > >
> > > 1. get a certificate for the broker and configured it in the broker
> > > keystore
> > > ssl.keystore.location=/home/kafka/certificate.p12
> > > ssl.keystore.password=xxxxx
> > > ssl.client.auth=required
> > > 2. get a certificate for the client and configured it in the client
> > > keystore
> > > ssl.keystore.location=/home/kafka/certificate.p12
> > > ssl.keystore.password=xxxxx
> > > 3. extracted the public certificate from the client certificate using
> > > keytool command
> > > keytool -export -file cert -keystore certificate.p12 -alias "12345"
> > > -storetype pkcs12 -storepass xxxxx
> > > 4. imported the certificate into broker truststore. broker truststore
> > > contains only the client 12345 certificate.
> > > keytool -keystore truststore.p12 -import -file cert -alias 12345
> > > -storetype pkcs12 -storepass xxxxx -noprompt
> > > 5. configured the truststore in the broker.
> > > ssl.truststore.location=/home/kafka/truststore.p12
> > > ssl.truststore.password=xxxxx
> > > 6. configured the truststore in client. client truststore contains CA
> > > certificates.
> > > ssl.truststore.location=/etc/pki/java/cacerts
> > > ssl.truststore.password=xxxxx
> > >
> > > When i run the broker and client i expect the broker to authenticate
> the
> > > client and establish ssl connection. but instead following error is
> > thrown.
> > > [2021-06-03 23:32:06,864] ERROR [AdminClient clientId=adminclient-1]
> > > Connection to node -1 (abc.com/10.129.140.212:9093) failed
> > authentication
> > > due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
> > > [2021-06-03 23:32:06,866] WARN [AdminClient clientId=adminclient-1]
> > > Metadata update failed due to authentication error
> > > (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
> > > org.apache.kafka.common.errors.SslAuthenticationException: SSL
> handshake
> > > failed
> > > Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
> > > message: server_hello
> > >
> > > I tried various things but nothing seems to work. when i replace the
> > broker
> > > truststore with /etc/pki/java/cacerts truststore file which contains
> only
> > > the CA certificate
> > > then it works fine. but it will authenticate any client which has
> > > certificate issued by the CA.
> > >
> > > what could be the issue ?
> > >
> >
>
Re: kafka 2 way ssl authentication
Posted by Dhirendra Singh <dh...@gmail.com>.
I tried the keytool command suggested by you. still getting the same error.
On Fri, Jun 4, 2021 at 10:50 AM Ran Lupovich <ra...@gmail.com> wrote:
> The default format is jks,
>
>
> use keytool to create a Java KeyStore (JKS) with the certificate and key
> for use by Kafka. You'll be prompted to create a new password for the
> resulting file as well as enter the password for the PKCS12 file from the
> previous step. Hang onto the new JKS password for use in configuration
> below.
>
> $ keytool -importkeystore -srckeystore server.p12 -destkeystore
> kafka.server.keystore.jks -srcstoretype pkcs12 -alias
> myserver.internal.net
>
> Note: It's safe to ignore the following warning from keytool.
>
> The JKS keystore uses a proprietary format. It is recommended to
> migrate to PKCS12 which is an industry standard format using "keytool
> -importkeystore -srckeystore server.p12 -destkeystore
> kafka.server.keystore.jks -srcstoretype pkcs12"
>
>
> בתאריך יום ו׳, 4 ביוני 2021, 07:40, מאת Dhirendra Singh <
> dhirendraks@gmail.com>:
>
> > I am trying to setup 2 way ssl authentication. My requirement is broker
> > should authenticate only specific clients.
> > My organization has a CA which issue all certificates in pkcs12 format.
> > steps i followed are as follows.
> >
> > 1. get a certificate for the broker and configured it in the broker
> > keystore
> > ssl.keystore.location=/home/kafka/certificate.p12
> > ssl.keystore.password=xxxxx
> > ssl.client.auth=required
> > 2. get a certificate for the client and configured it in the client
> > keystore
> > ssl.keystore.location=/home/kafka/certificate.p12
> > ssl.keystore.password=xxxxx
> > 3. extracted the public certificate from the client certificate using
> > keytool command
> > keytool -export -file cert -keystore certificate.p12 -alias "12345"
> > -storetype pkcs12 -storepass xxxxx
> > 4. imported the certificate into broker truststore. broker truststore
> > contains only the client 12345 certificate.
> > keytool -keystore truststore.p12 -import -file cert -alias 12345
> > -storetype pkcs12 -storepass xxxxx -noprompt
> > 5. configured the truststore in the broker.
> > ssl.truststore.location=/home/kafka/truststore.p12
> > ssl.truststore.password=xxxxx
> > 6. configured the truststore in client. client truststore contains CA
> > certificates.
> > ssl.truststore.location=/etc/pki/java/cacerts
> > ssl.truststore.password=xxxxx
> >
> > When i run the broker and client i expect the broker to authenticate the
> > client and establish ssl connection. but instead following error is
> thrown.
> > [2021-06-03 23:32:06,864] ERROR [AdminClient clientId=adminclient-1]
> > Connection to node -1 (abc.com/10.129.140.212:9093) failed
> authentication
> > due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
> > [2021-06-03 23:32:06,866] WARN [AdminClient clientId=adminclient-1]
> > Metadata update failed due to authentication error
> > (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
> > org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> > failed
> > Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
> > message: server_hello
> >
> > I tried various things but nothing seems to work. when i replace the
> broker
> > truststore with /etc/pki/java/cacerts truststore file which contains only
> > the CA certificate
> > then it works fine. but it will authenticate any client which has
> > certificate issued by the CA.
> >
> > what could be the issue ?
> >
>
Re: kafka 2 way ssl authentication
Posted by Ran Lupovich <ra...@gmail.com>.
The default format is jks,
use keytool to create a Java KeyStore (JKS) with the certificate and key
for use by Kafka. You'll be prompted to create a new password for the
resulting file as well as enter the password for the PKCS12 file from the
previous step. Hang onto the new JKS password for use in configuration
below.
$ keytool -importkeystore -srckeystore server.p12 -destkeystore
kafka.server.keystore.jks -srcstoretype pkcs12 -alias
myserver.internal.net
Note: It's safe to ignore the following warning from keytool.
The JKS keystore uses a proprietary format. It is recommended to
migrate to PKCS12 which is an industry standard format using "keytool
-importkeystore -srckeystore server.p12 -destkeystore
kafka.server.keystore.jks -srcstoretype pkcs12"
בתאריך יום ו׳, 4 ביוני 2021, 07:40, מאת Dhirendra Singh <
dhirendraks@gmail.com>:
> I am trying to setup 2 way ssl authentication. My requirement is broker
> should authenticate only specific clients.
> My organization has a CA which issue all certificates in pkcs12 format.
> steps i followed are as follows.
>
> 1. get a certificate for the broker and configured it in the broker
> keystore
> ssl.keystore.location=/home/kafka/certificate.p12
> ssl.keystore.password=xxxxx
> ssl.client.auth=required
> 2. get a certificate for the client and configured it in the client
> keystore
> ssl.keystore.location=/home/kafka/certificate.p12
> ssl.keystore.password=xxxxx
> 3. extracted the public certificate from the client certificate using
> keytool command
> keytool -export -file cert -keystore certificate.p12 -alias "12345"
> -storetype pkcs12 -storepass xxxxx
> 4. imported the certificate into broker truststore. broker truststore
> contains only the client 12345 certificate.
> keytool -keystore truststore.p12 -import -file cert -alias 12345
> -storetype pkcs12 -storepass xxxxx -noprompt
> 5. configured the truststore in the broker.
> ssl.truststore.location=/home/kafka/truststore.p12
> ssl.truststore.password=xxxxx
> 6. configured the truststore in client. client truststore contains CA
> certificates.
> ssl.truststore.location=/etc/pki/java/cacerts
> ssl.truststore.password=xxxxx
>
> When i run the broker and client i expect the broker to authenticate the
> client and establish ssl connection. but instead following error is thrown.
> [2021-06-03 23:32:06,864] ERROR [AdminClient clientId=adminclient-1]
> Connection to node -1 (abc.com/10.129.140.212:9093) failed authentication
> due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
> [2021-06-03 23:32:06,866] WARN [AdminClient clientId=adminclient-1]
> Metadata update failed due to authentication error
> (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> failed
> Caused by: javax.net.ssl.SSLProtocolException: Unexpected handshake
> message: server_hello
>
> I tried various things but nothing seems to work. when i replace the broker
> truststore with /etc/pki/java/cacerts truststore file which contains only
> the CA certificate
> then it works fine. but it will authenticate any client which has
> certificate issued by the CA.
>
> what could be the issue ?
>