You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Namita Jaokar <ja...@gmail.com> on 2022/10/07 09:26:13 UTC
SSL configuration in Apache Kafka
Hi All,
I am trying to enable SSL in my kafka broker and have generated required
certificates following the steps provided in the official documentation.
I have also made changes in Kafka's server.properties to enable SSL. On
starting zookeeper followed by kafka server startup, There are no error in
the logs in either of them.
However, When I configured my spring-boot producer application and tried
publishing the message, I am getting below error in the producer application
*Producer application error log:*
[Producer clientId=producer-1] Bootstrap broker localhost:9093 (id: -1
rack: null) disconnected
[Producer clientId=producer-1] Connection to node -1 (localhost/
127.0.0.1:9093) could not be established. Broker may not be available.
Simultaneously, The kafka server as well gives below error
*Kafka server log:*
INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication
with /127.0.0.1 (SSL handshake failed)
(org.apache.kafka.common.network.Selector)
I have added below properties in my producer application for SSL
Configuration:
spring.kafka.bootstrap-servers: localhost:9092,localhost:9093
spring.kafka.properties.ssl.endpoint.identification.algorithm:
spring.kafka.properties.ssl.protocol: SSL
spring.kafka.properties.ssl.trust-store-location:
pathtocert/kafka.server.truststore.jks
spring.kafka.properties.ssl.trust-store-password: mypassword
spring.kafka.properties.ssl.key-store-location:
pathtocert/kafka.server.keystore.jks
spring.kafka.properties.ssl.key-store-password: mypassword
spring.kafka.properties.ssl.key-password: mypassword
Below are my server.properties configurations of kafka server to enable SSL.
listeners=PLAINTEXT://:9092,SSL://localhost:9093
advertised.listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
ssl.endpoint.identification.algorithm=
ssl.client.auth=none
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1.3,TLSv1
ssl.keystore.type=jks
ssl.truststore.type=jks
ssl.keystore.location=pathtocert/kafka.server.keystore.jks
ssl.keystore.password=mypassword
ssl.key.password=password
ssl.truststore.location=pathtocert/kafka.server.truststore.jks
ssl.truststore.password=mypassword
security.inter.broker.protocol = SSL
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL
Can someone please help me resolve the issue.
Thanks & Regards,
Namita
Re: SSL configuration in Apache Kafka
Posted by Namita Jaokar <ja...@gmail.com>.
Hi Sunil,
I have tried replacing localhost with IP and have also changed
ssl.client.auth to required from none as I want my server and producer to
communicate via HTTPS.
However, this isn't working for me. When I try sending data to the topic
through ny java producer via an API, I'm getting error in my application
console Http method names must be tokens.
This probably happens when hitting an API which should be tried on http
over https. But in my case if I try running the producer api over http it
again gives me the same issue ase previous. While in Http it states Http
method names must be tokens.
Do you have any suggestions or advise into this? Am I missing some
configuration?
Please correct me in case any misconfigurations.
Thanks in advance.
Best Regards,
Namita
On Fri, 7 Oct, 2022, 17:57 sunil chaudhari, <su...@gmail.com>
wrote:
> You can try two things.
> Instead of localhost, can you publish the kafka service on Hostname?
>
> Since ur client.auth is none, can you try removing keystore from the
> producer?
>
> Regards,
> Sunil.
>
> On Fri, 7 Oct 2022 at 2:56 PM, Namita Jaokar <ja...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > I am trying to enable SSL in my kafka broker and have generated required
> > certificates following the steps provided in the official documentation.
> > I have also made changes in Kafka's server.properties to enable SSL. On
> > starting zookeeper followed by kafka server startup, There are no error
> in
> > the logs in either of them.
> > However, When I configured my spring-boot producer application and tried
> > publishing the message, I am getting below error in the producer
> > application
> >
> > *Producer application error log:*
> >
> > [Producer clientId=producer-1] Bootstrap broker localhost:9093 (id: -1
> > rack: null) disconnected
> >
> > [Producer clientId=producer-1] Connection to node -1 (localhost/
> > 127.0.0.1:9093) could not be established. Broker may not be available.
> >
> >
> >
> > Simultaneously, The kafka server as well gives below error
> >
> >
> > *Kafka server log:*
> >
> > INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed
> authentication
> > with /127.0.0.1 (SSL handshake failed)
> > (org.apache.kafka.common.network.Selector)
> >
> >
> > I have added below properties in my producer application for SSL
> > Configuration:
> >
> > spring.kafka.bootstrap-servers: localhost:9092,localhost:9093
> > spring.kafka.properties.ssl.endpoint.identification.algorithm:
> > spring.kafka.properties.ssl.protocol: SSL
> > spring.kafka.properties.ssl.trust-store-location:
> > pathtocert/kafka.server.truststore.jks
> > spring.kafka.properties.ssl.trust-store-password: mypassword
> > spring.kafka.properties.ssl.key-store-location:
> > pathtocert/kafka.server.keystore.jks
> > spring.kafka.properties.ssl.key-store-password: mypassword
> > spring.kafka.properties.ssl.key-password: mypassword
> >
> > Below are my server.properties configurations of kafka server to enable
> > SSL.
> >
> > listeners=PLAINTEXT://:9092,SSL://localhost:9093
> > advertised.listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
> > ssl.endpoint.identification.algorithm=
> > ssl.client.auth=none
> > ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1.3,TLSv1
> > ssl.keystore.type=jks
> > ssl.truststore.type=jks
> > ssl.keystore.location=pathtocert/kafka.server.keystore.jks
> > ssl.keystore.password=mypassword
> > ssl.key.password=password
> > ssl.truststore.location=pathtocert/kafka.server.truststore.jks
> > ssl.truststore.password=mypassword
> > security.inter.broker.protocol = SSL
> > listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL
> >
> >
> >
> > Can someone please help me resolve the issue.
> >
> >
> > Thanks & Regards,
> >
> > Namita
> >
>
Re: SSL configuration in Apache Kafka
Posted by sunil chaudhari <su...@gmail.com>.
You can try two things.
Instead of localhost, can you publish the kafka service on Hostname?
Since ur client.auth is none, can you try removing keystore from the
producer?
Regards,
Sunil.
On Fri, 7 Oct 2022 at 2:56 PM, Namita Jaokar <ja...@gmail.com>
wrote:
> Hi All,
>
> I am trying to enable SSL in my kafka broker and have generated required
> certificates following the steps provided in the official documentation.
> I have also made changes in Kafka's server.properties to enable SSL. On
> starting zookeeper followed by kafka server startup, There are no error in
> the logs in either of them.
> However, When I configured my spring-boot producer application and tried
> publishing the message, I am getting below error in the producer
> application
>
> *Producer application error log:*
>
> [Producer clientId=producer-1] Bootstrap broker localhost:9093 (id: -1
> rack: null) disconnected
>
> [Producer clientId=producer-1] Connection to node -1 (localhost/
> 127.0.0.1:9093) could not be established. Broker may not be available.
>
>
>
> Simultaneously, The kafka server as well gives below error
>
>
> *Kafka server log:*
>
> INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication
> with /127.0.0.1 (SSL handshake failed)
> (org.apache.kafka.common.network.Selector)
>
>
> I have added below properties in my producer application for SSL
> Configuration:
>
> spring.kafka.bootstrap-servers: localhost:9092,localhost:9093
> spring.kafka.properties.ssl.endpoint.identification.algorithm:
> spring.kafka.properties.ssl.protocol: SSL
> spring.kafka.properties.ssl.trust-store-location:
> pathtocert/kafka.server.truststore.jks
> spring.kafka.properties.ssl.trust-store-password: mypassword
> spring.kafka.properties.ssl.key-store-location:
> pathtocert/kafka.server.keystore.jks
> spring.kafka.properties.ssl.key-store-password: mypassword
> spring.kafka.properties.ssl.key-password: mypassword
>
> Below are my server.properties configurations of kafka server to enable
> SSL.
>
> listeners=PLAINTEXT://:9092,SSL://localhost:9093
> advertised.listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
> ssl.endpoint.identification.algorithm=
> ssl.client.auth=none
> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1.3,TLSv1
> ssl.keystore.type=jks
> ssl.truststore.type=jks
> ssl.keystore.location=pathtocert/kafka.server.keystore.jks
> ssl.keystore.password=mypassword
> ssl.key.password=password
> ssl.truststore.location=pathtocert/kafka.server.truststore.jks
> ssl.truststore.password=mypassword
> security.inter.broker.protocol = SSL
> listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL
>
>
>
> Can someone please help me resolve the issue.
>
>
> Thanks & Regards,
>
> Namita
>