Kafka SSL CA Change Issue

[Naresh R Kumar <Na...@ivycomptech.com>]

Hi Team,

we need some help regarding ca certificate authority change in kafka . Currently we are connecting in kafka using ssl implementation.

kafka version used is 1.1.1

below is server.properties

listeners=INT://$PVT_HOST_NAME:9094,EXT://$PVT_HOST_NAME:9092
advertised.listeners=INT://$PVT_HOST_NAME:9094,EXT://$PUB_HOST_NAME:9092
ssl.keystore.location=$SSL_DIR/broker.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=$SSL_DIR/broker.truststore.jks
SUPER_USERS_CONFIG=$SUPER_USERS_CONFIG"User:CN=br$c.broker.kafka-$CLUSTER_NAME-$ENV,OU=broker,O=server
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,EXT:SSL,INT:PLAINTEXT
inter.broker.listener.name=INT

options tried :-
1. generating new certificates and updating  into existing  keystore and truststore  (we are observing that client is able to connect using one ca is getting accepted old ca or new ca)
2. only private keys in keystore and root certs in truststore (we are observing that client is able to connect using one ca is getting accepted old ca or new ca)
3. muliple keystore and truststore files using comma separated format  (client connect is not working at all)
    eg ..
      ssl.keystore.location=$SSL_DIR/broker-oldca.keystore.jks,$SSL_DIR/broker-newca.keystore.jks
      ssl.truststore.location=$SSL_DIR/broker-oldca.truststore.jks,broker-newca.truststore.jks

can anyone please help us on this, as this change in authority will cause outage and connection issues with existing clients.

Current Result : only one certificate is working , eaither the old one  or new one
Expected Result : both the certificates(keystore & trust store ) should work, old one & new one.

Validation process : After updating the broker certificates we are trying to connect to broker ( from kafka tool) by using consumer certificates.



Thanks & Regards
R . Naresh Kumar

LCG-DF DevOps Engineer​

[cid:354c3de2-833f-4115-ad41-699919639207]

Re: Why need handle delete topic in topic change event

[Colin McCabe <cm...@apache.org>]

On Mon, Oct 11, 2021, at 03:06, 方晓兵 wrote:
> Hi Team, 
>
> I have a problem when I study kafka code in version 2.8.0.
>
> I see `controllerContext.removeTopic(topic)` have been called in 
> `TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not 
> only listen to child delete but also listen to child add. So every time 
> a topic is deleted, an invalid event will be triggered because 
> `controllerContext.removeTopic(topic)` have been called in 
> `TopicDeletionManager.completeDeleteTopic()` after delete this topic 
> zookeeper path. Is it code that needs to be optimized?
>
> Grateful and look forward to answers

Hi,

This code is going away once we remove ZK mode, which is the eventual goal of KIP-500. So it's probably not worth spending a lot of time on optimizing it.

Also, I was under the impression that ZooKeeper only gives you an EventType.NodeChildrenChanged, regardless of whether children were added, removed, or both. So I'm not sure how you would optimize this in the context of the old controller.

best,
Colin

Why need handle delete topic in topic change event

[方晓兵 <94...@gmail.com>]

Hi Team, 

I have a problem when I study kafka code in version 2.8.0.

I see `controllerContext.removeTopic(topic)` have been called in `TopicDeletionManager.completeDeleteTopic()`. TopicChangeHandler not only listen to child delete but also listen to child add. So every time a topic is deleted, an invalid event will be triggered because `controllerContext.removeTopic(topic)` have been called in `TopicDeletionManager.completeDeleteTopic()` after delete this topic zookeeper path. Is it code that needs to be optimized?

Grateful and look forward to answers