Topic creation fails on Kerberos/SASL enabled brokers

[amir masood khezrain <am...@yahoo.com.INVALID>]

Hello,

I have setup a Kerberos/SASL enabled Kafka cluster with three brokers. Everything is working except an issue I have with the topic creation. When a topic is created with a principal different from the principal of the leader broker, a znode is create under 'kafka/brokers/topics' with the name of the topic. However, the leader broker fails to create the partitions required for the topic as it does not have the right permissions. The error I get in the ‘state-change.log’ is:

 

org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /brokers/topics/my-topic/partitions

        at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68)

        ...

       at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)

Caused by: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /brokers/topics/my-topic/partitions

        at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)

        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)

        ... 23 more

 

The list of principals of brokers are:

- mykafka/my.hostname1.domain.com.AT.krb.realm

- mykafka/my.hostname2.domain.com.AT.krb.realm

- mykafka/my.hostname3.domain.com.AT.krb.realm

 

NOTE: I cannot make the principals of the brokers to be the same. The solutions that I have taken are:

 

1- Use of super.users:

 

I have set the super.users as follow but it was not successful. I might have made a mistake in the 'sasl.kerberos.principal.to.local.rules' setting, but not sure where!

 

super.users=User:mykafka;User:mykafka/my.hostname1.domain.com.AT.krb.realm;User:mykafka/my.hostname2.domain.com.AT.krb.realm;User:mykafka/my.hostname3.domain.com.AT.krb.realm

sasl.kerberos.principal.to.local.rules=RULE:[2:$1](.*)s/.*//,DEFAULT

 

2- Create the znode manually. I tried to create the topic znode manually and set the right permissions before creating the topic through kafka. Since kafka finds such a znode, it assumes the topic is already created!

 

I think the right way of doing this should be using super.users. But I don't know where I am making a mistake that it does not work. Any help is appreciated?