You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Raghu B <ra...@gmail.com> on 2016/12/14 23:42:25 UTC
Kafka ACL's with SSL Protocol is not working
Hi All,
I am trying to enable ACL's in my Kafka cluster with along with SSL
Protocol.
I tried with each and every parameters but no luck, so I need help to
enable the SSL(without Kerberos) and I am attaching all the configuration
details in this.
Kindly Help me.
*I tested SSL without ACL, it worked fine
(listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
*This is my Kafka server properties file:*
*############################# ACL SETTINGS #############################*
*auto.create.topics.enable=true*
*authorizer.class.name
<http://authorizer.class.name>=kafka.security.auth.SimpleAclAuthorizer*
*security.inter.broker.protocol=SSL*
*#allow.everyone.if.no.acl.found=true*
*#principal.builder.class=CustomizedPrincipalBuilderClass*
*#super.users=User:"CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
*#super.users=User:Raghu;User:Admin*
*#offsets.storage=kafka*
*#dual.commit.enabled=true*
*listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
*#listeners=PLAINTEXT://10.247.195.122:9092 <http://10.247.195.122:9092>*
*#listeners=PLAINTEXT://10.247.195.122:9092
<http://10.247.195.122:9092>,SSL://10.247.195.122:9093
<http://10.247.195.122:9093>*
*#advertised.listeners=PLAINTEXT://10.247.195.122:9092
<http://10.247.195.122:9092>*
*
ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
* ssl.keystore.password=123456*
* ssl.key.password=123456*
*
ssl.truststore.location=/home/raghu/kafka/security/server.truststore.jks*
* ssl.truststore.password=123456*
*Set the ACL from Authorizer CLI:*
> *bin/kafka-acls.sh --authorizer-properties
zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181> --list
--topic ssltopic*
*Current ACLs for resource `Topic:ssltopic`: *
* User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
C=Unknown has Allow permission for operations: Write from hosts: * *
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh
--broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic
ssltopic --producer.config client-ssl.properties*
*[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
*#group.id <http://group.id>=sslgroup*
*security.protocol=SSL*
*ssl.truststore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
*ssl.truststore.password=123456*
* #Configure Below if you use Client Auth*
*ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.keystore.jks*
*ssl.keystore.password=123456*
*ssl.key.password=123456*
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh
--bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
--new-consumer --consumer.config client-ssl.properties --topic ssltopic
--from-beginning*
*[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
(kafka.tools.ConsoleConsumer$)*
*org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized
to access group: console-consumer-52826*
Thanks in advance,
Raghu - raghu98499@gmail.com
Re: Kafka ACL's with SSL Protocol is not working
Posted by Raghav <ra...@gmail.com>.
Hi
I have very similar problem that Raghu faced.
How can I enable the log4j debug mode ?
Thanks.
On Thu, Dec 15, 2016 at 3:32 PM, Raghu B <ra...@gmail.com> wrote:
> Thanks Derar & Kiran, your suggestions are very useful.
>
> I enabled Log4J debug mode and found that my client is trying to connect to
> the Kafka server with the *User:ANONYMOUS, *It is really strange.
>
>
> I added a new Super.User with the name *User:ANONYMOUS *then I am able to
> send and receive the messages without any issues.
>
> And now the question is how can I set my username name from Anonymous to
> something like
> *User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> which
> comes from SSL cert/keystore.
>
> Please help me with your inputs.
>
> Thanks in Advance,
> Raghu
>
> On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com>
> wrote:
>
> > I have just noticed that I am using the user which is not configured in
> the
> > kafka server jaas config file..
> >
> >
> >
> > On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> > wrote:
> >
> > > Hi Raghu,
> > >
> > > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> > >
> > > after enabling debugging I see that authentication is being completed.
> I
> > > don't see any debug logs being generated for authorization part (I
> might
> > be
> > > missing something).
> > >
> > > you can also set the log level to debug in properties and see whats
> going
> > > on.
> > >
> > > Thanks,
> > > Kiran
> > >
> > > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> > > wrote:
> > >
> > >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> > what
> > >> the principal ID is by using keytool or openssl is not going to help
> > from
> > >> my experience. The best is to add some logging to output the SSL
> client
> > ID
> > >> in the org.apache.kafka.common.network.SslTransportLayer.
> > peerPrincipal()
> > >> .
> > >> The p.getName() is what you are looking at.
> > >>
> > >> Instead of adding it to the super user list in your server props file,
> > add
> > >> ACLs to that user using the kafka-acls.sh in the bin directory.
> > >>
> > >>
> > >>
> > >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com>
> wrote:
> > >>
> > >> > Thanks Shrikant for your reply, but I did consumer part also and
> more
> > >> over
> > >> > I am not facing this issue only with consumer, I am getting this
> > errors
> > >> > with producer as well as consumer
> > >> >
> > >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> > >> wrote:
> > >> >
> > >> > > You need to execute kafka-acls.sh with --consumer to enable
> > >> consumption
> > >> > > from kafka.
> > >> > >
> > >> > > _________________________________________________
> > >> > > Shrikant Patel | 817.367.4302
> > >> > > Enterprise Architecture Team
> > >> > > PDX-NHIN
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> > >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> > >> > > To: security@kafka.apache.org
> > >> > > Subject: Kafka ACL's with SSL Protocol is not working
> > >> > >
> > >> > > Hi All,
> > >> > >
> > >> > > I am trying to enable ACL's in my Kafka cluster with along with
> SSL
> > >> > > Protocol.
> > >> > >
> > >> > > I tried with each and every parameters but no luck, so I need help
> > to
> > >> > > enable the SSL(without Kerberos) and I am attaching all the
> > >> configuration
> > >> > > details in this.
> > >> > >
> > >> > > Kindly Help me.
> > >> > >
> > >> > >
> > >> > > *I tested SSL without ACL, it worked fine
> > >> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >)*
> > >> > >
> > >> > >
> > >> > > *This is my Kafka server properties file:*
> > >> > >
> > >> > > *############################# ACL SETTINGS
> > >> > #############################*
> > >> > >
> > >> > > *auto.create.topics.enable=true*
> > >> > >
> > >> > > *authorizer.class.name
> > >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> > >> Authorizer*
> > >> > >
> > >> > > *security.inter.broker.protocol=SSL*
> > >> > >
> > >> > > *#allow.everyone.if.no.acl.found=true*
> > >> > >
> > >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> > >> > >
> > >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > >> > >
> > >> > > *#super.users=User:Raghu;User:Admin*
> > >> > >
> > >> > > *#offsets.storage=kafka*
> > >> > >
> > >> > > *#dual.commit.enabled=true*
> > >> > >
> > >> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> > >> http://10.247.195.122:9092
> > >> > >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > >> > > <http://10.247.195.122:9093>*
> > >> > >
> > >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>*
> > >> > >
> > >> > >
> > >> > > *
> > >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> > keystore.jks*
> > >> > >
> > >> > > * ssl.keystore.password=123456*
> > >> > >
> > >> > > * ssl.key.password=123456*
> > >> > >
> > >> > > *
> > >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> > >> > truststore.jks*
> > >> > >
> > >> > > * ssl.truststore.password=123456*
> > >> > >
> > >> > >
> > >> > >
> > >> > > *Set the ACL from Authorizer CLI:*
> > >> > >
> > >> > > > *bin/kafka-acls.sh --authorizer-properties
> > >> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181
> >
> > >> > --list
> > >> > > --topic ssltopic*
> > >> > >
> > >> > > *Current ACLs for resource `Topic:ssltopic`: *
> > >> > >
> > >> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown,
> ST=Unknown,
> > >> > > C=Unknown has Allow permission for operations: Write from hosts:
> * *
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-producer.sh
> > >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> > >> --topic
> > >> > > ssltopic --producer.config client-ssl.properties*
> > >> > >
> > >> > >
> > >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> > >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat
> client-ssl.properties*
> > >> > >
> > >> > > *#group.id <http://group.id>=sslgroup*
> > >> > >
> > >> > > *security.protocol=SSL*
> > >> > >
> > >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> > >> > >
> > >> > > *ssl.truststore.password=123456*
> > >> > >
> > >> > > * #Configure Below if you use Client Auth*
> > >> > >
> > >> > >
> > >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> > >> > >
> > >> > > *ssl.keystore.password=123456*
> > >> > >
> > >> > > *ssl.key.password=123456*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-consumer.sh
> > >> > > --bootstrap-server 10.247.195.122:9093 <
> http://10.247.195.122:9093>
> > >> > > --new-consumer --consumer.config client-ssl.properties --topic
> > >> ssltopic
> > >> > > --from-beginning*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> > consumer:
> > >> > > (kafka.tools.ConsoleConsumer$)*
> > >> > >
> > >> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> > >> > > authorized to access group: console-consumer-52826*
> > >> > >
> > >> > >
> > >> > > Thanks in advance,
> > >> > >
> > >> > > Raghu - raghu98499@gmail.com
> > >> > > This e-mail and its contents (to include attachments) are the
> > >> property of
> > >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> > >> including
> > >> > > but not limited to Rx.com Community Healthcare Network, Inc. and
> its
> > >> > > subsidiaries, and may contain confidential and proprietary or
> > >> privileged
> > >> > > information. If you are not the intended recipient of this e-mail,
> > you
> > >> > are
> > >> > > hereby notified that any unauthorized disclosure, copying, or
> > >> > distribution
> > >> > > of this e-mail or of its attachments, or the taking of any
> > >> unauthorized
> > >> > > action based on information contained herein is strictly
> prohibited.
> > >> > > Unauthorized use of information contained herein may subject you
> to
> > >> civil
> > >> > > and criminal prosecution and penalties. If you are not the
> intended
> > >> > > recipient, please immediately notify the sender by telephone at
> > >> > > 800-433-5719 or return e-mail and permanently delete the original
> > >> > e-mail.
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > G.Kiran Kumar
> > >
> >
> >
> >
> > --
> > G.Kiran Kumar
> >
>
--
Raghav
Re: Kafka ACL's with SSL Protocol is not working
Posted by Raghu B <ra...@gmail.com>.
Thanks Gerard & Derar for your valuable suggestions but I am able to send
and receive messages with SSL (Without ACL configuration).
I used only SSL port on 9093 and Enabled Inter broker communication as SSL
but If I enable SSL it is creating the Issues.
Anyway Let me try once again from side from scratch and I will let you know.
You guys are awesome :)
On Fri, Dec 16, 2016 at 12:41 AM, Derar Alassi <de...@gmail.com>
wrote:
> Create proper JKS that has a certificate that is issued by a CA that is
> trusted by the Kafka brokers, and you expect a principal with the DN in
> your client cert. Spend more time on getting this done correctly and things
> will work fine.
>
> On Thu, Dec 15, 2016 at 9:11 PM, Gerard Klijs <ge...@openweb.nl> wrote:
>
> > Most likely something went wrong creating the keystores, causing the SSL
> > handshake to fail. Its important to have a valid chain, from the
> > certificate in the struststore, and then maybe intermediates tot the
> > keystore.
> >
> > On Fri, Dec 16, 2016, 00:32 Raghu B <ra...@gmail.com> wrote:
> >
> > Thanks Derar & Kiran, your suggestions are very useful.
> >
> > I enabled Log4J debug mode and found that my client is trying to connect
> to
> > the Kafka server with the *User:ANONYMOUS, *It is really strange.
> >
> >
> > I added a new Super.User with the name *User:ANONYMOUS *then I am able to
> > send and receive the messages without any issues.
> >
> > And now the question is how can I set my username name from Anonymous to
> > something like
> > *User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > which
> >
> > comes from SSL cert/keystore.
> >
> > Please help me with your inputs.
> >
> > Thanks in Advance,
> > Raghu
> >
> > On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com>
> > wrote:
> >
> > > I have just noticed that I am using the user which is not configured in
> > the
> > > kafka server jaas config file..
> > >
> > >
> > >
> > > On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> > > wrote:
> > >
> > > > Hi Raghu,
> > > >
> > > > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> > > >
> > > > after enabling debugging I see that authentication is being
> completed.
> > I
> > > > don't see any debug logs being generated for authorization part (I
> > might
> > > be
> > > > missing something).
> > > >
> > > > you can also set the log level to debug in properties and see whats
> > going
> > > > on.
> > > >
> > > > Thanks,
> > > > Kiran
> > > >
> > > > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <
> derar.alassi@gmail.com>
> > > > wrote:
> > > >
> > > >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> > > what
> > > >> the principal ID is by using keytool or openssl is not going to help
> > > from
> > > >> my experience. The best is to add some logging to output the SSL
> > client
> > > ID
> > > >> in the org.apache.kafka.common.network.SslTransportLayer.
> > > peerPrincipal()
> > > >> .
> > > >> The p.getName() is what you are looking at.
> > > >>
> > > >> Instead of adding it to the super user list in your server props
> file,
> > > add
> > > >> ACLs to that user using the kafka-acls.sh in the bin directory.
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com>
> > wrote:
> > > >>
> > > >> > Thanks Shrikant for your reply, but I did consumer part also and
> > more
> > > >> over
> > > >> > I am not facing this issue only with consumer, I am getting this
> > > errors
> > > >> > with producer as well as consumer
> > > >> >
> > > >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <
> SPatel@pdxinc.com>
> > > >> wrote:
> > > >> >
> > > >> > > You need to execute kafka-acls.sh with --consumer to enable
> > > >> consumption
> > > >> > > from kafka.
> > > >> > >
> > > >> > > _________________________________________________
> > > >> > > Shrikant Patel | 817.367.4302 <(817)%20367-4302>
> > > >> > > Enterprise Architecture Team
> > > >> > > PDX-NHIN
> > > >> > >
> > > >> > > -----Original Message-----
> > > >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> > > >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> > > >> > > To: security@kafka.apache.org
> > > >> > > Subject: Kafka ACL's with SSL Protocol is not working
> > > >> > >
> > > >> > > Hi All,
> > > >> > >
> > > >> > > I am trying to enable ACL's in my Kafka cluster with along with
> > SSL
> > > >> > > Protocol.
> > > >> > >
> > > >> > > I tried with each and every parameters but no luck, so I need
> help
> > > to
> > > >> > > enable the SSL(without Kerberos) and I am attaching all the
> > > >> configuration
> > > >> > > details in this.
> > > >> > >
> > > >> > > Kindly Help me.
> > > >> > >
> > > >> > >
> > > >> > > *I tested SSL without ACL, it worked fine
> > > >> > > (listeners=SSL://10.247.195.122:9093 <
> http://10.247.195.122:9093
> > >)*
> > > >> > >
> > > >> > >
> > > >> > > *This is my Kafka server properties file:*
> > > >> > >
> > > >> > > *############################# ACL SETTINGS
> > > >> > #############################*
> > > >> > >
> > > >> > > *auto.create.topics.enable=true*
> > > >> > >
> > > >> > > *authorizer.class.name
> > > >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> > > >> Authorizer*
> > > >> > >
> > > >> > > *security.inter.broker.protocol=SSL*
> > > >> > >
> > > >> > > *#allow.everyone.if.no.acl.found=true*
> > > >> > >
> > > >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> > > >> > >
> > > >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > > >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > > >> > >
> > > >> > > *#super.users=User:Raghu;User:Admin*
> > > >> > >
> > > >> > > *#offsets.storage=kafka*
> > > >> > >
> > > >> > > *#dual.commit.enabled=true*
> > > >> > >
> > > >> > > *listeners=SSL://10.247.195.122:9093 <
> http://10.247.195.122:9093
> > >*
> > > >> > >
> > > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> > > >> http://10.247.195.122:9092
> > > >> > >*
> > > >> > >
> > > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> > > >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > > >> > > <http://10.247.195.122:9093>*
> > > >> > >
> > > >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > > >> > > <http://10.247.195.122:9092>*
> > > >> > >
> > > >> > >
> > > >> > > *
> > > >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> > > keystore.jks*
> > > >> > >
> > > >> > > * ssl.keystore.password=123456*
> > > >> > >
> > > >> > > * ssl.key.password=123456*
> > > >> > >
> > > >> > > *
> > > >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> > > >> > truststore.jks*
> > > >> > >
> > > >> > > * ssl.truststore.password=123456*
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > *Set the ACL from Authorizer CLI:*
> > > >> > >
> > > >> > > > *bin/kafka-acls.sh --authorizer-properties
> > > >> > > zookeeper.connect=10.247.195.122:2181 <
> http://10.247.195.122:2181
> > >
> > > >> > --list
> > > >> > > --topic ssltopic*
> > > >> > >
> > > >> > > *Current ACLs for resource `Topic:ssltopic`: *
> > > >> > >
> > > >> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown,
> > ST=Unknown,
> > > >> > > C=Unknown has Allow permission for operations: Write from hosts:
> > * *
> > > >> > >
> > > >> > >
> > > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > > >> bin/kafka-console-producer.sh
> > > >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> > > >> --topic
> > > >> > > ssltopic --producer.config client-ssl.properties*
> > > >> > >
> > > >> > >
> > > >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata
> with
> > > >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > >> > > (org.apache.kafka.clients.NetworkClient)*
> > > >> > >
> > > >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata
> with
> > > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > >> > > (org.apache.kafka.clients.NetworkClient)*
> > > >> > >
> > > >> > >
> > > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat
> > client-ssl.properties*
> > > >> > >
> > > >> > > *#group.id <http://group.id>=sslgroup*
> > > >> > >
> > > >> > > *security.protocol=SSL*
> > > >> > >
> > > >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > > >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> > > >> > >
> > > >> > > *ssl.truststore.password=123456*
> > > >> > >
> > > >> > > * #Configure Below if you use Client Auth*
> > > >> > >
> > > >> > >
> > > >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > > >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> > > >> > >
> > > >> > > *ssl.keystore.password=123456*
> > > >> > >
> > > >> > > *ssl.key.password=123456*
> > > >> > >
> > > >> > >
> > > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > > >> bin/kafka-console-consumer.sh
> > > >> > > --bootstrap-server 10.247.195.122:9093 <
> > http://10.247.195.122:9093>
> > > >> > > --new-consumer --consumer.config client-ssl.properties --topic
> > > >> ssltopic
> > > >> > > --from-beginning*
> > > >> > >
> > > >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata
> with
> > > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > >> > > (org.apache.kafka.clients.NetworkClient)*
> > > >> > >
> > > >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> > > consumer:
> > > >> > > (kafka.tools.ConsoleConsumer$)*
> > > >> > >
> > > >> > > *org.apache.kafka.common.errors.GroupAuthorizationException:
> Not
> > > >> > > authorized to access group: console-consumer-52826*
> > > >> > >
> > > >> > >
> > > >> > > Thanks in advance,
> > > >> > >
> > > >> > > Raghu - raghu98499@gmail.com
> > > >> > > This e-mail and its contents (to include attachments) are the
> > > >> property of
> > > >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> > > >> including
> > > >> > > but not limited to Rx.com Community Healthcare Network, Inc. and
> > its
> > > >> > > subsidiaries, and may contain confidential and proprietary or
> > > >> privileged
> > > >> > > information. If you are not the intended recipient of this
> e-mail,
> > > you
> > > >> > are
> > > >> > > hereby notified that any unauthorized disclosure, copying, or
> > > >> > distribution
> > > >> > > of this e-mail or of its attachments, or the taking of any
> > > >> unauthorized
> > > >> > > action based on information contained herein is strictly
> > prohibited.
> > > >> > > Unauthorized use of information contained herein may subject you
> > to
> > > >> civil
> > > >> > > and criminal prosecution and penalties. If you are not the
> > intended
> > > >> > > recipient, please immediately notify the sender by telephone at
> > > >> > > 800-433-5719 <(800)%20433-5719> or return e-mail and
> permanently
> > delete the original
> > > >> > e-mail.
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > G.Kiran Kumar
> > > >
> > >
> > >
> > >
> > > --
> > > G.Kiran Kumar
> > >
> >
> >
>
Re: Kafka ACL's with SSL Protocol is not working
Posted by Derar Alassi <de...@gmail.com>.
Create proper JKS that has a certificate that is issued by a CA that is
trusted by the Kafka brokers, and you expect a principal with the DN in
your client cert. Spend more time on getting this done correctly and things
will work fine.
On Thu, Dec 15, 2016 at 9:11 PM, Gerard Klijs <ge...@openweb.nl> wrote:
> Most likely something went wrong creating the keystores, causing the SSL
> handshake to fail. Its important to have a valid chain, from the
> certificate in the struststore, and then maybe intermediates tot the
> keystore.
>
> On Fri, Dec 16, 2016, 00:32 Raghu B <ra...@gmail.com> wrote:
>
> Thanks Derar & Kiran, your suggestions are very useful.
>
> I enabled Log4J debug mode and found that my client is trying to connect to
> the Kafka server with the *User:ANONYMOUS, *It is really strange.
>
>
> I added a new Super.User with the name *User:ANONYMOUS *then I am able to
> send and receive the messages without any issues.
>
> And now the question is how can I set my username name from Anonymous to
> something like
> *User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> which
>
> comes from SSL cert/keystore.
>
> Please help me with your inputs.
>
> Thanks in Advance,
> Raghu
>
> On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com>
> wrote:
>
> > I have just noticed that I am using the user which is not configured in
> the
> > kafka server jaas config file..
> >
> >
> >
> > On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> > wrote:
> >
> > > Hi Raghu,
> > >
> > > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> > >
> > > after enabling debugging I see that authentication is being completed.
> I
> > > don't see any debug logs being generated for authorization part (I
> might
> > be
> > > missing something).
> > >
> > > you can also set the log level to debug in properties and see whats
> going
> > > on.
> > >
> > > Thanks,
> > > Kiran
> > >
> > > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> > > wrote:
> > >
> > >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> > what
> > >> the principal ID is by using keytool or openssl is not going to help
> > from
> > >> my experience. The best is to add some logging to output the SSL
> client
> > ID
> > >> in the org.apache.kafka.common.network.SslTransportLayer.
> > peerPrincipal()
> > >> .
> > >> The p.getName() is what you are looking at.
> > >>
> > >> Instead of adding it to the super user list in your server props file,
> > add
> > >> ACLs to that user using the kafka-acls.sh in the bin directory.
> > >>
> > >>
> > >>
> > >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com>
> wrote:
> > >>
> > >> > Thanks Shrikant for your reply, but I did consumer part also and
> more
> > >> over
> > >> > I am not facing this issue only with consumer, I am getting this
> > errors
> > >> > with producer as well as consumer
> > >> >
> > >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> > >> wrote:
> > >> >
> > >> > > You need to execute kafka-acls.sh with --consumer to enable
> > >> consumption
> > >> > > from kafka.
> > >> > >
> > >> > > _________________________________________________
> > >> > > Shrikant Patel | 817.367.4302 <(817)%20367-4302>
> > >> > > Enterprise Architecture Team
> > >> > > PDX-NHIN
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> > >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> > >> > > To: security@kafka.apache.org
> > >> > > Subject: Kafka ACL's with SSL Protocol is not working
> > >> > >
> > >> > > Hi All,
> > >> > >
> > >> > > I am trying to enable ACL's in my Kafka cluster with along with
> SSL
> > >> > > Protocol.
> > >> > >
> > >> > > I tried with each and every parameters but no luck, so I need help
> > to
> > >> > > enable the SSL(without Kerberos) and I am attaching all the
> > >> configuration
> > >> > > details in this.
> > >> > >
> > >> > > Kindly Help me.
> > >> > >
> > >> > >
> > >> > > *I tested SSL without ACL, it worked fine
> > >> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >)*
> > >> > >
> > >> > >
> > >> > > *This is my Kafka server properties file:*
> > >> > >
> > >> > > *############################# ACL SETTINGS
> > >> > #############################*
> > >> > >
> > >> > > *auto.create.topics.enable=true*
> > >> > >
> > >> > > *authorizer.class.name
> > >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> > >> Authorizer*
> > >> > >
> > >> > > *security.inter.broker.protocol=SSL*
> > >> > >
> > >> > > *#allow.everyone.if.no.acl.found=true*
> > >> > >
> > >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> > >> > >
> > >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > >> > >
> > >> > > *#super.users=User:Raghu;User:Admin*
> > >> > >
> > >> > > *#offsets.storage=kafka*
> > >> > >
> > >> > > *#dual.commit.enabled=true*
> > >> > >
> > >> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> > >> http://10.247.195.122:9092
> > >> > >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > >> > > <http://10.247.195.122:9093>*
> > >> > >
> > >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>*
> > >> > >
> > >> > >
> > >> > > *
> > >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> > keystore.jks*
> > >> > >
> > >> > > * ssl.keystore.password=123456*
> > >> > >
> > >> > > * ssl.key.password=123456*
> > >> > >
> > >> > > *
> > >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> > >> > truststore.jks*
> > >> > >
> > >> > > * ssl.truststore.password=123456*
> > >> > >
> > >> > >
> > >> > >
> > >> > > *Set the ACL from Authorizer CLI:*
> > >> > >
> > >> > > > *bin/kafka-acls.sh --authorizer-properties
> > >> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181
> >
> > >> > --list
> > >> > > --topic ssltopic*
> > >> > >
> > >> > > *Current ACLs for resource `Topic:ssltopic`: *
> > >> > >
> > >> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown,
> ST=Unknown,
> > >> > > C=Unknown has Allow permission for operations: Write from hosts:
> * *
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-producer.sh
> > >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> > >> --topic
> > >> > > ssltopic --producer.config client-ssl.properties*
> > >> > >
> > >> > >
> > >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> > >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat
> client-ssl.properties*
> > >> > >
> > >> > > *#group.id <http://group.id>=sslgroup*
> > >> > >
> > >> > > *security.protocol=SSL*
> > >> > >
> > >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> > >> > >
> > >> > > *ssl.truststore.password=123456*
> > >> > >
> > >> > > * #Configure Below if you use Client Auth*
> > >> > >
> > >> > >
> > >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> > >> > >
> > >> > > *ssl.keystore.password=123456*
> > >> > >
> > >> > > *ssl.key.password=123456*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-consumer.sh
> > >> > > --bootstrap-server 10.247.195.122:9093 <
> http://10.247.195.122:9093>
> > >> > > --new-consumer --consumer.config client-ssl.properties --topic
> > >> ssltopic
> > >> > > --from-beginning*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> > consumer:
> > >> > > (kafka.tools.ConsoleConsumer$)*
> > >> > >
> > >> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> > >> > > authorized to access group: console-consumer-52826*
> > >> > >
> > >> > >
> > >> > > Thanks in advance,
> > >> > >
> > >> > > Raghu - raghu98499@gmail.com
> > >> > > This e-mail and its contents (to include attachments) are the
> > >> property of
> > >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> > >> including
> > >> > > but not limited to Rx.com Community Healthcare Network, Inc. and
> its
> > >> > > subsidiaries, and may contain confidential and proprietary or
> > >> privileged
> > >> > > information. If you are not the intended recipient of this e-mail,
> > you
> > >> > are
> > >> > > hereby notified that any unauthorized disclosure, copying, or
> > >> > distribution
> > >> > > of this e-mail or of its attachments, or the taking of any
> > >> unauthorized
> > >> > > action based on information contained herein is strictly
> prohibited.
> > >> > > Unauthorized use of information contained herein may subject you
> to
> > >> civil
> > >> > > and criminal prosecution and penalties. If you are not the
> intended
> > >> > > recipient, please immediately notify the sender by telephone at
> > >> > > 800-433-5719 <(800)%20433-5719> or return e-mail and permanently
> delete the original
> > >> > e-mail.
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > G.Kiran Kumar
> > >
> >
> >
> >
> > --
> > G.Kiran Kumar
> >
>
>
Re: Kafka ACL's with SSL Protocol is not working
Posted by Gerard Klijs <ge...@openweb.nl>.
Most likely something went wrong creating the keystores, causing the SSL
handshake to fail. Its important to have a valid chain, from the
certificate in the struststore, and then maybe intermediates tot the
keystore.
On Fri, Dec 16, 2016, 00:32 Raghu B <ra...@gmail.com> wrote:
Thanks Derar & Kiran, your suggestions are very useful.
I enabled Log4J debug mode and found that my client is trying to connect to
the Kafka server with the *User:ANONYMOUS, *It is really strange.
I added a new Super.User with the name *User:ANONYMOUS *then I am able to
send and receive the messages without any issues.
And now the question is how can I set my username name from Anonymous to
something like
*User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
which
comes from SSL cert/keystore.
Please help me with your inputs.
Thanks in Advance,
Raghu
On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com> wrote:
> I have just noticed that I am using the user which is not configured in
the
> kafka server jaas config file..
>
>
>
> On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> wrote:
>
> > Hi Raghu,
> >
> > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> >
> > after enabling debugging I see that authentication is being completed. I
> > don't see any debug logs being generated for authorization part (I might
> be
> > missing something).
> >
> > you can also set the log level to debug in properties and see whats
going
> > on.
> >
> > Thanks,
> > Kiran
> >
> > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> > wrote:
> >
> >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> what
> >> the principal ID is by using keytool or openssl is not going to help
> from
> >> my experience. The best is to add some logging to output the SSL client
> ID
> >> in the org.apache.kafka.common.network.SslTransportLayer.
> peerPrincipal()
> >> .
> >> The p.getName() is what you are looking at.
> >>
> >> Instead of adding it to the super user list in your server props file,
> add
> >> ACLs to that user using the kafka-acls.sh in the bin directory.
> >>
> >>
> >>
> >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com> wrote:
> >>
> >> > Thanks Shrikant for your reply, but I did consumer part also and more
> >> over
> >> > I am not facing this issue only with consumer, I am getting this
> errors
> >> > with producer as well as consumer
> >> >
> >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> >> wrote:
> >> >
> >> > > You need to execute kafka-acls.sh with --consumer to enable
> >> consumption
> >> > > from kafka.
> >> > >
> >> > > _________________________________________________
> >> > > Shrikant Patel | 817.367.4302
> >> > > Enterprise Architecture Team
> >> > > PDX-NHIN
> >> > >
> >> > > -----Original Message-----
> >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> >> > > To: security@kafka.apache.org
> >> > > Subject: Kafka ACL's with SSL Protocol is not working
> >> > >
> >> > > Hi All,
> >> > >
> >> > > I am trying to enable ACL's in my Kafka cluster with along with SSL
> >> > > Protocol.
> >> > >
> >> > > I tried with each and every parameters but no luck, so I need help
> to
> >> > > enable the SSL(without Kerberos) and I am attaching all the
> >> configuration
> >> > > details in this.
> >> > >
> >> > > Kindly Help me.
> >> > >
> >> > >
> >> > > *I tested SSL without ACL, it worked fine
> >> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
> >> > >
> >> > >
> >> > > *This is my Kafka server properties file:*
> >> > >
> >> > > *############################# ACL SETTINGS
> >> > #############################*
> >> > >
> >> > > *auto.create.topics.enable=true*
> >> > >
> >> > > *authorizer.class.name
> >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> >> Authorizer*
> >> > >
> >> > > *security.inter.broker.protocol=SSL*
> >> > >
> >> > > *#allow.everyone.if.no.acl.found=true*
> >> > >
> >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> >> > >
> >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> >> > >
> >> > > *#super.users=User:Raghu;User:Admin*
> >> > >
> >> > > *#offsets.storage=kafka*
> >> > >
> >> > > *#dual.commit.enabled=true*
> >> > >
> >> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
> >> > >
> >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> >> http://10.247.195.122:9092
> >> > >*
> >> > >
> >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> >> > > <http://10.247.195.122:9093>*
> >> > >
> >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> >> > > <http://10.247.195.122:9092>*
> >> > >
> >> > >
> >> > > *
> >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> keystore.jks*
> >> > >
> >> > > * ssl.keystore.password=123456*
> >> > >
> >> > > * ssl.key.password=123456*
> >> > >
> >> > > *
> >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> >> > truststore.jks*
> >> > >
> >> > > * ssl.truststore.password=123456*
> >> > >
> >> > >
> >> > >
> >> > > *Set the ACL from Authorizer CLI:*
> >> > >
> >> > > > *bin/kafka-acls.sh --authorizer-properties
> >> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181>
> >> > --list
> >> > > --topic ssltopic*
> >> > >
> >> > > *Current ACLs for resource `Topic:ssltopic`: *
> >> > >
> >> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> >> > > C=Unknown has Allow permission for operations: Write from hosts: *
*
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> >> bin/kafka-console-producer.sh
> >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> >> --topic
> >> > > ssltopic --producer.config client-ssl.properties*
> >> > >
> >> > >
> >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
> >> > >
> >> > > *#group.id <http://group.id>=sslgroup*
> >> > >
> >> > > *security.protocol=SSL*
> >> > >
> >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> >> > >
> >> > > *ssl.truststore.password=123456*
> >> > >
> >> > > * #Configure Below if you use Client Auth*
> >> > >
> >> > >
> >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> >> > >
> >> > > *ssl.keystore.password=123456*
> >> > >
> >> > > *ssl.key.password=123456*
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> >> bin/kafka-console-consumer.sh
> >> > > --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> >> > > --new-consumer --consumer.config client-ssl.properties --topic
> >> ssltopic
> >> > > --from-beginning*
> >> > >
> >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> consumer:
> >> > > (kafka.tools.ConsoleConsumer$)*
> >> > >
> >> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> >> > > authorized to access group: console-consumer-52826*
> >> > >
> >> > >
> >> > > Thanks in advance,
> >> > >
> >> > > Raghu - raghu98499@gmail.com
> >> > > This e-mail and its contents (to include attachments) are the
> >> property of
> >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> >> including
> >> > > but not limited to Rx.com Community Healthcare Network, Inc. and
its
> >> > > subsidiaries, and may contain confidential and proprietary or
> >> privileged
> >> > > information. If you are not the intended recipient of this e-mail,
> you
> >> > are
> >> > > hereby notified that any unauthorized disclosure, copying, or
> >> > distribution
> >> > > of this e-mail or of its attachments, or the taking of any
> >> unauthorized
> >> > > action based on information contained herein is strictly
prohibited.
> >> > > Unauthorized use of information contained herein may subject you to
> >> civil
> >> > > and criminal prosecution and penalties. If you are not the intended
> >> > > recipient, please immediately notify the sender by telephone at
> >> > > 800-433-5719 or return e-mail and permanently delete the original
> >> > e-mail.
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > G.Kiran Kumar
> >
>
>
>
> --
> G.Kiran Kumar
>
Re: Kafka ACL's with SSL Protocol is not working
Posted by Raghu B <ra...@gmail.com>.
Thanks Derar & Kiran, your suggestions are very useful.
I enabled Log4J debug mode and found that my client is trying to connect to
the Kafka server with the *User:ANONYMOUS, *It is really strange.
I added a new Super.User with the name *User:ANONYMOUS *then I am able to
send and receive the messages without any issues.
And now the question is how can I set my username name from Anonymous to
something like
*User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"* which
comes from SSL cert/keystore.
Please help me with your inputs.
Thanks in Advance,
Raghu
On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com> wrote:
> I have just noticed that I am using the user which is not configured in the
> kafka server jaas config file..
>
>
>
> On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> wrote:
>
> > Hi Raghu,
> >
> > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> >
> > after enabling debugging I see that authentication is being completed. I
> > don't see any debug logs being generated for authorization part (I might
> be
> > missing something).
> >
> > you can also set the log level to debug in properties and see whats going
> > on.
> >
> > Thanks,
> > Kiran
> >
> > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> > wrote:
> >
> >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> what
> >> the principal ID is by using keytool or openssl is not going to help
> from
> >> my experience. The best is to add some logging to output the SSL client
> ID
> >> in the org.apache.kafka.common.network.SslTransportLayer.
> peerPrincipal()
> >> .
> >> The p.getName() is what you are looking at.
> >>
> >> Instead of adding it to the super user list in your server props file,
> add
> >> ACLs to that user using the kafka-acls.sh in the bin directory.
> >>
> >>
> >>
> >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com> wrote:
> >>
> >> > Thanks Shrikant for your reply, but I did consumer part also and more
> >> over
> >> > I am not facing this issue only with consumer, I am getting this
> errors
> >> > with producer as well as consumer
> >> >
> >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> >> wrote:
> >> >
> >> > > You need to execute kafka-acls.sh with --consumer to enable
> >> consumption
> >> > > from kafka.
> >> > >
> >> > > _________________________________________________
> >> > > Shrikant Patel | 817.367.4302
> >> > > Enterprise Architecture Team
> >> > > PDX-NHIN
> >> > >
> >> > > -----Original Message-----
> >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> >> > > To: security@kafka.apache.org
> >> > > Subject: Kafka ACL's with SSL Protocol is not working
> >> > >
> >> > > Hi All,
> >> > >
> >> > > I am trying to enable ACL's in my Kafka cluster with along with SSL
> >> > > Protocol.
> >> > >
> >> > > I tried with each and every parameters but no luck, so I need help
> to
> >> > > enable the SSL(without Kerberos) and I am attaching all the
> >> configuration
> >> > > details in this.
> >> > >
> >> > > Kindly Help me.
> >> > >
> >> > >
> >> > > *I tested SSL without ACL, it worked fine
> >> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
> >> > >
> >> > >
> >> > > *This is my Kafka server properties file:*
> >> > >
> >> > > *############################# ACL SETTINGS
> >> > #############################*
> >> > >
> >> > > *auto.create.topics.enable=true*
> >> > >
> >> > > *authorizer.class.name
> >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> >> Authorizer*
> >> > >
> >> > > *security.inter.broker.protocol=SSL*
> >> > >
> >> > > *#allow.everyone.if.no.acl.found=true*
> >> > >
> >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> >> > >
> >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> >> > >
> >> > > *#super.users=User:Raghu;User:Admin*
> >> > >
> >> > > *#offsets.storage=kafka*
> >> > >
> >> > > *#dual.commit.enabled=true*
> >> > >
> >> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
> >> > >
> >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> >> http://10.247.195.122:9092
> >> > >*
> >> > >
> >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> >> > > <http://10.247.195.122:9093>*
> >> > >
> >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> >> > > <http://10.247.195.122:9092>*
> >> > >
> >> > >
> >> > > *
> >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> keystore.jks*
> >> > >
> >> > > * ssl.keystore.password=123456*
> >> > >
> >> > > * ssl.key.password=123456*
> >> > >
> >> > > *
> >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> >> > truststore.jks*
> >> > >
> >> > > * ssl.truststore.password=123456*
> >> > >
> >> > >
> >> > >
> >> > > *Set the ACL from Authorizer CLI:*
> >> > >
> >> > > > *bin/kafka-acls.sh --authorizer-properties
> >> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181>
> >> > --list
> >> > > --topic ssltopic*
> >> > >
> >> > > *Current ACLs for resource `Topic:ssltopic`: *
> >> > >
> >> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> >> > > C=Unknown has Allow permission for operations: Write from hosts: * *
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> >> bin/kafka-console-producer.sh
> >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> >> --topic
> >> > > ssltopic --producer.config client-ssl.properties*
> >> > >
> >> > >
> >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
> >> > >
> >> > > *#group.id <http://group.id>=sslgroup*
> >> > >
> >> > > *security.protocol=SSL*
> >> > >
> >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> >> > >
> >> > > *ssl.truststore.password=123456*
> >> > >
> >> > > * #Configure Below if you use Client Auth*
> >> > >
> >> > >
> >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> >> > >
> >> > > *ssl.keystore.password=123456*
> >> > >
> >> > > *ssl.key.password=123456*
> >> > >
> >> > >
> >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> >> bin/kafka-console-consumer.sh
> >> > > --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> >> > > --new-consumer --consumer.config client-ssl.properties --topic
> >> ssltopic
> >> > > --from-beginning*
> >> > >
> >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> >> > > (org.apache.kafka.clients.NetworkClient)*
> >> > >
> >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> consumer:
> >> > > (kafka.tools.ConsoleConsumer$)*
> >> > >
> >> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> >> > > authorized to access group: console-consumer-52826*
> >> > >
> >> > >
> >> > > Thanks in advance,
> >> > >
> >> > > Raghu - raghu98499@gmail.com
> >> > > This e-mail and its contents (to include attachments) are the
> >> property of
> >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> >> including
> >> > > but not limited to Rx.com Community Healthcare Network, Inc. and its
> >> > > subsidiaries, and may contain confidential and proprietary or
> >> privileged
> >> > > information. If you are not the intended recipient of this e-mail,
> you
> >> > are
> >> > > hereby notified that any unauthorized disclosure, copying, or
> >> > distribution
> >> > > of this e-mail or of its attachments, or the taking of any
> >> unauthorized
> >> > > action based on information contained herein is strictly prohibited.
> >> > > Unauthorized use of information contained herein may subject you to
> >> civil
> >> > > and criminal prosecution and penalties. If you are not the intended
> >> > > recipient, please immediately notify the sender by telephone at
> >> > > 800-433-5719 or return e-mail and permanently delete the original
> >> > e-mail.
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > G.Kiran Kumar
> >
>
>
>
> --
> G.Kiran Kumar
>
Re: Kafka ACL's with SSL Protocol is not working
Posted by kiran kumar <ki...@gmail.com>.
I have just noticed that I am using the user which is not configured in the
kafka server jaas config file..
On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com> wrote:
> Hi Raghu,
>
> I am also facing the same issue but with the SASL_PLAINTEXT protocol.
>
> after enabling debugging I see that authentication is being completed. I
> don't see any debug logs being generated for authorization part (I might be
> missing something).
>
> you can also set the log level to debug in properties and see whats going
> on.
>
> Thanks,
> Kiran
>
> On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> wrote:
>
>> Make sure that the principal ID is exactly what Kafka sees. Guessing what
>> the principal ID is by using keytool or openssl is not going to help from
>> my experience. The best is to add some logging to output the SSL client ID
>> in the org.apache.kafka.common.network.SslTransportLayer.peerPrincipal()
>> .
>> The p.getName() is what you are looking at.
>>
>> Instead of adding it to the super user list in your server props file, add
>> ACLs to that user using the kafka-acls.sh in the bin directory.
>>
>>
>>
>> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com> wrote:
>>
>> > Thanks Shrikant for your reply, but I did consumer part also and more
>> over
>> > I am not facing this issue only with consumer, I am getting this errors
>> > with producer as well as consumer
>> >
>> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
>> wrote:
>> >
>> > > You need to execute kafka-acls.sh with --consumer to enable
>> consumption
>> > > from kafka.
>> > >
>> > > _________________________________________________
>> > > Shrikant Patel | 817.367.4302
>> > > Enterprise Architecture Team
>> > > PDX-NHIN
>> > >
>> > > -----Original Message-----
>> > > From: Raghu B [mailto:raghu98499@gmail.com]
>> > > Sent: Wednesday, December 14, 2016 5:42 PM
>> > > To: security@kafka.apache.org
>> > > Subject: Kafka ACL's with SSL Protocol is not working
>> > >
>> > > Hi All,
>> > >
>> > > I am trying to enable ACL's in my Kafka cluster with along with SSL
>> > > Protocol.
>> > >
>> > > I tried with each and every parameters but no luck, so I need help to
>> > > enable the SSL(without Kerberos) and I am attaching all the
>> configuration
>> > > details in this.
>> > >
>> > > Kindly Help me.
>> > >
>> > >
>> > > *I tested SSL without ACL, it worked fine
>> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
>> > >
>> > >
>> > > *This is my Kafka server properties file:*
>> > >
>> > > *############################# ACL SETTINGS
>> > #############################*
>> > >
>> > > *auto.create.topics.enable=true*
>> > >
>> > > *authorizer.class.name
>> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
>> Authorizer*
>> > >
>> > > *security.inter.broker.protocol=SSL*
>> > >
>> > > *#allow.everyone.if.no.acl.found=true*
>> > >
>> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
>> > >
>> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
>> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
>> > >
>> > > *#super.users=User:Raghu;User:Admin*
>> > >
>> > > *#offsets.storage=kafka*
>> > >
>> > > *#dual.commit.enabled=true*
>> > >
>> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
>> > >
>> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
>> http://10.247.195.122:9092
>> > >*
>> > >
>> > > *#listeners=PLAINTEXT://10.247.195.122:9092
>> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
>> > > <http://10.247.195.122:9093>*
>> > >
>> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
>> > > <http://10.247.195.122:9092>*
>> > >
>> > >
>> > > *
>> > > ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
>> > >
>> > > * ssl.keystore.password=123456*
>> > >
>> > > * ssl.key.password=123456*
>> > >
>> > > *
>> > > ssl.truststore.location=/home/raghu/kafka/security/server.
>> > truststore.jks*
>> > >
>> > > * ssl.truststore.password=123456*
>> > >
>> > >
>> > >
>> > > *Set the ACL from Authorizer CLI:*
>> > >
>> > > > *bin/kafka-acls.sh --authorizer-properties
>> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181>
>> > --list
>> > > --topic ssltopic*
>> > >
>> > > *Current ACLs for resource `Topic:ssltopic`: *
>> > >
>> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
>> > > C=Unknown has Allow permission for operations: Write from hosts: * *
>> > >
>> > >
>> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
>> bin/kafka-console-producer.sh
>> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
>> --topic
>> > > ssltopic --producer.config client-ssl.properties*
>> > >
>> > >
>> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
>> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
>> > > (org.apache.kafka.clients.NetworkClient)*
>> > >
>> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
>> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
>> > > (org.apache.kafka.clients.NetworkClient)*
>> > >
>> > >
>> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
>> > >
>> > > *#group.id <http://group.id>=sslgroup*
>> > >
>> > > *security.protocol=SSL*
>> > >
>> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
>> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
>> > >
>> > > *ssl.truststore.password=123456*
>> > >
>> > > * #Configure Below if you use Client Auth*
>> > >
>> > >
>> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
>> > > 11-0.10.1.0/ssl/client.keystore.jks*
>> > >
>> > > *ssl.keystore.password=123456*
>> > >
>> > > *ssl.key.password=123456*
>> > >
>> > >
>> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
>> bin/kafka-console-consumer.sh
>> > > --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
>> > > --new-consumer --consumer.config client-ssl.properties --topic
>> ssltopic
>> > > --from-beginning*
>> > >
>> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
>> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
>> > > (org.apache.kafka.clients.NetworkClient)*
>> > >
>> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
>> > > (kafka.tools.ConsoleConsumer$)*
>> > >
>> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
>> > > authorized to access group: console-consumer-52826*
>> > >
>> > >
>> > > Thanks in advance,
>> > >
>> > > Raghu - raghu98499@gmail.com
>> > > This e-mail and its contents (to include attachments) are the
>> property of
>> > > National Health Systems, Inc., its subsidiaries and affiliates,
>> including
>> > > but not limited to Rx.com Community Healthcare Network, Inc. and its
>> > > subsidiaries, and may contain confidential and proprietary or
>> privileged
>> > > information. If you are not the intended recipient of this e-mail, you
>> > are
>> > > hereby notified that any unauthorized disclosure, copying, or
>> > distribution
>> > > of this e-mail or of its attachments, or the taking of any
>> unauthorized
>> > > action based on information contained herein is strictly prohibited.
>> > > Unauthorized use of information contained herein may subject you to
>> civil
>> > > and criminal prosecution and penalties. If you are not the intended
>> > > recipient, please immediately notify the sender by telephone at
>> > > 800-433-5719 or return e-mail and permanently delete the original
>> > e-mail.
>> > >
>> >
>>
>
>
>
> --
> G.Kiran Kumar
>
--
G.Kiran Kumar
Re: Kafka ACL's with SSL Protocol is not working
Posted by kiran kumar <ki...@gmail.com>.
Hi Raghu,
I am also facing the same issue but with the SASL_PLAINTEXT protocol.
after enabling debugging I see that authentication is being completed. I
don't see any debug logs being generated for authorization part (I might be
missing something).
you can also set the log level to debug in properties and see whats going
on.
Thanks,
Kiran
On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
wrote:
> Make sure that the principal ID is exactly what Kafka sees. Guessing what
> the principal ID is by using keytool or openssl is not going to help from
> my experience. The best is to add some logging to output the SSL client ID
> in the org.apache.kafka.common.network.SslTransportLayer.peerPrincipal() .
> The p.getName() is what you are looking at.
>
> Instead of adding it to the super user list in your server props file, add
> ACLs to that user using the kafka-acls.sh in the bin directory.
>
>
>
> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com> wrote:
>
> > Thanks Shrikant for your reply, but I did consumer part also and more
> over
> > I am not facing this issue only with consumer, I am getting this errors
> > with producer as well as consumer
> >
> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> wrote:
> >
> > > You need to execute kafka-acls.sh with --consumer to enable consumption
> > > from kafka.
> > >
> > > _________________________________________________
> > > Shrikant Patel | 817.367.4302
> > > Enterprise Architecture Team
> > > PDX-NHIN
> > >
> > > -----Original Message-----
> > > From: Raghu B [mailto:raghu98499@gmail.com]
> > > Sent: Wednesday, December 14, 2016 5:42 PM
> > > To: security@kafka.apache.org
> > > Subject: Kafka ACL's with SSL Protocol is not working
> > >
> > > Hi All,
> > >
> > > I am trying to enable ACL's in my Kafka cluster with along with SSL
> > > Protocol.
> > >
> > > I tried with each and every parameters but no luck, so I need help to
> > > enable the SSL(without Kerberos) and I am attaching all the
> configuration
> > > details in this.
> > >
> > > Kindly Help me.
> > >
> > >
> > > *I tested SSL without ACL, it worked fine
> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
> > >
> > >
> > > *This is my Kafka server properties file:*
> > >
> > > *############################# ACL SETTINGS
> > #############################*
> > >
> > > *auto.create.topics.enable=true*
> > >
> > > *authorizer.class.name
> > > <http://authorizer.class.name>=kafka.security.auth.
> SimpleAclAuthorizer*
> > >
> > > *security.inter.broker.protocol=SSL*
> > >
> > > *#allow.everyone.if.no.acl.found=true*
> > >
> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> > >
> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > >
> > > *#super.users=User:Raghu;User:Admin*
> > >
> > > *#offsets.storage=kafka*
> > >
> > > *#dual.commit.enabled=true*
> > >
> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
> > >
> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> http://10.247.195.122:9092
> > >*
> > >
> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > > <http://10.247.195.122:9093>*
> > >
> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > > <http://10.247.195.122:9092>*
> > >
> > >
> > > *
> > > ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
> > >
> > > * ssl.keystore.password=123456*
> > >
> > > * ssl.key.password=123456*
> > >
> > > *
> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> > truststore.jks*
> > >
> > > * ssl.truststore.password=123456*
> > >
> > >
> > >
> > > *Set the ACL from Authorizer CLI:*
> > >
> > > > *bin/kafka-acls.sh --authorizer-properties
> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181>
> > --list
> > > --topic ssltopic*
> > >
> > > *Current ACLs for resource `Topic:ssltopic`: *
> > >
> > > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> > > C=Unknown has Allow permission for operations: Write from hosts: * *
> > >
> > >
> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh
> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic
> > > ssltopic --producer.config client-ssl.properties*
> > >
> > >
> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > (org.apache.kafka.clients.NetworkClient)*
> > >
> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > (org.apache.kafka.clients.NetworkClient)*
> > >
> > >
> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
> > >
> > > *#group.id <http://group.id>=sslgroup*
> > >
> > > *security.protocol=SSL*
> > >
> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> > >
> > > *ssl.truststore.password=123456*
> > >
> > > * #Configure Below if you use Client Auth*
> > >
> > >
> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > > 11-0.10.1.0/ssl/client.keystore.jks*
> > >
> > > *ssl.keystore.password=123456*
> > >
> > > *ssl.key.password=123456*
> > >
> > >
> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh
> > > --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> > > --new-consumer --consumer.config client-ssl.properties --topic ssltopic
> > > --from-beginning*
> > >
> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > > (org.apache.kafka.clients.NetworkClient)*
> > >
> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
> > > (kafka.tools.ConsoleConsumer$)*
> > >
> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> > > authorized to access group: console-consumer-52826*
> > >
> > >
> > > Thanks in advance,
> > >
> > > Raghu - raghu98499@gmail.com
> > > This e-mail and its contents (to include attachments) are the property
> of
> > > National Health Systems, Inc., its subsidiaries and affiliates,
> including
> > > but not limited to Rx.com Community Healthcare Network, Inc. and its
> > > subsidiaries, and may contain confidential and proprietary or
> privileged
> > > information. If you are not the intended recipient of this e-mail, you
> > are
> > > hereby notified that any unauthorized disclosure, copying, or
> > distribution
> > > of this e-mail or of its attachments, or the taking of any unauthorized
> > > action based on information contained herein is strictly prohibited.
> > > Unauthorized use of information contained herein may subject you to
> civil
> > > and criminal prosecution and penalties. If you are not the intended
> > > recipient, please immediately notify the sender by telephone at
> > > 800-433-5719 or return e-mail and permanently delete the original
> > e-mail.
> > >
> >
>
--
G.Kiran Kumar
Re: Kafka ACL's with SSL Protocol is not working
Posted by Derar Alassi <de...@gmail.com>.
Make sure that the principal ID is exactly what Kafka sees. Guessing what
the principal ID is by using keytool or openssl is not going to help from
my experience. The best is to add some logging to output the SSL client ID
in the org.apache.kafka.common.network.SslTransportLayer.peerPrincipal() .
The p.getName() is what you are looking at.
Instead of adding it to the super user list in your server props file, add
ACLs to that user using the kafka-acls.sh in the bin directory.
On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com> wrote:
> Thanks Shrikant for your reply, but I did consumer part also and more over
> I am not facing this issue only with consumer, I am getting this errors
> with producer as well as consumer
>
> On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com> wrote:
>
> > You need to execute kafka-acls.sh with --consumer to enable consumption
> > from kafka.
> >
> > _________________________________________________
> > Shrikant Patel | 817.367.4302
> > Enterprise Architecture Team
> > PDX-NHIN
> >
> > -----Original Message-----
> > From: Raghu B [mailto:raghu98499@gmail.com]
> > Sent: Wednesday, December 14, 2016 5:42 PM
> > To: security@kafka.apache.org
> > Subject: Kafka ACL's with SSL Protocol is not working
> >
> > Hi All,
> >
> > I am trying to enable ACL's in my Kafka cluster with along with SSL
> > Protocol.
> >
> > I tried with each and every parameters but no luck, so I need help to
> > enable the SSL(without Kerberos) and I am attaching all the configuration
> > details in this.
> >
> > Kindly Help me.
> >
> >
> > *I tested SSL without ACL, it worked fine
> > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
> >
> >
> > *This is my Kafka server properties file:*
> >
> > *############################# ACL SETTINGS
> #############################*
> >
> > *auto.create.topics.enable=true*
> >
> > *authorizer.class.name
> > <http://authorizer.class.name>=kafka.security.auth.SimpleAclAuthorizer*
> >
> > *security.inter.broker.protocol=SSL*
> >
> > *#allow.everyone.if.no.acl.found=true*
> >
> > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> >
> > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> >
> > *#super.users=User:Raghu;User:Admin*
> >
> > *#offsets.storage=kafka*
> >
> > *#dual.commit.enabled=true*
> >
> > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
> >
> > *#listeners=PLAINTEXT://10.247.195.122:9092 <http://10.247.195.122:9092
> >*
> >
> > *#listeners=PLAINTEXT://10.247.195.122:9092
> > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > <http://10.247.195.122:9093>*
> >
> > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > <http://10.247.195.122:9092>*
> >
> >
> > *
> > ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
> >
> > * ssl.keystore.password=123456*
> >
> > * ssl.key.password=123456*
> >
> > *
> > ssl.truststore.location=/home/raghu/kafka/security/server.
> truststore.jks*
> >
> > * ssl.truststore.password=123456*
> >
> >
> >
> > *Set the ACL from Authorizer CLI:*
> >
> > > *bin/kafka-acls.sh --authorizer-properties
> > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181>
> --list
> > --topic ssltopic*
> >
> > *Current ACLs for resource `Topic:ssltopic`: *
> >
> > * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> > C=Unknown has Allow permission for operations: Write from hosts: * *
> >
> >
> > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh
> > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic
> > ssltopic --producer.config client-ssl.properties*
> >
> >
> > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > (org.apache.kafka.clients.NetworkClient)*
> >
> > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > (org.apache.kafka.clients.NetworkClient)*
> >
> >
> > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
> >
> > *#group.id <http://group.id>=sslgroup*
> >
> > *security.protocol=SSL*
> >
> > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> >
> > *ssl.truststore.password=123456*
> >
> > * #Configure Below if you use Client Auth*
> >
> >
> > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > 11-0.10.1.0/ssl/client.keystore.jks*
> >
> > *ssl.keystore.password=123456*
> >
> > *ssl.key.password=123456*
> >
> >
> > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh
> > --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> > --new-consumer --consumer.config client-ssl.properties --topic ssltopic
> > --from-beginning*
> >
> > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > (org.apache.kafka.clients.NetworkClient)*
> >
> > *[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
> > (kafka.tools.ConsoleConsumer$)*
> >
> > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> > authorized to access group: console-consumer-52826*
> >
> >
> > Thanks in advance,
> >
> > Raghu - raghu98499@gmail.com
> > This e-mail and its contents (to include attachments) are the property of
> > National Health Systems, Inc., its subsidiaries and affiliates, including
> > but not limited to Rx.com Community Healthcare Network, Inc. and its
> > subsidiaries, and may contain confidential and proprietary or privileged
> > information. If you are not the intended recipient of this e-mail, you
> are
> > hereby notified that any unauthorized disclosure, copying, or
> distribution
> > of this e-mail or of its attachments, or the taking of any unauthorized
> > action based on information contained herein is strictly prohibited.
> > Unauthorized use of information contained herein may subject you to civil
> > and criminal prosecution and penalties. If you are not the intended
> > recipient, please immediately notify the sender by telephone at
> > 800-433-5719 or return e-mail and permanently delete the original
> e-mail.
> >
>
Re: Kafka ACL's with SSL Protocol is not working
Posted by Raghu B <ra...@gmail.com>.
Thanks Shrikant for your reply, but I did consumer part also and more over
I am not facing this issue only with consumer, I am getting this errors
with producer as well as consumer
On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com> wrote:
> You need to execute kafka-acls.sh with --consumer to enable consumption
> from kafka.
>
> _________________________________________________
> Shrikant Patel | 817.367.4302
> Enterprise Architecture Team
> PDX-NHIN
>
> -----Original Message-----
> From: Raghu B [mailto:raghu98499@gmail.com]
> Sent: Wednesday, December 14, 2016 5:42 PM
> To: security@kafka.apache.org
> Subject: Kafka ACL's with SSL Protocol is not working
>
> Hi All,
>
> I am trying to enable ACL's in my Kafka cluster with along with SSL
> Protocol.
>
> I tried with each and every parameters but no luck, so I need help to
> enable the SSL(without Kerberos) and I am attaching all the configuration
> details in this.
>
> Kindly Help me.
>
>
> *I tested SSL without ACL, it worked fine
> (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
>
>
> *This is my Kafka server properties file:*
>
> *############################# ACL SETTINGS #############################*
>
> *auto.create.topics.enable=true*
>
> *authorizer.class.name
> <http://authorizer.class.name>=kafka.security.auth.SimpleAclAuthorizer*
>
> *security.inter.broker.protocol=SSL*
>
> *#allow.everyone.if.no.acl.found=true*
>
> *#principal.builder.class=CustomizedPrincipalBuilderClass*
>
> *#super.users=User:"CN=writeuser,OU=Unknown,O=
> Unknown,L=Unknown,ST=Unknown,C=Unknown"*
>
> *#super.users=User:Raghu;User:Admin*
>
> *#offsets.storage=kafka*
>
> *#dual.commit.enabled=true*
>
> *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
>
> *#listeners=PLAINTEXT://10.247.195.122:9092 <http://10.247.195.122:9092>*
>
> *#listeners=PLAINTEXT://10.247.195.122:9092
> <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> <http://10.247.195.122:9093>*
>
> *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> <http://10.247.195.122:9092>*
>
>
> *
> ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
>
> * ssl.keystore.password=123456*
>
> * ssl.key.password=123456*
>
> *
> ssl.truststore.location=/home/raghu/kafka/security/server.truststore.jks*
>
> * ssl.truststore.password=123456*
>
>
>
> *Set the ACL from Authorizer CLI:*
>
> > *bin/kafka-acls.sh --authorizer-properties
> zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181> --list
> --topic ssltopic*
>
> *Current ACLs for resource `Topic:ssltopic`: *
>
> * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> C=Unknown has Allow permission for operations: Write from hosts: * *
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh
> --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic
> ssltopic --producer.config client-ssl.properties*
>
>
> *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
> *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
>
> *#group.id <http://group.id>=sslgroup*
>
> *security.protocol=SSL*
>
> *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
>
> *ssl.truststore.password=123456*
>
> * #Configure Below if you use Client Auth*
>
>
> *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> 11-0.10.1.0/ssl/client.keystore.jks*
>
> *ssl.keystore.password=123456*
>
> *ssl.key.password=123456*
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh
> --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> --new-consumer --consumer.config client-ssl.properties --topic ssltopic
> --from-beginning*
>
> *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
> *[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
> (kafka.tools.ConsoleConsumer$)*
>
> *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> authorized to access group: console-consumer-52826*
>
>
> Thanks in advance,
>
> Raghu - raghu98499@gmail.com
> This e-mail and its contents (to include attachments) are the property of
> National Health Systems, Inc., its subsidiaries and affiliates, including
> but not limited to Rx.com Community Healthcare Network, Inc. and its
> subsidiaries, and may contain confidential and proprietary or privileged
> information. If you are not the intended recipient of this e-mail, you are
> hereby notified that any unauthorized disclosure, copying, or distribution
> of this e-mail or of its attachments, or the taking of any unauthorized
> action based on information contained herein is strictly prohibited.
> Unauthorized use of information contained herein may subject you to civil
> and criminal prosecution and penalties. If you are not the intended
> recipient, please immediately notify the sender by telephone at
> 800-433-5719 or return e-mail and permanently delete the original e-mail.
>
RE: Kafka ACL's with SSL Protocol is not working
Posted by Shrikant Patel <SP...@pdxinc.com>.
You need to execute kafka-acls.sh with --consumer to enable consumption from kafka.
_________________________________________________
Shrikant Patel | 817.367.4302
Enterprise Architecture Team
PDX-NHIN
-----Original Message-----
From: Raghu B [mailto:raghu98499@gmail.com]
Sent: Wednesday, December 14, 2016 5:42 PM
To: security@kafka.apache.org
Subject: Kafka ACL's with SSL Protocol is not working
Hi All,
I am trying to enable ACL's in my Kafka cluster with along with SSL Protocol.
I tried with each and every parameters but no luck, so I need help to enable the SSL(without Kerberos) and I am attaching all the configuration details in this.
Kindly Help me.
*I tested SSL without ACL, it worked fine
(listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
*This is my Kafka server properties file:*
*############################# ACL SETTINGS #############################*
*auto.create.topics.enable=true*
*authorizer.class.name
<http://authorizer.class.name>=kafka.security.auth.SimpleAclAuthorizer*
*security.inter.broker.protocol=SSL*
*#allow.everyone.if.no.acl.found=true*
*#principal.builder.class=CustomizedPrincipalBuilderClass*
*#super.users=User:"CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
*#super.users=User:Raghu;User:Admin*
*#offsets.storage=kafka*
*#dual.commit.enabled=true*
*listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
*#listeners=PLAINTEXT://10.247.195.122:9092 <http://10.247.195.122:9092>*
*#listeners=PLAINTEXT://10.247.195.122:9092
<http://10.247.195.122:9092>,SSL://10.247.195.122:9093
<http://10.247.195.122:9093>*
*#advertised.listeners=PLAINTEXT://10.247.195.122:9092
<http://10.247.195.122:9092>*
*
ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
* ssl.keystore.password=123456*
* ssl.key.password=123456*
*
ssl.truststore.location=/home/raghu/kafka/security/server.truststore.jks*
* ssl.truststore.password=123456*
*Set the ACL from Authorizer CLI:*
> *bin/kafka-acls.sh --authorizer-properties
zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181> --list --topic ssltopic*
*Current ACLs for resource `Topic:ssltopic`: *
* User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown has Allow permission for operations: Write from hosts: * *
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic ssltopic --producer.config client-ssl.properties*
*[2016-12-13 14:53:45,839] WARN Error while fetching metadata with correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*[2016-12-13 14:53:45,984] WARN Error while fetching metadata with correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
*#group.id <http://group.id>=sslgroup*
*security.protocol=SSL*
*ssl.truststore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
*ssl.truststore.password=123456*
* #Configure Below if you use Client Auth*
*ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.keystore.jks*
*ssl.keystore.password=123456*
*ssl.key.password=123456*
*XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093> --new-consumer --consumer.config client-ssl.properties --topic ssltopic
--from-beginning*
*[2016-12-13 14:53:28,817] WARN Error while fetching metadata with correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)*
*[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
(kafka.tools.ConsoleConsumer$)*
*org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-52826*
Thanks in advance,
Raghu - raghu98499@gmail.com
This e-mail and its contents (to include attachments) are the property of National Health Systems, Inc., its subsidiaries and affiliates, including but not limited to Rx.com Community Healthcare Network, Inc. and its subsidiaries, and may contain confidential and proprietary or privileged information. If you are not the intended recipient of this e-mail, you are hereby notified that any unauthorized disclosure, copying, or distribution of this e-mail or of its attachments, or the taking of any unauthorized action based on information contained herein is strictly prohibited. Unauthorized use of information contained herein may subject you to civil and criminal prosecution and penalties. If you are not the intended recipient, please immediately notify the sender by telephone at 800-433-5719 or return e-mail and permanently delete the original e-mail.
Re: Kafka ACL's with SSL Protocol is not working
Posted by Clebert Suconic <cl...@gmail.com>.
I am afraid you made a mistake.. This is the activemq mail list, not
Kafka.
On Wed, Dec 14, 2016 at 6:42 PM, Raghu B <ra...@gmail.com> wrote:
> Hi All,
>
> I am trying to enable ACL's in my Kafka cluster with along with SSL
> Protocol.
>
> I tried with each and every parameters but no luck, so I need help to
> enable the SSL(without Kerberos) and I am attaching all the configuration
> details in this.
>
> Kindly Help me.
>
>
> *I tested SSL without ACL, it worked fine
> (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>)*
>
>
> *This is my Kafka server properties file:*
>
> *############################# ACL SETTINGS #############################*
>
> *auto.create.topics.enable=true*
>
> *authorizer.class.name
> <http://authorizer.class.name>=kafka.security.auth.SimpleAclAuthorizer*
>
> *security.inter.broker.protocol=SSL*
>
> *#allow.everyone.if.no.acl.found=true*
>
> *#principal.builder.class=CustomizedPrincipalBuilderClass*
>
>
*#super.users=User:"CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
>
> *#super.users=User:Raghu;User:Admin*
>
> *#offsets.storage=kafka*
>
> *#dual.commit.enabled=true*
>
> *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093>*
>
> *#listeners=PLAINTEXT://10.247.195.122:9092 <http://10.247.195.122:9092>*
>
> *#listeners=PLAINTEXT://10.247.195.122:9092
> <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> <http://10.247.195.122:9093>*
>
> *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> <http://10.247.195.122:9092>*
>
>
> *
> ssl.keystore.location=/home/raghu/kafka/security/server.keystore.jks*
>
> * ssl.keystore.password=123456*
>
> * ssl.key.password=123456*
>
> *
> ssl.truststore.location=/home/raghu/kafka/security/server.truststore.jks*
>
> * ssl.truststore.password=123456*
>
>
>
> *Set the ACL from Authorizer CLI:*
>
>> *bin/kafka-acls.sh --authorizer-properties
> zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181> --list
> --topic ssltopic*
>
> *Current ACLs for resource `Topic:ssltopic`: *
>
> * User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown,
> C=Unknown has Allow permission for operations: Write from hosts: * *
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-producer.sh
> --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093> --topic
> ssltopic --producer.config client-ssl.properties*
>
>
> *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
> *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat client-ssl.properties*
>
> *#group.id <http://group.id>=sslgroup*
>
> *security.protocol=SSL*
>
>
*ssl.truststore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
>
> *ssl.truststore.password=123456*
>
> * #Configure Below if you use Client Auth*
>
>
>
*ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.11-0.10.1.0/ssl/client.keystore.jks*
>
> *ssl.keystore.password=123456*
>
> *ssl.key.password=123456*
>
>
> *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ bin/kafka-console-consumer.sh
> --bootstrap-server 10.247.195.122:9093 <http://10.247.195.122:9093>
> --new-consumer --consumer.config client-ssl.properties --topic ssltopic
> --from-beginning*
>
> *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)*
>
> *[2016-12-13 14:53:28,819] ERROR Unknown error when running consumer:
> (kafka.tools.ConsoleConsumer$)*
>
> *org.apache.kafka.common.errors.GroupAuthorizationException: Not
authorized
> to access group: console-consumer-52826*
>
>
> Thanks in advance,
>
> Raghu - raghu98499@gmail.com
--
Clebert Suconic