Re: Kafka ACL's with SSL Protocol is not working

[Raghav <ra...@gmail.com>]

Hi

I have very similar problem that Raghu faced.

How can I enable the log4j debug mode ?

Thanks.

On Thu, Dec 15, 2016 at 3:32 PM, Raghu B <ra...@gmail.com> wrote:

> Thanks Derar & Kiran, your suggestions are very useful.
>
> I enabled Log4J debug mode and found that my client is trying to connect to
> the Kafka server with the *User:ANONYMOUS, *It is really strange.
>
>
> I added a new Super.User with the name *User:ANONYMOUS *then I am able to
> send and receive the messages without any issues.
>
> And now the question is how can I set my username name from Anonymous to
> something like
> *User:"CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> which
> comes from SSL cert/keystore.
>
> Please help me with your inputs.
>
> Thanks in Advance,
> Raghu
>
> On Thu, Dec 15, 2016 at 5:29 AM, kiran kumar <ki...@gmail.com>
> wrote:
>
> > I have just noticed that I am using the user which is not configured in
> the
> > kafka server jaas config file..
> >
> >
> >
> > On Thu, Dec 15, 2016 at 6:38 PM, kiran kumar <ki...@gmail.com>
> > wrote:
> >
> > > Hi Raghu,
> > >
> > > I am also facing the same issue but with the SASL_PLAINTEXT protocol.
> > >
> > > after enabling debugging I see that authentication is being completed.
> I
> > > don't see any debug logs being generated for authorization part (I
> might
> > be
> > > missing something).
> > >
> > > you can also set the log level to debug in properties and see whats
> going
> > > on.
> > >
> > > Thanks,
> > > Kiran
> > >
> > > On Thu, Dec 15, 2016 at 7:09 AM, Derar Alassi <de...@gmail.com>
> > > wrote:
> > >
> > >> Make sure that the principal ID is exactly what Kafka sees. Guessing
> > what
> > >> the principal ID is by using keytool or openssl is not going to help
> > from
> > >> my experience. The best is to add some logging to output the SSL
> client
> > ID
> > >> in the org.apache.kafka.common.network.SslTransportLayer.
> > peerPrincipal()
> > >> .
> > >> The p.getName() is what you are looking at.
> > >>
> > >> Instead of adding it to the super user list in your server props file,
> > add
> > >> ACLs to that user using the kafka-acls.sh in the bin directory.
> > >>
> > >>
> > >>
> > >> On Wed, Dec 14, 2016 at 3:57 PM, Raghu B <ra...@gmail.com>
> wrote:
> > >>
> > >> > Thanks Shrikant for your reply, but I did consumer part also and
> more
> > >> over
> > >> > I am not facing this issue only with consumer, I am getting this
> > errors
> > >> > with producer as well as consumer
> > >> >
> > >> > On Wed, Dec 14, 2016 at 3:53 PM, Shrikant Patel <SP...@pdxinc.com>
> > >> wrote:
> > >> >
> > >> > > You need to execute kafka-acls.sh with --consumer to enable
> > >> consumption
> > >> > > from kafka.
> > >> > >
> > >> > > _________________________________________________
> > >> > > Shrikant Patel  |  817.367.4302
> > >> > > Enterprise Architecture Team
> > >> > > PDX-NHIN
> > >> > >
> > >> > > -----Original Message-----
> > >> > > From: Raghu B [mailto:raghu98499@gmail.com]
> > >> > > Sent: Wednesday, December 14, 2016 5:42 PM
> > >> > > To: security@kafka.apache.org
> > >> > > Subject: Kafka ACL's with SSL Protocol is not working
> > >> > >
> > >> > > Hi All,
> > >> > >
> > >> > > I am trying to enable ACL's in my Kafka cluster with along with
> SSL
> > >> > > Protocol.
> > >> > >
> > >> > > I tried with each and every parameters but no luck, so I need help
> > to
> > >> > > enable the SSL(without Kerberos) and I am attaching all the
> > >> configuration
> > >> > > details in this.
> > >> > >
> > >> > > Kindly Help me.
> > >> > >
> > >> > >
> > >> > > *I tested SSL without ACL, it worked fine
> > >> > > (listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >)*
> > >> > >
> > >> > >
> > >> > > *This is my Kafka server properties file:*
> > >> > >
> > >> > > *############################# ACL SETTINGS
> > >> > #############################*
> > >> > >
> > >> > > *auto.create.topics.enable=true*
> > >> > >
> > >> > > *authorizer.class.name
> > >> > > <http://authorizer.class.name>=kafka.security.auth.SimpleAcl
> > >> Authorizer*
> > >> > >
> > >> > > *security.inter.broker.protocol=SSL*
> > >> > >
> > >> > > *#allow.everyone.if.no.acl.found=true*
> > >> > >
> > >> > > *#principal.builder.class=CustomizedPrincipalBuilderClass*
> > >> > >
> > >> > > *#super.users=User:"CN=writeuser,OU=Unknown,O=
> > >> > > Unknown,L=Unknown,ST=Unknown,C=Unknown"*
> > >> > >
> > >> > > *#super.users=User:Raghu;User:Admin*
> > >> > >
> > >> > > *#offsets.storage=kafka*
> > >> > >
> > >> > > *#dual.commit.enabled=true*
> > >> > >
> > >> > > *listeners=SSL://10.247.195.122:9093 <http://10.247.195.122:9093
> >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092 <
> > >> http://10.247.195.122:9092
> > >> > >*
> > >> > >
> > >> > > *#listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>,SSL://10.247.195.122:9093
> > >> > > <http://10.247.195.122:9093>*
> > >> > >
> > >> > > *#advertised.listeners=PLAINTEXT://10.247.195.122:9092
> > >> > > <http://10.247.195.122:9092>*
> > >> > >
> > >> > >
> > >> > > *
> > >> > > ssl.keystore.location=/home/raghu/kafka/security/server.
> > keystore.jks*
> > >> > >
> > >> > > *        ssl.keystore.password=123456*
> > >> > >
> > >> > > *        ssl.key.password=123456*
> > >> > >
> > >> > > *
> > >> > > ssl.truststore.location=/home/raghu/kafka/security/server.
> > >> > truststore.jks*
> > >> > >
> > >> > > *        ssl.truststore.password=123456*
> > >> > >
> > >> > >
> > >> > >
> > >> > > *Set the ACL from Authorizer CLI:*
> > >> > >
> > >> > > > *bin/kafka-acls.sh --authorizer-properties
> > >> > > zookeeper.connect=10.247.195.122:2181 <http://10.247.195.122:2181
> >
> > >> > --list
> > >> > > --topic ssltopic*
> > >> > >
> > >> > > *Current ACLs for resource `Topic:ssltopic`: *
> > >> > >
> > >> > > *  User:CN=writeuser, OU=Unknown, O=Unknown, L=Unknown,
> ST=Unknown,
> > >> > > C=Unknown has Allow permission for operations: Write from hosts:
> * *
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-producer.sh
> > >> > > --broker-list 10.247.195.122:9093 <http://10.247.195.122:9093>
> > >> --topic
> > >> > > ssltopic --producer.config client-ssl.properties*
> > >> > >
> > >> > >
> > >> > > *[2016-12-13 14:53:45,839] WARN Error while fetching metadata with
> > >> > > correlation id 0 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:45,984] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$ cat
> client-ssl.properties*
> > >> > >
> > >> > > *#group.id <http://group.id>=sslgroup*
> > >> > >
> > >> > > *security.protocol=SSL*
> > >> > >
> > >> > > *ssl.truststore.location=/Users/rbaddam/Desktop/Dev/
> > >> > > kafka_2.11-0.10.1.0/ssl/client.truststore.jks*
> > >> > >
> > >> > > *ssl.truststore.password=123456*
> > >> > >
> > >> > > * #Configure Below if you use Client Auth*
> > >> > >
> > >> > >
> > >> > > *ssl.keystore.location=/Users/rbaddam/Desktop/Dev/kafka_2.
> > >> > > 11-0.10.1.0/ssl/client.keystore.jks*
> > >> > >
> > >> > > *ssl.keystore.password=123456*
> > >> > >
> > >> > > *ssl.key.password=123456*
> > >> > >
> > >> > >
> > >> > > *XXXWMXXX-7:kafka_2.11-0.10.1.0 rbaddam$
> > >> bin/kafka-console-consumer.sh
> > >> > > --bootstrap-server 10.247.195.122:9093 <
> http://10.247.195.122:9093>
> > >> > > --new-consumer --consumer.config client-ssl.properties --topic
> > >> ssltopic
> > >> > > --from-beginning*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,817] WARN Error while fetching metadata with
> > >> > > correlation id 1 : {ssltopic=UNKNOWN_TOPIC_OR_PARTITION}
> > >> > > (org.apache.kafka.clients.NetworkClient)*
> > >> > >
> > >> > > *[2016-12-13 14:53:28,819] ERROR Unknown error when running
> > consumer:
> > >> > > (kafka.tools.ConsoleConsumer$)*
> > >> > >
> > >> > > *org.apache.kafka.common.errors.GroupAuthorizationException: Not
> > >> > > authorized to access group: console-consumer-52826*
> > >> > >
> > >> > >
> > >> > > Thanks in advance,
> > >> > >
> > >> > > Raghu - raghu98499@gmail.com
> > >> > > This e-mail and its contents (to include attachments) are the
> > >> property of
> > >> > > National Health Systems, Inc., its subsidiaries and affiliates,
> > >> including
> > >> > > but not limited to Rx.com Community Healthcare Network, Inc. and
> its
> > >> > > subsidiaries, and may contain confidential and proprietary or
> > >> privileged
> > >> > > information. If you are not the intended recipient of this e-mail,
> > you
> > >> > are
> > >> > > hereby notified that any unauthorized disclosure, copying, or
> > >> > distribution
> > >> > > of this e-mail or of its attachments, or the taking of any
> > >> unauthorized
> > >> > > action based on information contained herein is strictly
> prohibited.
> > >> > > Unauthorized use of information contained herein may subject you
> to
> > >> civil
> > >> > > and criminal prosecution and penalties. If you are not the
> intended
> > >> > > recipient, please immediately notify the sender by telephone at
> > >> > > 800-433-5719 or return e-mail and permanently delete the original
> > >> > e-mail.
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > G.Kiran Kumar
> > >
> >
> >
> >
> > --
> > G.Kiran Kumar
> >
>



-- 
Raghav