You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Manoj Murumkar <ma...@gmail.com> on 2017/08/30 21:19:43 UTC
Using ACLs without Kerberos
Hi,
We are evaluating how to put authorization in place for Kafka (around
topics, mostly). Is it a good idea to do this without Kerberos? I was
testing if a non-admin principal (OS user) can modify (add/remove) ACLs and
it seems like it's possible. If this is right behavior, it's insecure and
unusable. What do you guys think?
Thanks,
Manoj
Re: Re: Using ACLs without Kerberos
Posted by Manoj Murumkar <ma...@gmail.com>.
This is fixed as well, btw. My ZK config wasn't done properly.
On Tue, Sep 5, 2017 at 11:08 PM, Manoj Murumkar <ma...@gmail.com>
wrote:
> Thanks, SASL setup for ZK does the trick.
>
> Now, I am continuing the security setup with schema registry. I am
> configuring it to use same PLAINTEXT method for now.
>
> listeners=http://0.0.0.0:8081
> kafkastore.connection.url=localhost:2181
> kafkastore.topic=_schemas
> debug=true
> # Security specific params
> kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092
> #kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_
> PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT://
> or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092
> kafkastore.init.timeout.ms=10000
> kafkastore.security.protocol=SASL_PLAINTEXT
> zookeeper.set.acl=true
> kafkastore.sasl.mechanism=PLAIN
> kafkastore.group.id=schema-registry-consumer-group
>
> I have followed instructions (modified for my use case and latest version
> of confluent) from here: https://www.confluent.io/blog/
> securing-confluent-schema-registry-apache-kafka/
>
> When I try to start schema registry, it fails with following error:
>
> [2017-09-06 06:05:23,790] ERROR Server died unexpectedly:
> (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52)
> java.lang.IllegalArgumentException: Unable to subscribe to the Kafka
> topic _schemas backing this data store. Topic may not exist.
> at io.confluent.kafka.schemaregistry.storage.
> KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130)
>
>
> However, I have enabled schema registry user to produce/consume messages
> for _schemas topic and authorizer debug messages show the same:
>
> [2017-09-06 06:05:21,784] DEBUG operation = Read on resource =
> Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s
> chemaregistry has Allow permission for operations: All from hosts: *
> (kafka.authorizer.logger)
> [2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed
> Operation = Describe from host = 127.0.0.1 on resource
> = Topic:_schemas (kafka.authorizer.logger)
> [2017-09-06 06:05:22,788] DEBUG operation = Read on resource =
> Topic:_schemas from host = 127.0.0.1 is Allow based on acl =
> User:schemaregistry has Allow permission for operations: All from hosts: *
> (kafka.authorizer.logger)
> [2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed
> Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas
> (kafka.authorizer.logger)
>
> Any ideas of what might be happening here?
>
> Here're contents of my JAAS file that I use in the startup script:
>
>
> [root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start
> exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS}
> -Djava.security.auth.login.config=/etc/schema-registry/
> schema_registry_client_jaas.properties io.confluent.kafka.
> schemaregistry.rest.SchemaRegistryMain "$@"
>
>
> [root@or1010050208015 kafka]# cat /etc/schema-registry/schema_
> registry_client_jaas.properties
> /* Kafka Client */
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="schemaregistry"
> password="schemaRegistry";
> };
>
> /* Zookeeper client */
> Client {
> org.apache.zookeeper.server.auth.DigestLoginModule required
> username="zk_admin"
> password="zk_admin_secret";
> };
>
>
> On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <SP...@pdxinc.com> wrote:
>
>> Yes, as per my understanding for now the only way to secure ZK is SASL.
>> https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK
>> could also be secured using SSL.
>>
>> Also remember not use any OS user, anyone user on n\w who can connect to
>> ZK host:port will be able to modify the ACLs.
>>
>> - Shri
>>
>> -----Original Message-----
>> From: Manoj Murumkar [mailto:manoj.murumkar@gmail.com]
>> Sent: Thursday, August 31, 2017 11:03 AM
>> To: users@kafka.apache.org
>> Subject: [EXTERNAL] Re: Using ACLs without Kerberos
>>
>> ***** Notice: This email was received from an external source *****
>>
>> >> Current kafka-acls.sh script directly contacts zookeeper to create
>> ACLs.
>> Any OS user who got access to zookeeper can create ACLs for any Kafka
>> principal.
>>
>> Thanks for that point. Appreciate it.
>>
>> On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <ma...@gmail.com>
>> wrote:
>>
>> > There is no correlation between OS user and Kafka Principal/Username.
>> > Here user name refers to the principal associated with the kafka
>> > communication channel (Kerberos Principal, SASL/Plain username, Scram
>> > username, SSL
>> > certificate)
>> >
>> > Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
>> > Any OS user who got access to zookeeper can create ACLs for any Kafka
>> > principal.
>> > This can be restricted by enabling Zk Kerberos authentication.
>> > http://kafka.apache.org/documentation/#zk_authz
>> >
>> > From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
>> >
>> > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar
>> > <ma...@gmail.com>
>> > wrote:
>> >
>> > > Right, I am. Just to be clear, I am using kafka-acl script to
>> > define/remove
>> > > ACLs as a non-super user and it just works fine. I had expected it
>> > > to
>> > work
>> > > only for super users and not for regular users ('nex37045' is a
>> > > normal user).
>> > >
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
>> > > `Topic:test`:
>> > > User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > > User:skumarmu has Allow permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > > [nex37045@or1010051029033 ~]$
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --remove --allow-principal
>> > User:skumarmu
>> > > --operation Write --topic test
>> > > Are you sure you want to remove ACLs:
>> > > User:skumarmu has Allow permission for operations: Write from
>> > hosts: *
>> > > from resource `Topic:test`? (y/n)
>> > > y
>> > > Current ACLs for resource `Topic:test`:
>> > > User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
>> > > `Topic:test`:
>> > > User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > >
>> > >
>> > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar
>> > > <ma...@gmail.com>
>> > > wrote:
>> > >
>> > > > Looks like you are already using SASL/PLAIN mechanism. Kafka
>> > > > supports
>> > > SASL
>> > > > authentication framework.
>> > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms.
>> > > > you
>> > can
>> > > > enable SSL encryption also
>> > > >
>> > > > http://kafka.apache.org/documentation.html#security
>> > > >
>> > > >
>> > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
>> > > manoj.murumkar@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Thanks Manikumar. I am testing the setup documented here:
>> > > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
>> > acls-in-practice/
>> > > > > (SASL_PLAINTEXT).
>> > > > >
>> > > > > I haven't setup any authentication for the tests. Thinking about
>> > > > > it, authentication is a must have for authorization (so, kafka
>> > > > > knows
>> > who's
>> > > > > making resource request), right?
>> > > > >
>> > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
>> > manikumar.reddy@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Kafka default authorizer is used with secure authenticated
>> > > > > > channels (SSL,SASL,SCRAM).
>> > > > > > For plain text (non-secure) channels, the principal will be
>> > > > > > always ANONYMOUS. Here you can authorize by ip-address.
>> > > > > >
>> > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN
>> > > > > > or
>> > > SCRAM
>> > > > > > mechanisms with/without SSL.
>> > > > > > Pl, check Kafka docs for security considerations.
>> > > > > >
>> > > > > > >> I was testing if a non-admin principal (OS user) can modify
>> > > > > (add/remove)
>> > > > > > ACLs and
>> > > > > > it seems like it's possible.
>> > > > > > How are you trying? Can you give more details about your setup?
>> > What
>> > > > > > authentication mechanism, etc.?
>> > > > > >
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
>> > > > > manoj.murumkar@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > We are evaluating how to put authorization in place for
>> > > > > > > Kafka
>> > > (around
>> > > > > > > topics, mostly). Is it a good idea to do this without
>> > > > > > > Kerberos? I
>> > > was
>> > > > > > > testing if a non-admin principal (OS user) can modify
>> > (add/remove)
>> > > > ACLs
>> > > > > > and
>> > > > > > > it seems like it's possible. If this is right behavior, it's
>> > > insecure
>> > > > > and
>> > > > > > > unusable. What do you guys think?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > >
>> > > > > > > Manoj
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> This e-mail and its contents (to include attachments) are the property of
>> National Health Systems, Inc., its subsidiaries and affiliates, including
>> but not limited to Rx.com Community Healthcare Network, Inc. and its
>> subsidiaries, and may contain confidential and proprietary or privileged
>> information. If you are not the intended recipient of this e-mail, you are
>> hereby notified that any unauthorized disclosure, copying, or distribution
>> of this e-mail or of its attachments, or the taking of any unauthorized
>> action based on information contained herein is strictly prohibited.
>> Unauthorized use of information contained herein may subject you to civil
>> and criminal prosecution and penalties. If you are not the intended
>> recipient, please immediately notify the sender by telephone at
>> 800-433-5719 or return e-mail and permanently delete the original e-mail.
>>
>
>
Re: Re: Using ACLs without Kerberos
Posted by Manoj Murumkar <ma...@gmail.com>.
Thanks, SASL setup for ZK does the trick.
Now, I am continuing the security setup with schema registry. I am
configuring it to use same PLAINTEXT method for now.
listeners=http://0.0.0.0:8081
kafkastore.connection.url=localhost:2181
kafkastore.topic=_schemas
debug=true
# Security specific params
kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092
#kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT://or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092
kafkastore.init.timeout.ms=10000
kafkastore.security.protocol=SASL_PLAINTEXT
zookeeper.set.acl=true
kafkastore.sasl.mechanism=PLAIN
kafkastore.group.id=schema-registry-consumer-group
I have followed instructions (modified for my use case and latest version
of confluent) from here:
https://www.confluent.io/blog/securing-confluent-schema-registry-apache-kafka/
When I try to start schema registry, it fails with following error:
[2017-09-06 06:05:23,790] ERROR Server died unexpectedly:
(io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52)
java.lang.IllegalArgumentException: Unable to subscribe to the Kafka topic
_schemas backing this data store. Topic may not exist.
at
io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130)
However, I have enabled schema registry user to produce/consume messages
for _schemas topic and authorizer debug messages show the same:
[2017-09-06 06:05:21,784] DEBUG operation = Read on resource =
Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s
chemaregistry has Allow permission for operations: All from hosts: *
(kafka.authorizer.logger)
[2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed
Operation = Describe from host = 127.0.0.1 on resource
= Topic:_schemas (kafka.authorizer.logger)
[2017-09-06 06:05:22,788] DEBUG operation = Read on resource =
Topic:_schemas from host = 127.0.0.1 is Allow based on acl =
User:schemaregistry has Allow permission for operations: All from hosts: *
(kafka.authorizer.logger)
[2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed
Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas
(kafka.authorizer.logger)
Any ideas of what might be happening here?
Here're contents of my JAAS file that I use in the startup script:
[root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start
exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS}
-Djava.security.auth.login.config=/etc/schema-registry/schema_registry_client_jaas.properties
io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain "$@"
[root@or1010050208015 kafka]# cat
/etc/schema-registry/schema_registry_client_jaas.properties
/* Kafka Client */
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="schemaregistry"
password="schemaRegistry";
};
/* Zookeeper client */
Client {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="zk_admin"
password="zk_admin_secret";
};
On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <SP...@pdxinc.com> wrote:
> Yes, as per my understanding for now the only way to secure ZK is SASL.
> https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK
> could also be secured using SSL.
>
> Also remember not use any OS user, anyone user on n\w who can connect to
> ZK host:port will be able to modify the ACLs.
>
> - Shri
>
> -----Original Message-----
> From: Manoj Murumkar [mailto:manoj.murumkar@gmail.com]
> Sent: Thursday, August 31, 2017 11:03 AM
> To: users@kafka.apache.org
> Subject: [EXTERNAL] Re: Using ACLs without Kerberos
>
> ***** Notice: This email was received from an external source *****
>
> >> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> Any OS user who got access to zookeeper can create ACLs for any Kafka
> principal.
>
> Thanks for that point. Appreciate it.
>
> On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > There is no correlation between OS user and Kafka Principal/Username.
> > Here user name refers to the principal associated with the kafka
> > communication channel (Kerberos Principal, SASL/Plain username, Scram
> > username, SSL
> > certificate)
> >
> > Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> > Any OS user who got access to zookeeper can create ACLs for any Kafka
> > principal.
> > This can be restricted by enabling Zk Kerberos authentication.
> > http://kafka.apache.org/documentation/#zk_authz
> >
> > From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
> >
> > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar
> > <ma...@gmail.com>
> > wrote:
> >
> > > Right, I am. Just to be clear, I am using kafka-acl script to
> > define/remove
> > > ACLs as a non-super user and it just works fine. I had expected it
> > > to
> > work
> > > only for super users and not for regular users ('nex37045' is a
> > > normal user).
> > >
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > > `Topic:test`:
> > > User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > > User:skumarmu has Allow permission for operations: Write from
> hosts:
> > *
> > >
> > > [nex37045@or1010051029033 ~]$
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --remove --allow-principal
> > User:skumarmu
> > > --operation Write --topic test
> > > Are you sure you want to remove ACLs:
> > > User:skumarmu has Allow permission for operations: Write from
> > hosts: *
> > > from resource `Topic:test`? (y/n)
> > > y
> > > Current ACLs for resource `Topic:test`:
> > > User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > >
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > > `Topic:test`:
> > > User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > >
> > >
> > >
> > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar
> > > <ma...@gmail.com>
> > > wrote:
> > >
> > > > Looks like you are already using SASL/PLAIN mechanism. Kafka
> > > > supports
> > > SASL
> > > > authentication framework.
> > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms.
> > > > you
> > can
> > > > enable SSL encryption also
> > > >
> > > > http://kafka.apache.org/documentation.html#security
> > > >
> > > >
> > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
> > > manoj.murumkar@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks Manikumar. I am testing the setup documented here:
> > > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
> > acls-in-practice/
> > > > > (SASL_PLAINTEXT).
> > > > >
> > > > > I haven't setup any authentication for the tests. Thinking about
> > > > > it, authentication is a must have for authorization (so, kafka
> > > > > knows
> > who's
> > > > > making resource request), right?
> > > > >
> > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
> > manikumar.reddy@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Kafka default authorizer is used with secure authenticated
> > > > > > channels (SSL,SASL,SCRAM).
> > > > > > For plain text (non-secure) channels, the principal will be
> > > > > > always ANONYMOUS. Here you can authorize by ip-address.
> > > > > >
> > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN
> > > > > > or
> > > SCRAM
> > > > > > mechanisms with/without SSL.
> > > > > > Pl, check Kafka docs for security considerations.
> > > > > >
> > > > > > >> I was testing if a non-admin principal (OS user) can modify
> > > > > (add/remove)
> > > > > > ACLs and
> > > > > > it seems like it's possible.
> > > > > > How are you trying? Can you give more details about your setup?
> > What
> > > > > > authentication mechanism, etc.?
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > >
> > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > > > > manoj.murumkar@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We are evaluating how to put authorization in place for
> > > > > > > Kafka
> > > (around
> > > > > > > topics, mostly). Is it a good idea to do this without
> > > > > > > Kerberos? I
> > > was
> > > > > > > testing if a non-admin principal (OS user) can modify
> > (add/remove)
> > > > ACLs
> > > > > > and
> > > > > > > it seems like it's possible. If this is right behavior, it's
> > > insecure
> > > > > and
> > > > > > > unusable. What do you guys think?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Manoj
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> This e-mail and its contents (to include attachments) are the property of
> National Health Systems, Inc., its subsidiaries and affiliates, including
> but not limited to Rx.com Community Healthcare Network, Inc. and its
> subsidiaries, and may contain confidential and proprietary or privileged
> information. If you are not the intended recipient of this e-mail, you are
> hereby notified that any unauthorized disclosure, copying, or distribution
> of this e-mail or of its attachments, or the taking of any unauthorized
> action based on information contained herein is strictly prohibited.
> Unauthorized use of information contained herein may subject you to civil
> and criminal prosecution and penalties. If you are not the intended
> recipient, please immediately notify the sender by telephone at
> 800-433-5719 or return e-mail and permanently delete the original e-mail.
>
RE: Re: Using ACLs without Kerberos
Posted by Shrikant Patel <SP...@pdxinc.com>.
Yes, as per my understanding for now the only way to secure ZK is SASL. https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK could also be secured using SSL.
Also remember not use any OS user, anyone user on n\w who can connect to ZK host:port will be able to modify the ACLs.
- Shri
-----Original Message-----
From: Manoj Murumkar [mailto:manoj.murumkar@gmail.com]
Sent: Thursday, August 31, 2017 11:03 AM
To: users@kafka.apache.org
Subject: [EXTERNAL] Re: Using ACLs without Kerberos
***** Notice: This email was received from an external source *****
>> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
Any OS user who got access to zookeeper can create ACLs for any Kafka principal.
Thanks for that point. Appreciate it.
On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <ma...@gmail.com>
wrote:
> There is no correlation between OS user and Kafka Principal/Username.
> Here user name refers to the principal associated with the kafka
> communication channel (Kerberos Principal, SASL/Plain username, Scram
> username, SSL
> certificate)
>
> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> Any OS user who got access to zookeeper can create ACLs for any Kafka
> principal.
> This can be restricted by enabling Zk Kerberos authentication.
> http://kafka.apache.org/documentation/#zk_authz
>
> From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
>
> On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar
> <ma...@gmail.com>
> wrote:
>
> > Right, I am. Just to be clear, I am using kafka-acl script to
> define/remove
> > ACLs as a non-super user and it just works fine. I had expected it
> > to
> work
> > only for super users and not for regular users ('nex37045' is a
> > normal user).
> >
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> > User:skumarmu has Allow permission for operations: Write from hosts:
> *
> >
> > [nex37045@or1010051029033 ~]$
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --remove --allow-principal
> User:skumarmu
> > --operation Write --topic test
> > Are you sure you want to remove ACLs:
> > User:skumarmu has Allow permission for operations: Write from
> hosts: *
> > from resource `Topic:test`? (y/n)
> > y
> > Current ACLs for resource `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> >
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> >
> >
> >
> > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar
> > <ma...@gmail.com>
> > wrote:
> >
> > > Looks like you are already using SASL/PLAIN mechanism. Kafka
> > > supports
> > SASL
> > > authentication framework.
> > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms.
> > > you
> can
> > > enable SSL encryption also
> > >
> > > http://kafka.apache.org/documentation.html#security
> > >
> > >
> > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
> > manoj.murumkar@gmail.com>
> > > wrote:
> > >
> > > > Thanks Manikumar. I am testing the setup documented here:
> > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
> acls-in-practice/
> > > > (SASL_PLAINTEXT).
> > > >
> > > > I haven't setup any authentication for the tests. Thinking about
> > > > it, authentication is a must have for authorization (so, kafka
> > > > knows
> who's
> > > > making resource request), right?
> > > >
> > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
> manikumar.reddy@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Kafka default authorizer is used with secure authenticated
> > > > > channels (SSL,SASL,SCRAM).
> > > > > For plain text (non-secure) channels, the principal will be
> > > > > always ANONYMOUS. Here you can authorize by ip-address.
> > > > >
> > > > > It's adviced to run on secure channels. you can try SASL/PLAIN
> > > > > or
> > SCRAM
> > > > > mechanisms with/without SSL.
> > > > > Pl, check Kafka docs for security considerations.
> > > > >
> > > > > >> I was testing if a non-admin principal (OS user) can modify
> > > > (add/remove)
> > > > > ACLs and
> > > > > it seems like it's possible.
> > > > > How are you trying? Can you give more details about your setup?
> What
> > > > > authentication mechanism, etc.?
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > > > manoj.murumkar@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We are evaluating how to put authorization in place for
> > > > > > Kafka
> > (around
> > > > > > topics, mostly). Is it a good idea to do this without
> > > > > > Kerberos? I
> > was
> > > > > > testing if a non-admin principal (OS user) can modify
> (add/remove)
> > > ACLs
> > > > > and
> > > > > > it seems like it's possible. If this is right behavior, it's
> > insecure
> > > > and
> > > > > > unusable. What do you guys think?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Manoj
> > > > > >
> > > > >
> > > >
> > >
> >
>
This e-mail and its contents (to include attachments) are the property of National Health Systems, Inc., its subsidiaries and affiliates, including but not limited to Rx.com Community Healthcare Network, Inc. and its subsidiaries, and may contain confidential and proprietary or privileged information. If you are not the intended recipient of this e-mail, you are hereby notified that any unauthorized disclosure, copying, or distribution of this e-mail or of its attachments, or the taking of any unauthorized action based on information contained herein is strictly prohibited. Unauthorized use of information contained herein may subject you to civil and criminal prosecution and penalties. If you are not the intended recipient, please immediately notify the sender by telephone at 800-433-5719 or return e-mail and permanently delete the original e-mail.
Re: Using ACLs without Kerberos
Posted by Manoj Murumkar <ma...@gmail.com>.
>> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
Any OS user who got access to zookeeper can create ACLs for any Kafka
principal.
Thanks for that point. Appreciate it.
On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <ma...@gmail.com>
wrote:
> There is no correlation between OS user and Kafka Principal/Username.
> Here user name refers to the principal associated with the kafka
> communication
> channel (Kerberos Principal, SASL/Plain username, Scram username, SSL
> certificate)
>
> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> Any OS user who got access to zookeeper can create ACLs for any Kafka
> principal.
> This can be restricted by enabling Zk Kerberos authentication.
> http://kafka.apache.org/documentation/#zk_authz
>
> From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
>
> On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar <ma...@gmail.com>
> wrote:
>
> > Right, I am. Just to be clear, I am using kafka-acl script to
> define/remove
> > ACLs as a non-super user and it just works fine. I had expected it to
> work
> > only for super users and not for regular users ('nex37045' is a normal
> > user).
> >
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --list
> > Current ACLs for resource `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> > User:skumarmu has Allow permission for operations: Write from hosts:
> *
> >
> > [nex37045@or1010051029033 ~]$
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --remove --allow-principal
> User:skumarmu
> > --operation Write --topic test
> > Are you sure you want to remove ACLs:
> > User:skumarmu has Allow permission for operations: Write from
> hosts: *
> > from resource `Topic:test`? (y/n)
> > y
> > Current ACLs for resource `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> >
> > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > zookeeper.connect=localhost:2181 --list
> > Current ACLs for resource `Topic:test`:
> > User:nex37045 has Deny permission for operations: Write from hosts:
> *
> >
> >
> >
> > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Looks like you are already using SASL/PLAIN mechanism. Kafka supports
> > SASL
> > > authentication framework.
> > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. you
> can
> > > enable SSL encryption also
> > >
> > > http://kafka.apache.org/documentation.html#security
> > >
> > >
> > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
> > manoj.murumkar@gmail.com>
> > > wrote:
> > >
> > > > Thanks Manikumar. I am testing the setup documented here:
> > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
> acls-in-practice/
> > > > (SASL_PLAINTEXT).
> > > >
> > > > I haven't setup any authentication for the tests. Thinking about it,
> > > > authentication is a must have for authorization (so, kafka knows
> who's
> > > > making resource request), right?
> > > >
> > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
> manikumar.reddy@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Kafka default authorizer is used with secure authenticated channels
> > > > > (SSL,SASL,SCRAM).
> > > > > For plain text (non-secure) channels, the principal will be always
> > > > > ANONYMOUS. Here you can authorize by ip-address.
> > > > >
> > > > > It's adviced to run on secure channels. you can try SASL/PLAIN or
> > SCRAM
> > > > > mechanisms with/without SSL.
> > > > > Pl, check Kafka docs for security considerations.
> > > > >
> > > > > >> I was testing if a non-admin principal (OS user) can modify
> > > > (add/remove)
> > > > > ACLs and
> > > > > it seems like it's possible.
> > > > > How are you trying? Can you give more details about your setup?
> What
> > > > > authentication mechanism, etc.?
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > > > manoj.murumkar@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We are evaluating how to put authorization in place for Kafka
> > (around
> > > > > > topics, mostly). Is it a good idea to do this without Kerberos? I
> > was
> > > > > > testing if a non-admin principal (OS user) can modify
> (add/remove)
> > > ACLs
> > > > > and
> > > > > > it seems like it's possible. If this is right behavior, it's
> > insecure
> > > > and
> > > > > > unusable. What do you guys think?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Manoj
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Using ACLs without Kerberos
Posted by Manikumar <ma...@gmail.com>.
There is no correlation between OS user and Kafka Principal/Username.
Here user name refers to the principal associated with the kafka
communication
channel (Kerberos Principal, SASL/Plain username, Scram username, SSL
certificate)
Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
Any OS user who got access to zookeeper can create ACLs for any Kafka
principal.
This can be restricted by enabling Zk Kerberos authentication.
http://kafka.apache.org/documentation/#zk_authz
From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar <ma...@gmail.com>
wrote:
> Right, I am. Just to be clear, I am using kafka-acl script to define/remove
> ACLs as a non-super user and it just works fine. I had expected it to work
> only for super users and not for regular users ('nex37045' is a normal
> user).
>
> [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> zookeeper.connect=localhost:2181 --list
> Current ACLs for resource `Topic:test`:
> User:nex37045 has Deny permission for operations: Write from hosts: *
> User:skumarmu has Allow permission for operations: Write from hosts: *
>
> [nex37045@or1010051029033 ~]$
> [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> zookeeper.connect=localhost:2181 --remove --allow-principal User:skumarmu
> --operation Write --topic test
> Are you sure you want to remove ACLs:
> User:skumarmu has Allow permission for operations: Write from hosts: *
> from resource `Topic:test`? (y/n)
> y
> Current ACLs for resource `Topic:test`:
> User:nex37045 has Deny permission for operations: Write from hosts: *
>
> [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> zookeeper.connect=localhost:2181 --list
> Current ACLs for resource `Topic:test`:
> User:nex37045 has Deny permission for operations: Write from hosts: *
>
>
>
> On Thu, Aug 31, 2017 at 7:16 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Looks like you are already using SASL/PLAIN mechanism. Kafka supports
> SASL
> > authentication framework.
> > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. you can
> > enable SSL encryption also
> >
> > http://kafka.apache.org/documentation.html#security
> >
> >
> > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
> manoj.murumkar@gmail.com>
> > wrote:
> >
> > > Thanks Manikumar. I am testing the setup documented here:
> > > https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/
> > > (SASL_PLAINTEXT).
> > >
> > > I haven't setup any authentication for the tests. Thinking about it,
> > > authentication is a must have for authorization (so, kafka knows who's
> > > making resource request), right?
> > >
> > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <manikumar.reddy@gmail.com
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Kafka default authorizer is used with secure authenticated channels
> > > > (SSL,SASL,SCRAM).
> > > > For plain text (non-secure) channels, the principal will be always
> > > > ANONYMOUS. Here you can authorize by ip-address.
> > > >
> > > > It's adviced to run on secure channels. you can try SASL/PLAIN or
> SCRAM
> > > > mechanisms with/without SSL.
> > > > Pl, check Kafka docs for security considerations.
> > > >
> > > > >> I was testing if a non-admin principal (OS user) can modify
> > > (add/remove)
> > > > ACLs and
> > > > it seems like it's possible.
> > > > How are you trying? Can you give more details about your setup? What
> > > > authentication mechanism, etc.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > > manoj.murumkar@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We are evaluating how to put authorization in place for Kafka
> (around
> > > > > topics, mostly). Is it a good idea to do this without Kerberos? I
> was
> > > > > testing if a non-admin principal (OS user) can modify (add/remove)
> > ACLs
> > > > and
> > > > > it seems like it's possible. If this is right behavior, it's
> insecure
> > > and
> > > > > unusable. What do you guys think?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Manoj
> > > > >
> > > >
> > >
> >
>
Re: Using ACLs without Kerberos
Posted by Manoj Murumkar <ma...@gmail.com>.
Right, I am. Just to be clear, I am using kafka-acl script to define/remove
ACLs as a non-super user and it just works fine. I had expected it to work
only for super users and not for regular users ('nex37045' is a normal
user).
[nex37045@or1010051029033 ~]$ kafka-acls --authorizer
kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
zookeeper.connect=localhost:2181 --list
Current ACLs for resource `Topic:test`:
User:nex37045 has Deny permission for operations: Write from hosts: *
User:skumarmu has Allow permission for operations: Write from hosts: *
[nex37045@or1010051029033 ~]$
[nex37045@or1010051029033 ~]$ kafka-acls --authorizer
kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
zookeeper.connect=localhost:2181 --remove --allow-principal User:skumarmu
--operation Write --topic test
Are you sure you want to remove ACLs:
User:skumarmu has Allow permission for operations: Write from hosts: *
from resource `Topic:test`? (y/n)
y
Current ACLs for resource `Topic:test`:
User:nex37045 has Deny permission for operations: Write from hosts: *
[nex37045@or1010051029033 ~]$ kafka-acls --authorizer
kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
zookeeper.connect=localhost:2181 --list
Current ACLs for resource `Topic:test`:
User:nex37045 has Deny permission for operations: Write from hosts: *
On Thu, Aug 31, 2017 at 7:16 AM, Manikumar <ma...@gmail.com>
wrote:
> Looks like you are already using SASL/PLAIN mechanism. Kafka supports SASL
> authentication framework.
> KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. you can
> enable SSL encryption also
>
> http://kafka.apache.org/documentation.html#security
>
>
> On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <ma...@gmail.com>
> wrote:
>
> > Thanks Manikumar. I am testing the setup documented here:
> > https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/
> > (SASL_PLAINTEXT).
> >
> > I haven't setup any authentication for the tests. Thinking about it,
> > authentication is a must have for authorization (so, kafka knows who's
> > making resource request), right?
> >
> > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Kafka default authorizer is used with secure authenticated channels
> > > (SSL,SASL,SCRAM).
> > > For plain text (non-secure) channels, the principal will be always
> > > ANONYMOUS. Here you can authorize by ip-address.
> > >
> > > It's adviced to run on secure channels. you can try SASL/PLAIN or SCRAM
> > > mechanisms with/without SSL.
> > > Pl, check Kafka docs for security considerations.
> > >
> > > >> I was testing if a non-admin principal (OS user) can modify
> > (add/remove)
> > > ACLs and
> > > it seems like it's possible.
> > > How are you trying? Can you give more details about your setup? What
> > > authentication mechanism, etc.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > manoj.murumkar@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We are evaluating how to put authorization in place for Kafka (around
> > > > topics, mostly). Is it a good idea to do this without Kerberos? I was
> > > > testing if a non-admin principal (OS user) can modify (add/remove)
> ACLs
> > > and
> > > > it seems like it's possible. If this is right behavior, it's insecure
> > and
> > > > unusable. What do you guys think?
> > > >
> > > > Thanks,
> > > >
> > > > Manoj
> > > >
> > >
> >
>
Re: Using ACLs without Kerberos
Posted by Manikumar <ma...@gmail.com>.
Looks like you are already using SASL/PLAIN mechanism. Kafka supports SASL
authentication framework.
KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms. you can
enable SSL encryption also
http://kafka.apache.org/documentation.html#security
On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <ma...@gmail.com>
wrote:
> Thanks Manikumar. I am testing the setup documented here:
> https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/
> (SASL_PLAINTEXT).
>
> I haven't setup any authentication for the tests. Thinking about it,
> authentication is a must have for authorization (so, kafka knows who's
> making resource request), right?
>
> On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Kafka default authorizer is used with secure authenticated channels
> > (SSL,SASL,SCRAM).
> > For plain text (non-secure) channels, the principal will be always
> > ANONYMOUS. Here you can authorize by ip-address.
> >
> > It's adviced to run on secure channels. you can try SASL/PLAIN or SCRAM
> > mechanisms with/without SSL.
> > Pl, check Kafka docs for security considerations.
> >
> > >> I was testing if a non-admin principal (OS user) can modify
> (add/remove)
> > ACLs and
> > it seems like it's possible.
> > How are you trying? Can you give more details about your setup? What
> > authentication mechanism, etc.?
> >
> >
> > Thanks,
> >
> >
> > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> manoj.murumkar@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > We are evaluating how to put authorization in place for Kafka (around
> > > topics, mostly). Is it a good idea to do this without Kerberos? I was
> > > testing if a non-admin principal (OS user) can modify (add/remove) ACLs
> > and
> > > it seems like it's possible. If this is right behavior, it's insecure
> and
> > > unusable. What do you guys think?
> > >
> > > Thanks,
> > >
> > > Manoj
> > >
> >
>
Re: Using ACLs without Kerberos
Posted by Manoj Murumkar <ma...@gmail.com>.
Thanks Manikumar. I am testing the setup documented here:
https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/
(SASL_PLAINTEXT).
I haven't setup any authentication for the tests. Thinking about it,
authentication is a must have for authorization (so, kafka knows who's
making resource request), right?
On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <ma...@gmail.com>
wrote:
> Hi,
>
> Kafka default authorizer is used with secure authenticated channels
> (SSL,SASL,SCRAM).
> For plain text (non-secure) channels, the principal will be always
> ANONYMOUS. Here you can authorize by ip-address.
>
> It's adviced to run on secure channels. you can try SASL/PLAIN or SCRAM
> mechanisms with/without SSL.
> Pl, check Kafka docs for security considerations.
>
> >> I was testing if a non-admin principal (OS user) can modify (add/remove)
> ACLs and
> it seems like it's possible.
> How are you trying? Can you give more details about your setup? What
> authentication mechanism, etc.?
>
>
> Thanks,
>
>
> On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > We are evaluating how to put authorization in place for Kafka (around
> > topics, mostly). Is it a good idea to do this without Kerberos? I was
> > testing if a non-admin principal (OS user) can modify (add/remove) ACLs
> and
> > it seems like it's possible. If this is right behavior, it's insecure and
> > unusable. What do you guys think?
> >
> > Thanks,
> >
> > Manoj
> >
>
Re: Using ACLs without Kerberos
Posted by Manikumar <ma...@gmail.com>.
Hi,
Kafka default authorizer is used with secure authenticated channels
(SSL,SASL,SCRAM).
For plain text (non-secure) channels, the principal will be always
ANONYMOUS. Here you can authorize by ip-address.
It's adviced to run on secure channels. you can try SASL/PLAIN or SCRAM
mechanisms with/without SSL.
Pl, check Kafka docs for security considerations.
>> I was testing if a non-admin principal (OS user) can modify (add/remove)
ACLs and
it seems like it's possible.
How are you trying? Can you give more details about your setup? What
authentication mechanism, etc.?
Thanks,
On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <ma...@gmail.com>
wrote:
> Hi,
>
> We are evaluating how to put authorization in place for Kafka (around
> topics, mostly). Is it a good idea to do this without Kerberos? I was
> testing if a non-admin principal (OS user) can modify (add/remove) ACLs and
> it seems like it's possible. If this is right behavior, it's insecure and
> unusable. What do you guys think?
>
> Thanks,
>
> Manoj
>