You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Gautam Borad <gb...@gmail.com> on 2016/04/11 11:36:00 UTC
Re: Does apache kafka authorizer must Dependent on kerberos ?
+ dev list
Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in
secure(kerberos) mode.
On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com>
wrote:
> Hi gborad：
>
>
> Long time no communication, I hope you have time to help me answer this question, thank you.
>
> Does apache kafka authorizer must Dependent on kerberos ?
>
>
> Also have a error as follow
>
>
> [2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
> java.lang.NullPointerException
> at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
>
> at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
>
> at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
>
> at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
>
> at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
> at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
> at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
> at scala.Option.map(Option.scala:146)
> at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
> at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
> at kafka.Kafka$.main(Kafka.scala:67)
> at kafka.Kafka.main(Kafka.scala)
>
> Regards
> yuhuan.li
>
> ------------------------------
> liyuhuan1991@126.com
>
--
Regards,
Gautam.
Re: Does apache kafka authorizer must Dependent on kerberos ?
Posted by Don Bosco Durai <bo...@apache.org>.
Just to make sure I understood it correctly, your Kafka is Kerberized?
If your Kafka is kerberized, then the User should get proper user values.
Thanks
Bosco
From: "liyuhuan1991@126.com" <li...@126.com>
Date: Tuesday, April 12, 2016 at 7:19 PM
To: Don Bosco Durai <bo...@apache.org>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
Cc: "kafka@harsha.io" <ka...@harsha.io>, Gautam Borad <gb...@gmail.com>, "allal@ebay.com" <al...@ebay.com>
Subject: Re: Re: Does apache kafka authorizer must Dependent on kerberos ?
Hi all
My test is based on kafka 0.9.0 and ranger-0.5.2-rc1 , if there is not Kerberos, only IP based policies makes sense.
but if I would like to authorize to some users , the audit log user filed is ANONYMOUS. so I guess user should get from kerberos
liyuhuan1991@126.com
From: Don Bosco Durai
Date: 2016-04-13 08:12
To: dev@ranger.incubator.apache.org; liyuhuan1991@126.com
CC: kafka@harsha.io
Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
Alok, thanks for confirming.
I looked into the code. We are catching the below error and ignoring it.
Yuhuan, does Kafka work even after getting this error? Technically it should, unless something fails later. Just note, if there is not Kerberos, then only IP based policies makes sense.
Thanks
Bosco
On 4/12/16, 5:04 PM, "allal@ebay.com" <al...@ebay.com> wrote:
>@Bosco my testing was around verifying ability to authorize access to Secure Kafka simultaneously over a secure and insecure channel.
>
>To give context to others on this list: Rationale for the above testing was to avoid regression for users who are already using a secure cluster with insecure Kafka. Note that existing applications don’t have any security so aim of testing was to ensure that those legacy applications would continue to work via insecure channel support. Over time such users would transition applications over to secure channel to leveraged from fine grained authorization. Those findings are captured here <https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin>.
>
>Thanks
>
>
>
>
>On 4/12/16, 3:21 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
>
>>+Harsha
>>
>>This is an interesting issue. In the final release version, if I am not wrong, Kafka started supporting both secure and non-secure in the same deployment, but by using different ports.
>>
>>I have copied Harsha from the Kafka community. He should be able to answer what was finally included in Kafka 0.9.
>>
>>I didn’t the test Ranger+Kafka without Kerberos, but technically, we should be able to support non-secure deployment with IP based policies.
>>
>>Alok, I remember you trying IP based policies. Were there in a non-kerberos mode?
>>
>>Thanks
>>
>>Bosco
>>
>>
>>From: Gautam Borad <gb...@gmail.com>
>>Reply-To: <de...@ranger.incubator.apache.org>
>>Date: Monday, April 11, 2016 at 2:36 AM
>>To: "liyuhuan1991@126.com" <li...@126.com>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
>>Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
>>
>>+ dev list
>>
>>Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in secure(kerberos) mode.
>>
>>On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com> wrote:
>>Hi gborad：
>>
>>Long time no communication, I hope you have time to help me answer this question, thank you.
>>
>>Does apache kafka authorizer must Dependent on kerberos ?
>>
>>
>>Also have a error as follow
>>
>>[2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>>java.lang.NullPointerException
>>at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
>>at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
>>at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
>>at scala.Option.map(Option.scala:146)
>>at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
>>at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
>>at kafka.Kafka$.main(Kafka.scala:67)
>>at kafka.Kafka.main(Kafka.scala)
>>
>>Regards
>>yuhuan.li
>>
>>liyuhuan1991@126.com
>>
>>
>>
>>--
>>Regards,
>>Gautam.
>>
Re: Re: Does apache kafka authorizer must Dependent on kerberos ?
Posted by "liyuhuan1991@126.com" <li...@126.com>.
Hi all
My test is based on kafka 0.9.0 and ranger-0.5.2-rc1 , if there is not Kerberos, only IP based policies makes sense.
but if I would like to authorize to some users , the audit log user filed is ANONYMOUS. so I guess user should get from kerberos
liyuhuan1991@126.com
From: Don Bosco Durai
Date: 2016-04-13 08:12
To: dev@ranger.incubator.apache.org; liyuhuan1991@126.com
CC: kafka@harsha.io
Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
Alok, thanks for confirming.
I looked into the code. We are catching the below error and ignoring it.
Yuhuan, does Kafka work even after getting this error? Technically it should, unless something fails later. Just note, if there is not Kerberos, then only IP based policies makes sense.
Thanks
Bosco
On 4/12/16, 5:04 PM, "allal@ebay.com" <al...@ebay.com> wrote:
>@Bosco my testing was around verifying ability to authorize access to Secure Kafka simultaneously over a secure and insecure channel.
>
>To give context to others on this list: Rationale for the above testing was to avoid regression for users who are already using a secure cluster with insecure Kafka. Note that existing applications don’t have any security so aim of testing was to ensure that those legacy applications would continue to work via insecure channel support. Over time such users would transition applications over to secure channel to leveraged from fine grained authorization. Those findings are captured here <https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin>.
>
>Thanks
>
>
>
>
>On 4/12/16, 3:21 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
>
>>+Harsha
>>
>>This is an interesting issue. In the final release version, if I am not wrong, Kafka started supporting both secure and non-secure in the same deployment, but by using different ports.
>>
>>I have copied Harsha from the Kafka community. He should be able to answer what was finally included in Kafka 0.9.
>>
>>I didn’t the test Ranger+Kafka without Kerberos, but technically, we should be able to support non-secure deployment with IP based policies.
>>
>>Alok, I remember you trying IP based policies. Were there in a non-kerberos mode?
>>
>>Thanks
>>
>>Bosco
>>
>>
>>From: Gautam Borad <gb...@gmail.com>
>>Reply-To: <de...@ranger.incubator.apache.org>
>>Date: Monday, April 11, 2016 at 2:36 AM
>>To: "liyuhuan1991@126.com" <li...@126.com>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
>>Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
>>
>>+ dev list
>>
>>Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in secure(kerberos) mode.
>>
>>On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com> wrote:
>>Hi gborad：
>>
>>Long time no communication, I hope you have time to help me answer this question, thank you.
>>
>>Does apache kafka authorizer must Dependent on kerberos ?
>>
>>
>>Also have a error as follow
>>
>>[2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>>java.lang.NullPointerException
>>at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
>>at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
>>at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
>>at scala.Option.map(Option.scala:146)
>>at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
>>at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
>>at kafka.Kafka$.main(Kafka.scala:67)
>>at kafka.Kafka.main(Kafka.scala)
>>
>>Regards
>>yuhuan.li
>>
>>liyuhuan1991@126.com
>>
>>
>>
>>--
>>Regards,
>>Gautam.
>>
Re: Does apache kafka authorizer must Dependent on kerberos ?
Posted by Don Bosco Durai <bo...@apache.org>.
Alok, thanks for confirming.
I looked into the code. We are catching the below error and ignoring it.
Yuhuan, does Kafka work even after getting this error? Technically it should, unless something fails later. Just note, if there is not Kerberos, then only IP based policies makes sense.
Thanks
Bosco
On 4/12/16, 5:04 PM, "allal@ebay.com" <al...@ebay.com> wrote:
>@Bosco my testing was around verifying ability to authorize access to Secure Kafka simultaneously over a secure and insecure channel.
>
>To give context to others on this list: Rationale for the above testing was to avoid regression for users who are already using a secure cluster with insecure Kafka. Note that existing applications don’t have any security so aim of testing was to ensure that those legacy applications would continue to work via insecure channel support. Over time such users would transition applications over to secure channel to leveraged from fine grained authorization. Those findings are captured here <https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin>.
>
>Thanks
>
>
>
>
>On 4/12/16, 3:21 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
>
>>+Harsha
>>
>>This is an interesting issue. In the final release version, if I am not wrong, Kafka started supporting both secure and non-secure in the same deployment, but by using different ports.
>>
>>I have copied Harsha from the Kafka community. He should be able to answer what was finally included in Kafka 0.9.
>>
>>I didn’t the test Ranger+Kafka without Kerberos, but technically, we should be able to support non-secure deployment with IP based policies.
>>
>>Alok, I remember you trying IP based policies. Were there in a non-kerberos mode?
>>
>>Thanks
>>
>>Bosco
>>
>>
>>From: Gautam Borad <gb...@gmail.com>
>>Reply-To: <de...@ranger.incubator.apache.org>
>>Date: Monday, April 11, 2016 at 2:36 AM
>>To: "liyuhuan1991@126.com" <li...@126.com>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
>>Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
>>
>>+ dev list
>>
>>Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in secure(kerberos) mode.
>>
>>On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com> wrote:
>>Hi gborad：
>>
>>Long time no communication, I hope you have time to help me answer this question, thank you.
>>
>>Does apache kafka authorizer must Dependent on kerberos ?
>>
>>
>>Also have a error as follow
>>
>>[2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>>java.lang.NullPointerException
>>at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
>>at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
>>at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
>>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
>>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
>>at scala.Option.map(Option.scala:146)
>>at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
>>at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
>>at kafka.Kafka$.main(Kafka.scala:67)
>>at kafka.Kafka.main(Kafka.scala)
>>
>>Regards
>>yuhuan.li
>>
>>liyuhuan1991@126.com
>>
>>
>>
>>--
>>Regards,
>>Gautam.
>>
Re: Does apache kafka authorizer must Dependent on kerberos ?
Posted by "allal@ebay.com" <al...@ebay.com>.
@Bosco my testing was around verifying ability to authorize access to Secure Kafka simultaneously over a secure and insecure channel.
To give context to others on this list: Rationale for the above testing was to avoid regression for users who are already using a secure cluster with insecure Kafka. Note that existing applications don’t have any security so aim of testing was to ensure that those legacy applications would continue to work via insecure channel support. Over time such users would transition applications over to secure channel to leveraged from fine grained authorization. Those findings are captured here <https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin>.
Thanks
On 4/12/16, 3:21 PM, "Don Bosco Durai" <bo...@apache.org> wrote:
>+Harsha
>
>This is an interesting issue. In the final release version, if I am not wrong, Kafka started supporting both secure and non-secure in the same deployment, but by using different ports.
>
>I have copied Harsha from the Kafka community. He should be able to answer what was finally included in Kafka 0.9.
>
>I didn’t the test Ranger+Kafka without Kerberos, but technically, we should be able to support non-secure deployment with IP based policies.
>
>Alok, I remember you trying IP based policies. Were there in a non-kerberos mode?
>
>Thanks
>
>Bosco
>
>
>From: Gautam Borad <gb...@gmail.com>
>Reply-To: <de...@ranger.incubator.apache.org>
>Date: Monday, April 11, 2016 at 2:36 AM
>To: "liyuhuan1991@126.com" <li...@126.com>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
>Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
>
>+ dev list
>
>Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in secure(kerberos) mode.
>
>On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com> wrote:
>Hi gborad：
>
>Long time no communication, I hope you have time to help me answer this question, thank you.
>
>Does apache kafka authorizer must Dependent on kerberos ?
>
>
>Also have a error as follow
>
>[2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>java.lang.NullPointerException
>at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
>at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
>at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
>at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
>at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
>at scala.Option.map(Option.scala:146)
>at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
>at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
>at kafka.Kafka$.main(Kafka.scala:67)
>at kafka.Kafka.main(Kafka.scala)
>
>Regards
>yuhuan.li
>
>liyuhuan1991@126.com
>
>
>
>--
>Regards,
>Gautam.
>
Re: Does apache kafka authorizer must Dependent on kerberos ?
Posted by Don Bosco Durai <bo...@apache.org>.
+Harsha
This is an interesting issue. In the final release version, if I am not wrong, Kafka started supporting both secure and non-secure in the same deployment, but by using different ports.
I have copied Harsha from the Kafka community. He should be able to answer what was finally included in Kafka 0.9.
I didn’t the test Ranger+Kafka without Kerberos, but technically, we should be able to support non-secure deployment with IP based policies.
Alok, I remember you trying IP based policies. Were there in a non-kerberos mode?
Thanks
Bosco
From: Gautam Borad <gb...@gmail.com>
Reply-To: <de...@ranger.incubator.apache.org>
Date: Monday, April 11, 2016 at 2:36 AM
To: "liyuhuan1991@126.com" <li...@126.com>, "dev@ranger.incubator.apache.org" <de...@ranger.incubator.apache.org>
Subject: Re: Does apache kafka authorizer must Dependent on kerberos ?
+ dev list
Hi Yuhuan, Yes, you are right, the KAFKA plugins works only in secure(kerberos) mode.
On Mon, Apr 11, 2016 at 1:54 PM, liyuhuan1991@126.com <li...@126.com> wrote:
Hi gborad：
Long time no communication, I hope you have time to help me answer this question, thank you.
Does apache kafka authorizer must Dependent on kerberos ?
Also have a error as follow
[2016-04-11 15:51:39,775] ERROR Error getting principal. (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
java.lang.NullPointerException
at org.apache.kafka.common.security.kerberos.Login.<init>(Login.java:98)
at org.apache.kafka.common.security.kerberos.LoginManager.<init>(LoginManager.java:44)
at org.apache.kafka.common.security.kerberos.LoginManager.acquireLoginManager(LoginManager.java:85)
at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
at org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:197)
at kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:195)
at scala.Option.map(Option.scala:146)
at kafka.server.KafkaServer.startup(KafkaServer.scala:195)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
Regards
yuhuan.li
liyuhuan1991@126.com
--
Regards,
Gautam.