You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Edoardo Comar <EC...@uk.ibm.com> on 2018/03/29 13:51:19 UTC
[DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Hi all,
We have submitted KIP-277 to give users permission to manage the lifecycle
of a defined set of topics;
the current ACL checks are for permission to create *any* topic and on
delete for permission against the *named* topics.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-277+-+Fine+Grained+ACL+for+CreateTopics+API
Feedback and suggestions are welcome, thanks.
Edo & Mickael
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Guozhang Wang <wa...@gmail.com>.
Thanks for the KIP. The updated wiki LGTM to me now.
I'd also want to have wildcard support for create / delete topic ACLs for
Streams usage :) But I'd agree the feature itself may deserve a separate
KIP.
Guozhang
On Tue, Apr 17, 2018 at 4:40 AM, Edoardo Comar <EC...@uk.ibm.com> wrote:
> We updated the KIP addressing the backward compatibilty issue,
>
> proposing to change the current ACL check for creating a topic T,
> from
> CREATE on Cluster
> to
> CREATE on Cluster OR CREATE on Topic(T).
>
> leaving the enhancement of wildcard support to a future KIP.
>
> please let us know of any further thoughts, we'd like to start a vote next
> week.
>
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From: Edoardo Comar <EC...@uk.ibm.com>
> To: dev@kafka.apache.org
> Date: 17/04/2018 12:12
> Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
> API
>
>
>
> Thanks Colin,
>
> > I think the reason why CreateTopics requires CREATE CLUSTER is that
> creating topics was considered something that only an administrator would
> want to do.
> I think this may no longer be the case ... Streams/KSQL application create
>
> intermediate topics where only the start of the name, not the full name is
>
> known a priori.
> This use case screams for regex support, for which we think a different
> KIP is needed.
>
> > Do they need to be sandboxed in some other way (i.e. don't allow a
> lower-privileged app to create a topic with 10,000 partitions on 1,000
> brokers).
> That kind of permission should be handled by a policy, not by an
> authorizer.
> Though we know that Policies at the moment do not have a Principal they
> can base their decision upon.
> That is the subject of KIP-201
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> apache.org_confluence_display_KAFKA_KIP-2D201-253A-
> 2BRationalising-2BPolicy-2Binterfaces&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=
> EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=5tMuNgNq5NoJTeZ_
> hXOkqifRtB6gItQLBzBreZz2yQs&s=ThPZTu8X3UdBOSUmUt-oBIP5FH_
> vTElS8bo6NNM2Mpk&e=
>
> (which is a bit stuck at the moment?)
>
>
> > So it seems logical for CREATE CLUSTER to continue to allow the
> administrator to create things such as topics. I don't think we should
> remove this permission.
>
> Ok, so as stated in the KIP
> > An alternative that we want to discuss with the community is to favour
> compatibility rather than simplicity,
> > and consider existing "Create Cluster" permission as equivalent to
> "Create Any Topics", so that Create Cluster is allowed, skip the specific
> Create Topic check.
>
> So based on yours and the community feedback we'll revise the kip by
> including, not rejecting that approach.
> This is still much simpler than wildcard support and goes some way towards
>
> having a better fine grained topic authorization support.
>
> Edo
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From: Colin McCabe <cm...@apache.org>
> To: dev@kafka.apache.org
> Date: 11/04/2018 16:18
> Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
> API
>
>
>
> Hi Edoardo,
>
> Permissions on the Cluster singleton are very powerful. For example,
> ALTER CLUSTER gives you the ability to add or remove any other ACLs you
> like (essentially unlimited permissions). ALTERCONFIGS CLUSTER give syou
> the ability to reconfigure the brokers, and so forth. The general idea is
>
> that administrators have the ability to do things to the whole cluster,
> regular users don't. So it seems logical for CREATE CLUSTER to continue
> to allow the administrator to create things such as topics. I don't think
>
> we should remove this permission.
>
> I think the reason why CreateTopics requires CREATE CLUSTER is that
> creating topics was considered something that only an administrator would
> want to do. If we want regular applications to be able to create topics
> as well, we should think carefully about what their usage pattern would
> be. Would they create topics with names that are known to the
> administrator ahead of time? Or would the topic names be hard to predict?
>
> Do they need to be sandboxed in some other way (i.e. don't allow a
> lower-privileged app to create a topic with 10,000 partitions on 1,000
> brokers).
>
> We don't have the concept of the "owner" of a topic, so deletion becomes
> clunky. Perhaps someone gave me CREATE CLUSTER permission, but no DELETE
> TOPIC permissions. So I can create lots and lots of topics, but never
> delete anything. If the admin doesn't know ahead of time what topic names
>
> I'll be creating, the admin's choice is to give me no delete permissions,
> or to give me DELETE TOPIC * -- neither of which are good choices.
>
> best,
> Colin
>
>
> On Thu, Mar 29, 2018, at 06:51, Edoardo Comar wrote:
> > Hi all,
> >
> > We have submitted KIP-277 to give users permission to manage the
> lifecycle
> > of a defined set of topics;
> > the current ACL checks are for permission to create *any* topic and on
> > delete for permission against the *named* topics.
> >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-
> 2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwICaQ&
> c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0
> OaeRo7hgW4_tQ&m=FGD_jPbAE8c40Y0jBkt8mhk-ZPYcM2uxwmfyNyKpvyI&s=
> BdqgK33Sxwbmy-KHmnVt1TfG-HXc44Tef1SZ81ESers&e=
>
>
> >
> > Feedback and suggestions are welcome, thanks.
> >
> > Edo & Mickael
> > --------------------------------------------------
> >
> > Edoardo Comar
> >
> > IBM Message Hub
> >
> > IBM UK Ltd, Hursley Park, SO21 2JN
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
>
>
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
--
-- Guozhang
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <EC...@uk.ibm.com>.
We updated the KIP addressing the backward compatibilty issue,
proposing to change the current ACL check for creating a topic T,
from
CREATE on Cluster
to
CREATE on Cluster OR CREATE on Topic(T).
leaving the enhancement of wildcard support to a future KIP.
please let us know of any further thoughts, we'd like to start a vote next
week.
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Edoardo Comar <EC...@uk.ibm.com>
To: dev@kafka.apache.org
Date: 17/04/2018 12:12
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Thanks Colin,
> I think the reason why CreateTopics requires CREATE CLUSTER is that
creating topics was considered something that only an administrator would
want to do.
I think this may no longer be the case ... Streams/KSQL application create
intermediate topics where only the start of the name, not the full name is
known a priori.
This use case screams for regex support, for which we think a different
KIP is needed.
> Do they need to be sandboxed in some other way (i.e. don't allow a
lower-privileged app to create a topic with 10,000 partitions on 1,000
brokers).
That kind of permission should be handled by a policy, not by an
authorizer.
Though we know that Policies at the moment do not have a Principal they
can base their decision upon.
That is the subject of KIP-201
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D201-253A-2BRationalising-2BPolicy-2Binterfaces&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=5tMuNgNq5NoJTeZ_hXOkqifRtB6gItQLBzBreZz2yQs&s=ThPZTu8X3UdBOSUmUt-oBIP5FH_vTElS8bo6NNM2Mpk&e=
(which is a bit stuck at the moment?)
> So it seems logical for CREATE CLUSTER to continue to allow the
administrator to create things such as topics. I don't think we should
remove this permission.
Ok, so as stated in the KIP
> An alternative that we want to discuss with the community is to favour
compatibility rather than simplicity,
> and consider existing "Create Cluster" permission as equivalent to
"Create Any Topics", so that Create Cluster is allowed, skip the specific
Create Topic check.
So based on yours and the community feedback we'll revise the kip by
including, not rejecting that approach.
This is still much simpler than wildcard support and goes some way towards
having a better fine grained topic authorization support.
Edo
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Colin McCabe <cm...@apache.org>
To: dev@kafka.apache.org
Date: 11/04/2018 16:18
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Hi Edoardo,
Permissions on the Cluster singleton are very powerful. For example,
ALTER CLUSTER gives you the ability to add or remove any other ACLs you
like (essentially unlimited permissions). ALTERCONFIGS CLUSTER give syou
the ability to reconfigure the brokers, and so forth. The general idea is
that administrators have the ability to do things to the whole cluster,
regular users don't. So it seems logical for CREATE CLUSTER to continue
to allow the administrator to create things such as topics. I don't think
we should remove this permission.
I think the reason why CreateTopics requires CREATE CLUSTER is that
creating topics was considered something that only an administrator would
want to do. If we want regular applications to be able to create topics
as well, we should think carefully about what their usage pattern would
be. Would they create topics with names that are known to the
administrator ahead of time? Or would the topic names be hard to predict?
Do they need to be sandboxed in some other way (i.e. don't allow a
lower-privileged app to create a topic with 10,000 partitions on 1,000
brokers).
We don't have the concept of the "owner" of a topic, so deletion becomes
clunky. Perhaps someone gave me CREATE CLUSTER permission, but no DELETE
TOPIC permissions. So I can create lots and lots of topics, but never
delete anything. If the admin doesn't know ahead of time what topic names
I'll be creating, the admin's choice is to give me no delete permissions,
or to give me DELETE TOPIC * -- neither of which are good choices.
best,
Colin
On Thu, Mar 29, 2018, at 06:51, Edoardo Comar wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the
lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=FGD_jPbAE8c40Y0jBkt8mhk-ZPYcM2uxwmfyNyKpvyI&s=BdqgK33Sxwbmy-KHmnVt1TfG-HXc44Tef1SZ81ESers&e=
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <EC...@uk.ibm.com>.
Thanks Colin,
> I think the reason why CreateTopics requires CREATE CLUSTER is that
creating topics was considered something that only an administrator would
want to do.
I think this may no longer be the case ... Streams/KSQL application create
intermediate topics where only the start of the name, not the full name is
known a priori.
This use case screams for regex support, for which we think a different
KIP is needed.
> Do they need to be sandboxed in some other way (i.e. don't allow a
lower-privileged app to create a topic with 10,000 partitions on 1,000
brokers).
That kind of permission should be handled by a policy, not by an
authorizer.
Though we know that Policies at the moment do not have a Principal they
can base their decision upon.
That is the subject of KIP-201
https://cwiki.apache.org/confluence/display/KAFKA/KIP-201%3A+Rationalising+Policy+interfaces
(which is a bit stuck at the moment?)
> So it seems logical for CREATE CLUSTER to continue to allow the
administrator to create things such as topics. I don't think we should
remove this permission.
Ok, so as stated in the KIP
> An alternative that we want to discuss with the community is to favour
compatibility rather than simplicity,
> and consider existing "Create Cluster" permission as equivalent to
"Create Any Topics", so that Create Cluster is allowed, skip the specific
Create Topic check.
So based on yours and the community feedback we'll revise the kip by
including, not rejecting that approach.
This is still much simpler than wildcard support and goes some way towards
having a better fine grained topic authorization support.
Edo
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Colin McCabe <cm...@apache.org>
To: dev@kafka.apache.org
Date: 11/04/2018 16:18
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Hi Edoardo,
Permissions on the Cluster singleton are very powerful. For example,
ALTER CLUSTER gives you the ability to add or remove any other ACLs you
like (essentially unlimited permissions). ALTERCONFIGS CLUSTER give syou
the ability to reconfigure the brokers, and so forth. The general idea is
that administrators have the ability to do things to the whole cluster,
regular users don't. So it seems logical for CREATE CLUSTER to continue
to allow the administrator to create things such as topics. I don't think
we should remove this permission.
I think the reason why CreateTopics requires CREATE CLUSTER is that
creating topics was considered something that only an administrator would
want to do. If we want regular applications to be able to create topics
as well, we should think carefully about what their usage pattern would
be. Would they create topics with names that are known to the
administrator ahead of time? Or would the topic names be hard to predict?
Do they need to be sandboxed in some other way (i.e. don't allow a
lower-privileged app to create a topic with 10,000 partitions on 1,000
brokers).
We don't have the concept of the "owner" of a topic, so deletion becomes
clunky. Perhaps someone gave me CREATE CLUSTER permission, but no DELETE
TOPIC permissions. So I can create lots and lots of topics, but never
delete anything. If the admin doesn't know ahead of time what topic names
I'll be creating, the admin's choice is to give me no delete permissions,
or to give me DELETE TOPIC * -- neither of which are good choices.
best,
Colin
On Thu, Mar 29, 2018, at 06:51, Edoardo Comar wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the
lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=FGD_jPbAE8c40Y0jBkt8mhk-ZPYcM2uxwmfyNyKpvyI&s=BdqgK33Sxwbmy-KHmnVt1TfG-HXc44Tef1SZ81ESers&e=
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Colin McCabe <cm...@apache.org>.
Hi Edoardo,
Permissions on the Cluster singleton are very powerful. For example, ALTER CLUSTER gives you the ability to add or remove any other ACLs you like (essentially unlimited permissions). ALTERCONFIGS CLUSTER give syou the ability to reconfigure the brokers, and so forth. The general idea is that administrators have the ability to do things to the whole cluster, regular users don't. So it seems logical for CREATE CLUSTER to continue to allow the administrator to create things such as topics. I don't think we should remove this permission.
I think the reason why CreateTopics requires CREATE CLUSTER is that creating topics was considered something that only an administrator would want to do. If we want regular applications to be able to create topics as well, we should think carefully about what their usage pattern would be. Would they create topics with names that are known to the administrator ahead of time? Or would the topic names be hard to predict? Do they need to be sandboxed in some other way (i.e. don't allow a lower-privileged app to create a topic with 10,000 partitions on 1,000 brokers).
We don't have the concept of the "owner" of a topic, so deletion becomes clunky. Perhaps someone gave me CREATE CLUSTER permission, but no DELETE TOPIC permissions. So I can create lots and lots of topics, but never delete anything. If the admin doesn't know ahead of time what topic names I'll be creating, the admin's choice is to give me no delete permissions, or to give me DELETE TOPIC * -- neither of which are good choices.
best,
Colin
On Thu, Mar 29, 2018, at 06:51, Edoardo Comar wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-277+-+Fine+Grained+ACL+for+CreateTopics+API
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <ed...@gmail.com>.
While the vote is still in progress on the [VOTE] thread, (still needing an
extra binding one :-)
we have updated the PR to reflect the current KIP and noted that the check
is performed on two distinct code paths: auto-creation and explicit
creation of a topic.
Edo
On 17 April 2018 at 18:30, Vahid S Hashemian <va...@us.ibm.com>
wrote:
> Hi Edo,
>
> Thanks for addressing that concern in the KIP.
> And I agree that in the long run the create cluster permission should be
> deprecated.
>
> --Vahid
>
>
>
>
> From: Edoardo Comar <EC...@uk.ibm.com>
> To: dev@kafka.apache.org
> Date: 04/17/2018 03:52 AM
> Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
> API
>
>
>
> Thanks Vahid,
>
> as described in the rejected section, we wanted to get feedback on the
> point :
> > An alternative that we want to discuss with the community is to favour
> compatibility rather than simplicity,
> > and consider existing "Create Cluster" permission as equivalent to
> "Create Any Topics", so that Create Cluster is allowed, skip the specific
> Create Topic check.
>
> From the few replies so far, including yours, it seems that having a
> composite check like
> allowed = "has Create Cluster OR has Create Topic(TopicName) "
>
> is the preferred way to go for backward compatibility.
>
> Though we'd like to plan a deprecation for the Create Cluster check, if
> wildcard support in ACLs will be added in the future.
>
> thoughts ?
>
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
>
>
>
> From: "Vahid S Hashemian" <va...@us.ibm.com>
> To: dev@kafka.apache.org
> Date: 04/04/2018 16:41
> Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
> API
>
>
>
> Hi Edo, Mickael,
>
> The intent of this KIP seems to be rather similar to KIP-231 (Improve the
> Required ACL of ListGroups API).
> The feedback I received on that KIP was to allow for backward
> compatibility, and, as a result, the Describe(Cluster) ACL was preserved;
> and a Describe(Group) ACL was introduced.
> I am wondering if both KIPs should follow the same principles in that
> regard.
>
> Thanks.
> --Vahid
>
>
>
> From: Edoardo Comar <EC...@uk.ibm.com>
> To: dev <de...@kafka.apache.org>
> Date: 03/29/2018 06:51 AM
> Subject: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
>
>
>
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the lifecycle
>
>
>
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.
> apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-
> 2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwIBAg&
> c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-
> kjJc7uSVcviKUc&m=fFqzioVsBbv-HQSz8mOPYfz25CJAudbGSgJ3JItDVeE&s=
> DzzeKHrh6r3G5Elm179qbdDLf9OC6e67zqR7d4vnre0&e=
>
>
>
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
>
>
>
--
"When the people fear their government, there is tyranny; when the
government fears the people, there is liberty." [Thomas Jefferson]
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Edo,
Thanks for addressing that concern in the KIP.
And I agree that in the long run the create cluster permission should be
deprecated.
--Vahid
From: Edoardo Comar <EC...@uk.ibm.com>
To: dev@kafka.apache.org
Date: 04/17/2018 03:52 AM
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Thanks Vahid,
as described in the rejected section, we wanted to get feedback on the
point :
> An alternative that we want to discuss with the community is to favour
compatibility rather than simplicity,
> and consider existing "Create Cluster" permission as equivalent to
"Create Any Topics", so that Create Cluster is allowed, skip the specific
Create Topic check.
From the few replies so far, including yours, it seems that having a
composite check like
allowed = "has Create Cluster OR has Create Topic(TopicName) "
is the preferred way to go for backward compatibility.
Though we'd like to plan a deprecation for the Create Cluster check, if
wildcard support in ACLs will be added in the future.
thoughts ?
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: "Vahid S Hashemian" <va...@us.ibm.com>
To: dev@kafka.apache.org
Date: 04/04/2018 16:41
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Hi Edo, Mickael,
The intent of this KIP seems to be rather similar to KIP-231 (Improve the
Required ACL of ListGroups API).
The feedback I received on that KIP was to allow for backward
compatibility, and, as a result, the Describe(Cluster) ACL was preserved;
and a Describe(Group) ACL was introduced.
I am wondering if both KIPs should follow the same principles in that
regard.
Thanks.
--Vahid
From: Edoardo Comar <EC...@uk.ibm.com>
To: dev <de...@kafka.apache.org>
Date: 03/29/2018 06:51 AM
Subject: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Hi all,
We have submitted KIP-277 to give users permission to manage the lifecycle
of a defined set of topics;
the current ACL checks are for permission to create *any* topic and on
delete for permission against the *named* topics.
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=fFqzioVsBbv-HQSz8mOPYfz25CJAudbGSgJ3JItDVeE&s=DzzeKHrh6r3G5Elm179qbdDLf9OC6e67zqR7d4vnre0&e=
Feedback and suggestions are welcome, thanks.
Edo & Mickael
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <EC...@uk.ibm.com>.
Thanks Vahid,
as described in the rejected section, we wanted to get feedback on the
point :
> An alternative that we want to discuss with the community is to favour
compatibility rather than simplicity,
> and consider existing "Create Cluster" permission as equivalent to
"Create Any Topics", so that Create Cluster is allowed, skip the specific
Create Topic check.
From the few replies so far, including yours, it seems that having a
composite check like
allowed = "has Create Cluster OR has Create Topic(TopicName) "
is the preferred way to go for backward compatibility.
Though we'd like to plan a deprecation for the Create Cluster check, if
wildcard support in ACLs will be added in the future.
thoughts ?
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: "Vahid S Hashemian" <va...@us.ibm.com>
To: dev@kafka.apache.org
Date: 04/04/2018 16:41
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Hi Edo, Mickael,
The intent of this KIP seems to be rather similar to KIP-231 (Improve the
Required ACL of ListGroups API).
The feedback I received on that KIP was to allow for backward
compatibility, and, as a result, the Describe(Cluster) ACL was preserved;
and a Describe(Group) ACL was introduced.
I am wondering if both KIPs should follow the same principles in that
regard.
Thanks.
--Vahid
From: Edoardo Comar <EC...@uk.ibm.com>
To: dev <de...@kafka.apache.org>
Date: 03/29/2018 06:51 AM
Subject: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Hi all,
We have submitted KIP-277 to give users permission to manage the lifecycle
of a defined set of topics;
the current ACL checks are for permission to create *any* topic and on
delete for permission against the *named* topics.
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=fFqzioVsBbv-HQSz8mOPYfz25CJAudbGSgJ3JItDVeE&s=DzzeKHrh6r3G5Elm179qbdDLf9OC6e67zqR7d4vnre0&e=
Feedback and suggestions are welcome, thanks.
Edo & Mickael
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Edo, Mickael,
The intent of this KIP seems to be rather similar to KIP-231 (Improve the
Required ACL of ListGroups API).
The feedback I received on that KIP was to allow for backward
compatibility, and, as a result, the Describe(Cluster) ACL was preserved;
and a Describe(Group) ACL was introduced.
I am wondering if both KIPs should follow the same principles in that
regard.
Thanks.
--Vahid
From: Edoardo Comar <EC...@uk.ibm.com>
To: dev <de...@kafka.apache.org>
Date: 03/29/2018 06:51 AM
Subject: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Hi all,
We have submitted KIP-277 to give users permission to manage the lifecycle
of a defined set of topics;
the current ACL checks are for permission to create *any* topic and on
delete for permission against the *named* topics.
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSVcviKUc&m=fFqzioVsBbv-HQSz8mOPYfz25CJAudbGSgJ3JItDVeE&s=DzzeKHrh6r3G5Elm179qbdDLf9OC6e67zqR7d4vnre0&e=
Feedback and suggestions are welcome, thanks.
Edo & Mickael
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <EC...@uk.ibm.com>.
Thanks for your comments Stephane.
I too would like to get 'wildcard' support.
I'd like to associate an acl to a regex instead of a specific resource
name (or everything),
and then the authz check is then performed against the actual resource
name.
This could be attainable with a more sophisticated implementation of
Authorizer.
One immediate use case is to allow an authenticated user to manage
topics/groups and txns that have a given prefix.
This would be the case of Streams applications too, wouldn't it ?
Our Kip is simply about giving the ability for a user to clean up after
himself :-)
It is bad practice to be able to create resources but not to delete them,
and the only current alternative is to give a user the ability to create
and delete any topic,
and that authority may be too broad in some organizations.
cheers
Edo
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Stephane Maarek <st...@simplemachines.com.au>
To: dev@kafka.apache.org
Date: 29/03/2018 18:11
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Not against, but this needs to support regex for support for Kafka streams
application that create many topics with complex names
On Thu., 29 Mar. 2018, 7:21 pm Edoardo Comar, <EC...@uk.ibm.com> wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the
lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
>
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D277-2B-2D-2BFine-2BGrained-2BACL-2Bfor-2BCreateTopics-2BAPI&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=uZGGpiYQPMpPZ2QpZfv5GWdjwWiTIu7Oox8zoBEo-3E&s=y8kJf6lUAsyU6SVgaXy39LCL0JJ35aqg793SxC88PaQ&e=
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Stephane Maarek <st...@simplemachines.com.au>.
Not against, but this needs to support regex for support for Kafka streams
application that create many topics with complex names
On Thu., 29 Mar. 2018, 7:21 pm Edoardo Comar, <EC...@uk.ibm.com> wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-277+-+Fine+Grained+ACL+for+CreateTopics+API
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Edoardo Comar <EC...@uk.ibm.com>.
Thanks Ismael,
in the 'rejected' section we describe an approach that would allow for
compatibility but we'd keep the authz checks simple rather than establish
a precedent
of a logical OR between different checks (cluster or topic(t)) .
What would be your ideal story for compatibility ?
I prefer to think of wildcard support as a separate concern.
And keep this KIP as simple as it is.
Wildcard support could come from a more sophisticated implementation of
the Authorizer;
one that interprets the Resource name associated with ACLs in
kafka.security.auth.Authorizer.addAcls(Set[Acl], Resource)
as a regex to be matched against the actual Resource name in
kafka.security.auth.Authorizer.authorize(RequestChannel.Session,
Operation, Resource)
Thoughts?
cheers
Edo
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
From: Ismael Juma <is...@juma.me.uk>
To: dev <de...@kafka.apache.org>
Date: 29/03/2018 21:37
Subject: Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics
API
Sent by: ismaelj@gmail.com
Thanks for the KIP. I think this is going in the right direction, but we
need a better compatibility story. Also, it's worth considering whether we
want to tackle better wildcard support at the same time.
Ismael
On Thu, Mar 29, 2018 at 6:51 AM, Edoardo Comar <EC...@uk.ibm.com> wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the
lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
>
https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=EzRhmSah4IHsUZVekRUIINhltZK7U0OaeRo7hgW4_tQ&m=q1hztQq7OjhpHNkewaZcug47eAE6Rc0HTDHwzte4zyg&s=Pyt5ScR9iFsNT5o7O33csRafz0nbZEMckyHZgHWGp5w&e=
> 277+-+Fine+Grained+ACL+for+CreateTopics+API
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Matt Farmer <ma...@frmr.me>.
Thanks for this KIP. I can think of some ways we would apply this.
I, too, am ~ on the compatibility story though, however I'm not sure
which way I'd prefer we go at this moment.
On Thu, Mar 29, 2018 at 4:36 PM, Ismael Juma <is...@juma.me.uk> wrote:
> Thanks for the KIP. I think this is going in the right direction, but we
> need a better compatibility story. Also, it's worth considering whether we
> want to tackle better wildcard support at the same time.
>
> Ismael
>
> On Thu, Mar 29, 2018 at 6:51 AM, Edoardo Comar <EC...@uk.ibm.com> wrote:
>
> > Hi all,
> >
> > We have submitted KIP-277 to give users permission to manage the
> lifecycle
> > of a defined set of topics;
> > the current ACL checks are for permission to create *any* topic and on
> > delete for permission against the *named* topics.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 277+-+Fine+Grained+ACL+for+CreateTopics+API
> >
> > Feedback and suggestions are welcome, thanks.
> >
> > Edo & Mickael
> > --------------------------------------------------
> >
> > Edoardo Comar
> >
> > IBM Message Hub
> >
> > IBM UK Ltd, Hursley Park, SO21 2JN
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
> >
>
Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API
Posted by Ismael Juma <is...@juma.me.uk>.
Thanks for the KIP. I think this is going in the right direction, but we
need a better compatibility story. Also, it's worth considering whether we
want to tackle better wildcard support at the same time.
Ismael
On Thu, Mar 29, 2018 at 6:51 AM, Edoardo Comar <EC...@uk.ibm.com> wrote:
> Hi all,
>
> We have submitted KIP-277 to give users permission to manage the lifecycle
> of a defined set of topics;
> the current ACL checks are for permission to create *any* topic and on
> delete for permission against the *named* topics.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 277+-+Fine+Grained+ACL+for+CreateTopics+API
>
> Feedback and suggestions are welcome, thanks.
>
> Edo & Mickael
> --------------------------------------------------
>
> Edoardo Comar
>
> IBM Message Hub
>
> IBM UK Ltd, Hursley Park, SO21 2JN
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>