You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Christopher Shannon <ch...@gmail.com> on 2021/06/08 18:09:12 UTC
[DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Hi Everyone,
I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic creation
The KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
I submitted the following PR: https://github.com/apache/kafka/pull/10845
I know it might be too late based on the schedule but it's a small change
so it would be nice to get this in time for 3.0.0.
Thanks,
Chris
Re: [DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Posted by Christopher Shannon <ch...@gmail.com>.
As a quick follow up after I sent my last response I realized that passing
the principal to the create topic policy still probably doesn't work
because while you have the user you wouldn't know the permissions allowed.
You could mark the principal as a super user or something but the ACL
authorizer is what loads everything so you really need to have access to
the ACLs loaded to control and make decisions per user so I don't know that
passing the principal to the create topic policy really solves the issue.
About the only thing you could do with the principal is just check if it's
a super user or not which is better but not perfect.
On Wed, Jun 23, 2021 at 8:49 AM Christopher Shannon <
christopher.l.shannon@gmail.com> wrote:
> Hi Ismael,
>
> Thanks for the response.
>
> I initially tried to use a create topic policy but the issue with the
> CreateTopicPolicy and AlterConfigPolicy API is the authenticated user
> making the request is not known as it is not passed as part of the metadata
> to the validation request. In my situation I want to allow administrators
> (or other users granted permission) the ability to set explicit topic
> configs but certain other users I would want to deny this ability. So
> essentially I need the following:
>
> 1) Allow admins or super users the ability to create topics and also set
> explicit configs on the topics
> 2) Allow some users the ability to auto-create topics but not set explicit
> configs (only given cluster defaults)
> 3) Deny creation of topics entirely to other users
>
> One alternative could be to pass the user principal to a create topic
> policy and alter config policy. I actually had thought of this but figured
> ACLs were better since the goal here is really more permissions based so
> ACLs seemed like a better fit.
>
> I updated my KIP to add the goals I listed above and also show the option
> to pass a user principal to rejected alternatives. However, if you think it
> is better I am open to going with the different approach of providing the
> user principal as part of the metadata to the CreateTopicPolicy and
> AlterConfigPolicies.
>
> Chris
>
> On Wed, Jun 23, 2021 at 8:27 AM Ismael Juma <is...@juma.me.uk> wrote:
>
>> Hi Christopher,
>>
>> Thanks for the KIP. Have you considered using create topic policies to
>> enforce the configuration you desire even when create topics is used
>> directly? You could disallow all create topics requests that set explicit
>> topic configs (i.e. don't rely on cluster defaults).
>>
>> This would work better since may rely on the create topics protocol api
>> since it's the recommended way to create topics.
>>
>> Ismael
>>
>> On Tue, Jun 8, 2021, 11:09 AM Christopher Shannon <
>> christopher.l.shannon@gmail.com> wrote:
>>
>> > Hi Everyone,
>> >
>> > I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic
>> > creation
>> >
>> > The KIP can be found here:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
>> >
>> > I submitted the following PR:
>> https://github.com/apache/kafka/pull/10845
>> >
>> > I know it might be too late based on the schedule but it's a small
>> change
>> > so it would be nice to get this in time for 3.0.0.
>> >
>> > Thanks,
>> > Chris
>> >
>>
>
Re: [DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Posted by Christopher Shannon <ch...@gmail.com>.
Hi Ismael,
Thanks for the response.
I initially tried to use a create topic policy but the issue with the
CreateTopicPolicy and AlterConfigPolicy API is the authenticated user
making the request is not known as it is not passed as part of the metadata
to the validation request. In my situation I want to allow administrators
(or other users granted permission) the ability to set explicit topic
configs but certain other users I would want to deny this ability. So
essentially I need the following:
1) Allow admins or super users the ability to create topics and also set
explicit configs on the topics
2) Allow some users the ability to auto-create topics but not set explicit
configs (only given cluster defaults)
3) Deny creation of topics entirely to other users
One alternative could be to pass the user principal to a create topic
policy and alter config policy. I actually had thought of this but figured
ACLs were better since the goal here is really more permissions based so
ACLs seemed like a better fit.
I updated my KIP to add the goals I listed above and also show the option
to pass a user principal to rejected alternatives. However, if you think it
is better I am open to going with the different approach of providing the
user principal as part of the metadata to the CreateTopicPolicy and
AlterConfigPolicies.
Chris
On Wed, Jun 23, 2021 at 8:27 AM Ismael Juma <is...@juma.me.uk> wrote:
> Hi Christopher,
>
> Thanks for the KIP. Have you considered using create topic policies to
> enforce the configuration you desire even when create topics is used
> directly? You could disallow all create topics requests that set explicit
> topic configs (i.e. don't rely on cluster defaults).
>
> This would work better since may rely on the create topics protocol api
> since it's the recommended way to create topics.
>
> Ismael
>
> On Tue, Jun 8, 2021, 11:09 AM Christopher Shannon <
> christopher.l.shannon@gmail.com> wrote:
>
> > Hi Everyone,
> >
> > I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic
> > creation
> >
> > The KIP can be found here:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
> >
> > I submitted the following PR: https://github.com/apache/kafka/pull/10845
> >
> > I know it might be too late based on the schedule but it's a small change
> > so it would be nice to get this in time for 3.0.0.
> >
> > Thanks,
> > Chris
> >
>
Re: [DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Christopher,
Thanks for the KIP. Have you considered using create topic policies to
enforce the configuration you desire even when create topics is used
directly? You could disallow all create topics requests that set explicit
topic configs (i.e. don't rely on cluster defaults).
This would work better since may rely on the create topics protocol api
since it's the recommended way to create topics.
Ismael
On Tue, Jun 8, 2021, 11:09 AM Christopher Shannon <
christopher.l.shannon@gmail.com> wrote:
> Hi Everyone,
>
> I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic
> creation
>
> The KIP can be found here:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
>
> I submitted the following PR: https://github.com/apache/kafka/pull/10845
>
> I know it might be too late based on the schedule but it's a small change
> so it would be nice to get this in time for 3.0.0.
>
> Thanks,
> Chris
>
Re: [DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Posted by Christopher Shannon <ch...@gmail.com>.
Hi Everyone,
It has been a couple weeks so figured I'd ping one last time to see if
anyone had any interest in this. This feature would be quite useful to me
(and hopefully others) and should be a pretty small change and backwards
compatible.
Chris
On Fri, Jun 11, 2021 at 9:24 AM Christopher Shannon <
christopher.l.shannon@gmail.com> wrote:
> Does anyone have any thoughts or feedback on this KIP?
>
> On Tue, Jun 8, 2021 at 2:09 PM Christopher Shannon <
> christopher.l.shannon@gmail.com> wrote:
>
>> Hi Everyone,
>>
>> I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic
>> creation
>>
>> The KIP can be found here:
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
>>
>> I submitted the following PR: https://github.com/apache/kafka/pull/10845
>>
>> I know it might be too late based on the schedule but it's a small change
>> so it would be nice to get this in time for 3.0.0.
>>
>> Thanks,
>> Chris
>>
>
Re: [DISCUSS] KIP-755: Add new AUTO_CREATE ACL for auto topic creation
Posted by Christopher Shannon <ch...@gmail.com>.
Does anyone have any thoughts or feedback on this KIP?
On Tue, Jun 8, 2021 at 2:09 PM Christopher Shannon <
christopher.l.shannon@gmail.com> wrote:
> Hi Everyone,
>
> I'd like to propose KIP-755: Add new AUTO_CREATE ACL for auto topic
> creation
>
> The KIP can be found here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-755%3A+Add+new+AUTO_CREATE+ACL+for+auto+topic+creation
>
> I submitted the following PR: https://github.com/apache/kafka/pull/10845
>
> I know it might be too late based on the schedule but it's a small change
> so it would be nice to get this in time for 3.0.0.
>
> Thanks,
> Chris
>