You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Dong Lin <li...@gmail.com> on 2017/12/04 21:40:44 UTC
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Hey Vahid,
Thanks for the KIP. If I understand the you correctly, you want client to
be able to list all the groups for which it currently has the describe
access.
As of now the ListGroupRequest does not allow user to specify the group. If
user does not have the Describe Cluster access, ListGroupResponse will
return error. This KIP proposes to change the semantics of
ListGroupsResponse such that ListGroupResponse will return the subset of
groups for which the user has the Describe access. And if the does not have
Describe access to any group, ListGroupResponse will return an empty list
with no error.
In my opinion this changes the semantics of ListGroupsResponse in a
counter-intuitive way. Usually we use the ACL to determine whether the
operation on the specified object can be performed or not. The response
should provide either an error message or the result for the specified
object. I couldn't remember a case where the ACL is used to filter the
result without providing error. Do you think this could be a problem for
this KIP?
Thanks,
Dong
On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
vahidhashemian@us.ibm.com> wrote:
> Completing the subject line :)
>
>
>
> From: "Vahid S Hashemian" <va...@us.ibm.com>
> To: dev <de...@kafka.apache.org>
> Date: 11/29/2017 03:17 PM
> Subject: [DISCUSS] KIP-231:
>
>
>
> Hi everyone,
>
> I started KIP-231 to propose a small change to the required ACL of
> ListGroups API (in response to KAFKA-5638):
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=
> jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJ
> c7uSVcviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=
> eu378oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
>
> Your feedback and suggestions are welcome!
>
> Thanks.
> --Vahid
>
>
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Ismael Juma <is...@juma.me.uk>.
One comment below.
On Mon, Dec 4, 2017 at 11:40 PM, Dong Lin <li...@gmail.com> wrote:
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error.
>
We do this in the metadata request that requests all topics. We just return
the topics that the user has describe access to.
Ismael
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Dong Lin <li...@gmail.com>.
I forgot another question. Can you provide a use-case where a user wants to
list all groups for which he/she has the Describe access?
Thanks,
Dong
On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
> Hey Vahid,
>
> Thanks for the KIP. If I understand the you correctly, you want client to
> be able to list all the groups for which it currently has the describe
> access.
>
> As of now the ListGroupRequest does not allow user to specify the group.
> If user does not have the Describe Cluster access, ListGroupResponse will
> return error. This KIP proposes to change the semantics of
> ListGroupsResponse such that ListGroupResponse will return the subset of
> groups for which the user has the Describe access. And if the does not have
> Describe access to any group, ListGroupResponse will return an empty list
> with no error.
>
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error. Do you think this could be a problem for
> this KIP?
>
> Thanks,
> Dong
>
>
> On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> vahidhashemian@us.ibm.com> wrote:
>
>> Completing the subject line :)
>>
>>
>>
>> From: "Vahid S Hashemian" <va...@us.ibm.com>
>> To: dev <de...@kafka.apache.org>
>> Date: 11/29/2017 03:17 PM
>> Subject: [DISCUSS] KIP-231:
>>
>>
>>
>> Hi everyone,
>>
>> I started KIP-231 to propose a small change to the required ACL of
>> ListGroups API (in response to KAFKA-5638):
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
>> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
>> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
>> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
>> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
>> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
>>
>> Your feedback and suggestions are welcome!
>>
>> Thanks.
>> --Vahid
>>
>>
>>
>>
>>
>>
>>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Ted Yu <yu...@gmail.com>.
I agree with Dong on maintaining the semantics.
The user should know which group(s) he / she is allowed to describe.
Cheers
On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
> Hey Vahid,
>
> Thanks for the KIP. If I understand the you correctly, you want client to
> be able to list all the groups for which it currently has the describe
> access.
>
> As of now the ListGroupRequest does not allow user to specify the group. If
> user does not have the Describe Cluster access, ListGroupResponse will
> return error. This KIP proposes to change the semantics of
> ListGroupsResponse such that ListGroupResponse will return the subset of
> groups for which the user has the Describe access. And if the does not have
> Describe access to any group, ListGroupResponse will return an empty list
> with no error.
>
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error. Do you think this could be a problem for
> this KIP?
>
> Thanks,
> Dong
>
>
> On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> vahidhashemian@us.ibm.com> wrote:
>
> > Completing the subject line :)
> >
> >
> >
> > From: "Vahid S Hashemian" <va...@us.ibm.com>
> > To: dev <de...@kafka.apache.org>
> > Date: 11/29/2017 03:17 PM
> > Subject: [DISCUSS] KIP-231:
> >
> >
> >
> > Hi everyone,
> >
> > I started KIP-231 to propose a small change to the required ACL of
> > ListGroups API (in response to KAFKA-5638):
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> > pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> > 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=
> > jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJ
> > c7uSVcviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=
> > eu378oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> >
> > Your feedback and suggestions are welcome!
> >
> > Thanks.
> > --Vahid
> >
> >
> >
> >
> >
> >
> >
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Dong Lin <li...@gmail.com>.
Hey Vahid,
Thank you for the explanation.
I am still wondering whether you have a specific use-case for user to be
able to list the groups for which he/she has Describe access. I tried to
think through this but couldn't find a specific use-case for this feature
yet. I understand that admin of the Kafka cluster may want to know the
number of consumer groups in the cluster because the number of consumer
groups can affect the performance of the Kafka service. But this motivation
does not hold for non-admin user. I image that a typical user probably
already knows the group it wants to access, in which case he/she can
use DescribeGroupsRequest to query the information of this group. On the
other hand, if user does not already know the list of groups it has access
it, I don't know how user is going to use this information after he/she is
given a list of strings as the group Id. Could you help me understand
whether there is use-case for this feature?
Alternatively, maybe this KIP is solely driven by the idea that user should
be able to list the groups for which he/she has Describe access. In this
case, do you think it would be reasonable for the external service which is
used to store the ACL to provide this capability of query groupId list
based on the user credential?
Thanks,
Dong
On Mon, Dec 4, 2017 at 3:42 PM, Vahid S Hashemian <vahidhashemian@us.ibm.com
> wrote:
> Hi Dong,
>
> Thanks for reviewing the KIP and providing your feedback.
>
> As you mentioned, the KIP suggests a modification to the semantics of
> ListGroupResponse.
> The main reason for submitting the KIP and suggesting a modification is
> that there are issues with the current one (further explained in the KIP):
> 1) listing groups is an all or nothing action, and 2) a Read access on a
> group does not mean the user can list that group.
>
> I understand the solution outlined in the KIP may not be the best one, and
> I have been debating that myself even after opening it up for discussion.
> That solution was the one discussed with Jason on the corresponding JIRA.
> I would also prefer an authorization
> error in case none of the two accesses are in place, which should, at
> least, partially address your concern (the ACL would filter, but would
> also return authorization errors if needed). If you can think of something
> on that front please let me know.
>
> Regarding the use case you asked for: if I'm just a user in the cluster
> sharing it with other users, I should be able to see the groups of my
> consumers without being exposed to other users' groups in the cluster.
>
> I hope I addressed your concerns. If I did not, or if I missed anything,
> please let me know. Thanks.
>
> Regards.
> --Vahid
>
>
>
>
> From: Dong Lin <li...@gmail.com>
> To: dev@kafka.apache.org
> Date: 12/04/2017 01:43 PM
> Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> ListGroups API
>
>
>
> I forgot another question. Can you provide a use-case where a user wants
> to
> list all groups for which he/she has the Describe access?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
>
> > Hey Vahid,
> >
> > Thanks for the KIP. If I understand the you correctly, you want client
> to
> > be able to list all the groups for which it currently has the describe
> > access.
> >
> > As of now the ListGroupRequest does not allow user to specify the group.
> > If user does not have the Describe Cluster access, ListGroupResponse
> will
> > return error. This KIP proposes to change the semantics of
> > ListGroupsResponse such that ListGroupResponse will return the subset of
> > groups for which the user has the Describe access. And if the does not
> have
> > Describe access to any group, ListGroupResponse will return an empty
> list
> > with no error.
> >
> > In my opinion this changes the semantics of ListGroupsResponse in a
> > counter-intuitive way. Usually we use the ACL to determine whether the
> > operation on the specified object can be performed or not. The response
> > should provide either an error message or the result for the specified
> > object. I couldn't remember a case where the ACL is used to filter the
> > result without providing error. Do you think this could be a problem for
> > this KIP?
> >
> > Thanks,
> > Dong
> >
> >
> > On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> > vahidhashemian@us.ibm.com> wrote:
> >
> >> Completing the subject line :)
> >>
> >>
> >>
> >> From: "Vahid S Hashemian" <va...@us.ibm.com>
> >> To: dev <de...@kafka.apache.org>
> >> Date: 11/29/2017 03:17 PM
> >> Subject: [DISCUSS] KIP-231:
> >>
> >>
> >>
> >> Hi everyone,
> >>
> >> I started KIP-231 to propose a small change to the required ACL of
> >> ListGroups API (in response to KAFKA-5638):
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> >> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> >> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
> >> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
> >> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
> >> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> >>
> >> Your feedback and suggestions are welcome!
> >>
> >> Thanks.
> >> --Vahid
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Dong Lin <li...@gmail.com>.
Hey Vahid, Ismael,
Good point regarding the similar filter in OffsetFetchResponse
and TopicMetadataResponse. This addresses my concern.
Thanks,
Dong
On Tue, Dec 5, 2017 at 2:42 PM, Vahid S Hashemian <vahidhashemian@us.ibm.com
> wrote:
> Hi Dong,
>
> Thanks for your message.
>
> My driving point for this KIP was a -seemingly- broken logic rather than a
> particular use case.
> I also considered the semantics of the ListGroups API and its expected
> behavior: To me, a user who can fetch/commit offsets in some consumer
> groups should be able to run ListGroups and see those groups (and nothing
> else) in the result. This expected behavior conveys (kind of) a use case
> by itself.
>
> Regarding your second question, I see the external service for managing
> ACLs should not be directly concerned with any Kafka specific
> functionality.
> Plus, as Ismael mentioned, we already implement something very similar for
> the TopicMetadataRequest API (when metadata of all topics is requested).
>
> Thanks again for reviewing the KIP closely and sharing your feedback.
>
> Regards.
> --Vahid
>
>
>
>
> From: Dong Lin <li...@gmail.com>
> To: dev@kafka.apache.org
> Date: 12/04/2017 04:52 PM
> Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> ListGroups API
>
>
>
> Hey Vahid,
>
> Thank you for the explanation.
>
> I am still wondering whether you have a specific use-case for user to be
> able to list the groups for which he/she has Describe access. I tried to
> think through this but couldn't find a specific use-case for this feature
> yet. I understand that admin of the Kafka cluster may want to know the
> number of consumer groups in the cluster because the number of consumer
> groups can affect the performance of the Kafka service. But this
> motivation
> does not hold for non-admin user. I image that a typical user probably
> already knows the group it wants to access, in which case he/she can
> use DescribeGroupsRequest to query the information of this group. On the
> other hand, if user does not already know the list of groups it has access
> it, I don't know how user is going to use this information after he/she is
> given a list of strings as the group Id. Could you help me understand
> whether there is use-case for this feature?
>
> Alternatively, maybe this KIP is solely driven by the idea that user
> should
> be able to list the groups for which he/she has Describe access. In this
> case, do you think it would be reasonable for the external service which
> is
> used to store the ACL to provide this capability of query groupId list
> based on the user credential?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 3:42 PM, Vahid S Hashemian
> <vahidhashemian@us.ibm.com
> > wrote:
>
> > Hi Dong,
> >
> > Thanks for reviewing the KIP and providing your feedback.
> >
> > As you mentioned, the KIP suggests a modification to the semantics of
> > ListGroupResponse.
> > The main reason for submitting the KIP and suggesting a modification is
> > that there are issues with the current one (further explained in the
> KIP):
> > 1) listing groups is an all or nothing action, and 2) a Read access on a
> > group does not mean the user can list that group.
> >
> > I understand the solution outlined in the KIP may not be the best one,
> and
> > I have been debating that myself even after opening it up for
> discussion.
> > That solution was the one discussed with Jason on the corresponding
> JIRA.
> > I would also prefer an authorization
> > error in case none of the two accesses are in place, which should, at
> > least, partially address your concern (the ACL would filter, but would
> > also return authorization errors if needed). If you can think of
> something
> > on that front please let me know.
> >
> > Regarding the use case you asked for: if I'm just a user in the cluster
> > sharing it with other users, I should be able to see the groups of my
> > consumers without being exposed to other users' groups in the cluster.
> >
> > I hope I addressed your concerns. If I did not, or if I missed anything,
> > please let me know. Thanks.
> >
> > Regards.
> > --Vahid
> >
> >
> >
> >
> > From: Dong Lin <li...@gmail.com>
> > To: dev@kafka.apache.org
> > Date: 12/04/2017 01:43 PM
> > Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> > ListGroups API
> >
> >
> >
> > I forgot another question. Can you provide a use-case where a user wants
> > to
> > list all groups for which he/she has the Describe access?
> >
> > Thanks,
> > Dong
> >
> >
> > On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
> >
> > > Hey Vahid,
> > >
> > > Thanks for the KIP. If I understand the you correctly, you want client
> > to
> > > be able to list all the groups for which it currently has the describe
> > > access.
> > >
> > > As of now the ListGroupRequest does not allow user to specify the
> group.
> > > If user does not have the Describe Cluster access, ListGroupResponse
> > will
> > > return error. This KIP proposes to change the semantics of
> > > ListGroupsResponse such that ListGroupResponse will return the subset
> of
> > > groups for which the user has the Describe access. And if the does not
> > have
> > > Describe access to any group, ListGroupResponse will return an empty
> > list
> > > with no error.
> > >
> > > In my opinion this changes the semantics of ListGroupsResponse in a
> > > counter-intuitive way. Usually we use the ACL to determine whether the
> > > operation on the specified object can be performed or not. The
> response
> > > should provide either an error message or the result for the specified
> > > object. I couldn't remember a case where the ACL is used to filter the
> > > result without providing error. Do you think this could be a problem
> for
> > > this KIP?
> > >
> > > Thanks,
> > > Dong
> > >
> > >
> > > On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> > > vahidhashemian@us.ibm.com> wrote:
> > >
> > >> Completing the subject line :)
> > >>
> > >>
> > >>
> > >> From: "Vahid S Hashemian" <va...@us.ibm.com>
> > >> To: dev <de...@kafka.apache.org>
> > >> Date: 11/29/2017 03:17 PM
> > >> Subject: [DISCUSS] KIP-231:
> > >>
> > >>
> > >>
> > >> Hi everyone,
> > >>
> > >> I started KIP-231 to propose a small change to the required ACL of
> > >> ListGroups API (in response to KAFKA-5638):
> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> > >> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> > >> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
> > >> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
> > >> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
> > >> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> > >>
> > >> Your feedback and suggestions are welcome!
> > >>
> > >> Thanks.
> > >> --Vahid
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> >
> >
> >
> >
> >
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Dong,
Thanks for your message.
My driving point for this KIP was a -seemingly- broken logic rather than a
particular use case.
I also considered the semantics of the ListGroups API and its expected
behavior: To me, a user who can fetch/commit offsets in some consumer
groups should be able to run ListGroups and see those groups (and nothing
else) in the result. This expected behavior conveys (kind of) a use case
by itself.
Regarding your second question, I see the external service for managing
ACLs should not be directly concerned with any Kafka specific
functionality.
Plus, as Ismael mentioned, we already implement something very similar for
the TopicMetadataRequest API (when metadata of all topics is requested).
Thanks again for reviewing the KIP closely and sharing your feedback.
Regards.
--Vahid
From: Dong Lin <li...@gmail.com>
To: dev@kafka.apache.org
Date: 12/04/2017 04:52 PM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
Hey Vahid,
Thank you for the explanation.
I am still wondering whether you have a specific use-case for user to be
able to list the groups for which he/she has Describe access. I tried to
think through this but couldn't find a specific use-case for this feature
yet. I understand that admin of the Kafka cluster may want to know the
number of consumer groups in the cluster because the number of consumer
groups can affect the performance of the Kafka service. But this
motivation
does not hold for non-admin user. I image that a typical user probably
already knows the group it wants to access, in which case he/she can
use DescribeGroupsRequest to query the information of this group. On the
other hand, if user does not already know the list of groups it has access
it, I don't know how user is going to use this information after he/she is
given a list of strings as the group Id. Could you help me understand
whether there is use-case for this feature?
Alternatively, maybe this KIP is solely driven by the idea that user
should
be able to list the groups for which he/she has Describe access. In this
case, do you think it would be reasonable for the external service which
is
used to store the ACL to provide this capability of query groupId list
based on the user credential?
Thanks,
Dong
On Mon, Dec 4, 2017 at 3:42 PM, Vahid S Hashemian
<vahidhashemian@us.ibm.com
> wrote:
> Hi Dong,
>
> Thanks for reviewing the KIP and providing your feedback.
>
> As you mentioned, the KIP suggests a modification to the semantics of
> ListGroupResponse.
> The main reason for submitting the KIP and suggesting a modification is
> that there are issues with the current one (further explained in the
KIP):
> 1) listing groups is an all or nothing action, and 2) a Read access on a
> group does not mean the user can list that group.
>
> I understand the solution outlined in the KIP may not be the best one,
and
> I have been debating that myself even after opening it up for
discussion.
> That solution was the one discussed with Jason on the corresponding
JIRA.
> I would also prefer an authorization
> error in case none of the two accesses are in place, which should, at
> least, partially address your concern (the ACL would filter, but would
> also return authorization errors if needed). If you can think of
something
> on that front please let me know.
>
> Regarding the use case you asked for: if I'm just a user in the cluster
> sharing it with other users, I should be able to see the groups of my
> consumers without being exposed to other users' groups in the cluster.
>
> I hope I addressed your concerns. If I did not, or if I missed anything,
> please let me know. Thanks.
>
> Regards.
> --Vahid
>
>
>
>
> From: Dong Lin <li...@gmail.com>
> To: dev@kafka.apache.org
> Date: 12/04/2017 01:43 PM
> Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> ListGroups API
>
>
>
> I forgot another question. Can you provide a use-case where a user wants
> to
> list all groups for which he/she has the Describe access?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
>
> > Hey Vahid,
> >
> > Thanks for the KIP. If I understand the you correctly, you want client
> to
> > be able to list all the groups for which it currently has the describe
> > access.
> >
> > As of now the ListGroupRequest does not allow user to specify the
group.
> > If user does not have the Describe Cluster access, ListGroupResponse
> will
> > return error. This KIP proposes to change the semantics of
> > ListGroupsResponse such that ListGroupResponse will return the subset
of
> > groups for which the user has the Describe access. And if the does not
> have
> > Describe access to any group, ListGroupResponse will return an empty
> list
> > with no error.
> >
> > In my opinion this changes the semantics of ListGroupsResponse in a
> > counter-intuitive way. Usually we use the ACL to determine whether the
> > operation on the specified object can be performed or not. The
response
> > should provide either an error message or the result for the specified
> > object. I couldn't remember a case where the ACL is used to filter the
> > result without providing error. Do you think this could be a problem
for
> > this KIP?
> >
> > Thanks,
> > Dong
> >
> >
> > On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> > vahidhashemian@us.ibm.com> wrote:
> >
> >> Completing the subject line :)
> >>
> >>
> >>
> >> From: "Vahid S Hashemian" <va...@us.ibm.com>
> >> To: dev <de...@kafka.apache.org>
> >> Date: 11/29/2017 03:17 PM
> >> Subject: [DISCUSS] KIP-231:
> >>
> >>
> >>
> >> Hi everyone,
> >>
> >> I started KIP-231 to propose a small change to the required ACL of
> >> ListGroups API (in response to KAFKA-5638):
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> >> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> >> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
> >> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
> >> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
> >> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> >>
> >> Your feedback and suggestions are welcome!
> >>
> >> Thanks.
> >> --Vahid
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
I forgot to mention that OffsetFetch is another example that is affected
by such filtering.
Starting from version 2 and when offsets of all topics are requested, only
topics to which user has describe access will appear in the result.
If user does not have describe access to any topic (assuming Describe
Group access is in place) an empty list is returned without any
authorization error.
--Vahid
From: "Vahid S Hashemian" <va...@us.ibm.com>
To: dev@kafka.apache.org
Date: 12/05/2017 02:42 PM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
Hi Dong,
Thanks for your message.
My driving point for this KIP was a -seemingly- broken logic rather than a
particular use case.
I also considered the semantics of the ListGroups API and its expected
behavior: To me, a user who can fetch/commit offsets in some consumer
groups should be able to run ListGroups and see those groups (and nothing
else) in the result. This expected behavior conveys (kind of) a use case
by itself.
Regarding your second question, I see the external service for managing
ACLs should not be directly concerned with any Kafka specific
functionality.
Plus, as Ismael mentioned, we already implement something very similar for
the TopicMetadataRequest API (when metadata of all topics is requested).
Thanks again for reviewing the KIP closely and sharing your feedback.
Regards.
--Vahid
From: Dong Lin <li...@gmail.com>
To: dev@kafka.apache.org
Date: 12/04/2017 04:52 PM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
Hey Vahid,
Thank you for the explanation.
I am still wondering whether you have a specific use-case for user to be
able to list the groups for which he/she has Describe access. I tried to
think through this but couldn't find a specific use-case for this feature
yet. I understand that admin of the Kafka cluster may want to know the
number of consumer groups in the cluster because the number of consumer
groups can affect the performance of the Kafka service. But this
motivation
does not hold for non-admin user. I image that a typical user probably
already knows the group it wants to access, in which case he/she can
use DescribeGroupsRequest to query the information of this group. On the
other hand, if user does not already know the list of groups it has access
it, I don't know how user is going to use this information after he/she is
given a list of strings as the group Id. Could you help me understand
whether there is use-case for this feature?
Alternatively, maybe this KIP is solely driven by the idea that user
should
be able to list the groups for which he/she has Describe access. In this
case, do you think it would be reasonable for the external service which
is
used to store the ACL to provide this capability of query groupId list
based on the user credential?
Thanks,
Dong
On Mon, Dec 4, 2017 at 3:42 PM, Vahid S Hashemian
<vahidhashemian@us.ibm.com
> wrote:
> Hi Dong,
>
> Thanks for reviewing the KIP and providing your feedback.
>
> As you mentioned, the KIP suggests a modification to the semantics of
> ListGroupResponse.
> The main reason for submitting the KIP and suggesting a modification is
> that there are issues with the current one (further explained in the
KIP):
> 1) listing groups is an all or nothing action, and 2) a Read access on a
> group does not mean the user can list that group.
>
> I understand the solution outlined in the KIP may not be the best one,
and
> I have been debating that myself even after opening it up for
discussion.
> That solution was the one discussed with Jason on the corresponding
JIRA.
> I would also prefer an authorization
> error in case none of the two accesses are in place, which should, at
> least, partially address your concern (the ACL would filter, but would
> also return authorization errors if needed). If you can think of
something
> on that front please let me know.
>
> Regarding the use case you asked for: if I'm just a user in the cluster
> sharing it with other users, I should be able to see the groups of my
> consumers without being exposed to other users' groups in the cluster.
>
> I hope I addressed your concerns. If I did not, or if I missed anything,
> please let me know. Thanks.
>
> Regards.
> --Vahid
>
>
>
>
> From: Dong Lin <li...@gmail.com>
> To: dev@kafka.apache.org
> Date: 12/04/2017 01:43 PM
> Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> ListGroups API
>
>
>
> I forgot another question. Can you provide a use-case where a user wants
> to
> list all groups for which he/she has the Describe access?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
>
> > Hey Vahid,
> >
> > Thanks for the KIP. If I understand the you correctly, you want client
> to
> > be able to list all the groups for which it currently has the describe
> > access.
> >
> > As of now the ListGroupRequest does not allow user to specify the
group.
> > If user does not have the Describe Cluster access, ListGroupResponse
> will
> > return error. This KIP proposes to change the semantics of
> > ListGroupsResponse such that ListGroupResponse will return the subset
of
> > groups for which the user has the Describe access. And if the does not
> have
> > Describe access to any group, ListGroupResponse will return an empty
> list
> > with no error.
> >
> > In my opinion this changes the semantics of ListGroupsResponse in a
> > counter-intuitive way. Usually we use the ACL to determine whether the
> > operation on the specified object can be performed or not. The
response
> > should provide either an error message or the result for the specified
> > object. I couldn't remember a case where the ACL is used to filter the
> > result without providing error. Do you think this could be a problem
for
> > this KIP?
> >
> > Thanks,
> > Dong
> >
> >
> > On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> > vahidhashemian@us.ibm.com> wrote:
> >
> >> Completing the subject line :)
> >>
> >>
> >>
> >> From: "Vahid S Hashemian" <va...@us.ibm.com>
> >> To: dev <de...@kafka.apache.org>
> >> Date: 11/29/2017 03:17 PM
> >> Subject: [DISCUSS] KIP-231:
> >>
> >>
> >>
> >> Hi everyone,
> >>
> >> I started KIP-231 to propose a small change to the required ACL of
> >> ListGroups API (in response to KAFKA-5638):
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> >> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> >> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
> >> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
> >> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
> >> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> >>
> >> Your feedback and suggestions are welcome!
> >>
> >> Thanks.
> >> --Vahid
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Dong,
Thanks for reviewing the KIP and providing your feedback.
As you mentioned, the KIP suggests a modification to the semantics of
ListGroupResponse.
The main reason for submitting the KIP and suggesting a modification is
that there are issues with the current one (further explained in the KIP):
1) listing groups is an all or nothing action, and 2) a Read access on a
group does not mean the user can list that group.
I understand the solution outlined in the KIP may not be the best one, and
I have been debating that myself even after opening it up for discussion.
That solution was the one discussed with Jason on the corresponding JIRA.
I would also prefer an authorization
error in case none of the two accesses are in place, which should, at
least, partially address your concern (the ACL would filter, but would
also return authorization errors if needed). If you can think of something
on that front please let me know.
Regarding the use case you asked for: if I'm just a user in the cluster
sharing it with other users, I should be able to see the groups of my
consumers without being exposed to other users' groups in the cluster.
I hope I addressed your concerns. If I did not, or if I missed anything,
please let me know. Thanks.
Regards.
--Vahid
From: Dong Lin <li...@gmail.com>
To: dev@kafka.apache.org
Date: 12/04/2017 01:43 PM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
I forgot another question. Can you provide a use-case where a user wants
to
list all groups for which he/she has the Describe access?
Thanks,
Dong
On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
> Hey Vahid,
>
> Thanks for the KIP. If I understand the you correctly, you want client
to
> be able to list all the groups for which it currently has the describe
> access.
>
> As of now the ListGroupRequest does not allow user to specify the group.
> If user does not have the Describe Cluster access, ListGroupResponse
will
> return error. This KIP proposes to change the semantics of
> ListGroupsResponse such that ListGroupResponse will return the subset of
> groups for which the user has the Describe access. And if the does not
have
> Describe access to any group, ListGroupResponse will return an empty
list
> with no error.
>
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error. Do you think this could be a problem for
> this KIP?
>
> Thanks,
> Dong
>
>
> On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> vahidhashemian@us.ibm.com> wrote:
>
>> Completing the subject line :)
>>
>>
>>
>> From: "Vahid S Hashemian" <va...@us.ibm.com>
>> To: dev <de...@kafka.apache.org>
>> Date: 11/29/2017 03:17 PM
>> Subject: [DISCUSS] KIP-231:
>>
>>
>>
>> Hi everyone,
>>
>> I started KIP-231 to propose a small change to the required ACL of
>> ListGroups API (in response to KAFKA-5638):
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
>> pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
>> 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=jf
>> _iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJc7uSV
>> cviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=eu378
>> oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
>>
>> Your feedback and suggestions are welcome!
>>
>> Thanks.
>> --Vahid
>>
>>
>>
>>
>>
>>
>>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Ismael,
Thanks for the pointer.
That's a good example of how we already implemented a similar filtering.
--Vahid
From: Ismael Juma <is...@juma.me.uk>
To: dev <de...@kafka.apache.org>
Date: 12/05/2017 01:24 AM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
Sent by: ismaelj@gmail.com
One comment below.
On Mon, Dec 4, 2017 at 11:40 PM, Dong Lin <li...@gmail.com> wrote:
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error.
>
We do this in the metadata request that requests all topics. We just
return
the topics that the user has describe access to.
Ismael
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Ted Yu <yu...@gmail.com>.
I saw your reply in the other email.
I meant that permissions the person managing brokers gives the user should
be known a priori.
Otherwise there would be security hole somewhere.
Cheers
On Mon, Dec 4, 2017 at 3:47 PM, Vahid S Hashemian <vahidhashemian@us.ibm.com
> wrote:
> Hi Ted,
>
> Thanks for the feedback. I tried to address Dong's comments on an earlier
> email. I hope that addresses your concern too.
> If not, please continue with the discussion until we hopefully agree on
> what (if anything) needs to be done.
>
> I'm also not sure if I understand your point about "The user should know
> which group(s) he / she is allowed to describe." in relationship to the
> KIP. Perhaps you can clarify?
>
> Thanks.
> --Vahid
>
>
>
>
> From: Ted Yu <yu...@gmail.com>
> To: dev@kafka.apache.org
> Date: 12/04/2017 02:01 PM
> Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
> ListGroups API
>
>
>
> I agree with Dong on maintaining the semantics.
>
> The user should know which group(s) he / she is allowed to describe.
>
> Cheers
>
> On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
>
> > Hey Vahid,
> >
> > Thanks for the KIP. If I understand the you correctly, you want client
> to
> > be able to list all the groups for which it currently has the describe
> > access.
> >
> > As of now the ListGroupRequest does not allow user to specify the group.
> If
> > user does not have the Describe Cluster access, ListGroupResponse will
> > return error. This KIP proposes to change the semantics of
> > ListGroupsResponse such that ListGroupResponse will return the subset of
> > groups for which the user has the Describe access. And if the does not
> have
> > Describe access to any group, ListGroupResponse will return an empty
> list
> > with no error.
> >
> > In my opinion this changes the semantics of ListGroupsResponse in a
> > counter-intuitive way. Usually we use the ACL to determine whether the
> > operation on the specified object can be performed or not. The response
> > should provide either an error message or the result for the specified
> > object. I couldn't remember a case where the ACL is used to filter the
> > result without providing error. Do you think this could be a problem for
> > this KIP?
> >
> > Thanks,
> > Dong
> >
> >
> > On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> > vahidhashemian@us.ibm.com> wrote:
> >
> > > Completing the subject line :)
> > >
> > >
> > >
> > > From: "Vahid S Hashemian" <va...@us.ibm.com>
> > > To: dev <de...@kafka.apache.org>
> > > Date: 11/29/2017 03:17 PM
> > > Subject: [DISCUSS] KIP-231:
> > >
> > >
> > >
> > > Hi everyone,
> > >
> > > I started KIP-231 to propose a small change to the required ACL of
> > > ListGroups API (in response to KAFKA-5638):
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> > > pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> > > 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=
> > > jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJ
> > > c7uSVcviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=
> > > eu378oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> > >
> > > Your feedback and suggestions are welcome!
> > >
> > > Thanks.
> > > --Vahid
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
>
>
Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Hi Ted,
Thanks for the feedback. I tried to address Dong's comments on an earlier
email. I hope that addresses your concern too.
If not, please continue with the discussion until we hopefully agree on
what (if anything) needs to be done.
I'm also not sure if I understand your point about "The user should know
which group(s) he / she is allowed to describe." in relationship to the
KIP. Perhaps you can clarify?
Thanks.
--Vahid
From: Ted Yu <yu...@gmail.com>
To: dev@kafka.apache.org
Date: 12/04/2017 02:01 PM
Subject: Re: [DISCUSS] KIP-231: Improve the Required ACL of
ListGroups API
I agree with Dong on maintaining the semantics.
The user should know which group(s) he / she is allowed to describe.
Cheers
On Mon, Dec 4, 2017 at 1:40 PM, Dong Lin <li...@gmail.com> wrote:
> Hey Vahid,
>
> Thanks for the KIP. If I understand the you correctly, you want client
to
> be able to list all the groups for which it currently has the describe
> access.
>
> As of now the ListGroupRequest does not allow user to specify the group.
If
> user does not have the Describe Cluster access, ListGroupResponse will
> return error. This KIP proposes to change the semantics of
> ListGroupsResponse such that ListGroupResponse will return the subset of
> groups for which the user has the Describe access. And if the does not
have
> Describe access to any group, ListGroupResponse will return an empty
list
> with no error.
>
> In my opinion this changes the semantics of ListGroupsResponse in a
> counter-intuitive way. Usually we use the ACL to determine whether the
> operation on the specified object can be performed or not. The response
> should provide either an error message or the result for the specified
> object. I couldn't remember a case where the ACL is used to filter the
> result without providing error. Do you think this could be a problem for
> this KIP?
>
> Thanks,
> Dong
>
>
> On Wed, Nov 29, 2017 at 3:18 PM, Vahid S Hashemian <
> vahidhashemian@us.ibm.com> wrote:
>
> > Completing the subject line :)
> >
> >
> >
> > From: "Vahid S Hashemian" <va...@us.ibm.com>
> > To: dev <de...@kafka.apache.org>
> > Date: 11/29/2017 03:17 PM
> > Subject: [DISCUSS] KIP-231:
> >
> >
> >
> > Hi everyone,
> >
> > I started KIP-231 to propose a small change to the required ACL of
> > ListGroups API (in response to KAFKA-5638):
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a
> > pache.org_confluence_display_KAFKA_KIP-2D231-253A-2BImprove-
> > 2Bthe-2BRequired-2BACL-2Bof-2BListGroups-2BAPI&d=DwIFAg&c=
> > jf_iaSHvJObTbx-siA1ZOg&r=Q_itwloTQj3_xUKl7Nzswo6KE4Nj-kjJ
> > c7uSVcviKUc&m=XjHVTsIl7t-z0NBesB0U-ptMMm6mmpy3UqS8TjJM5yM&s=
> > eu378oaLvC0Wzbfcz15Rwo4nqdrO11ENLK6v9Kq9Z6w&e=
> >
> > Your feedback and suggestions are welcome!
> >
> > Thanks.
> > --Vahid
> >
> >
> >
> >
> >
> >
> >
>