You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Rajini Sivaram <ra...@googlemail.com> on 2016/06/07 20:55:44 UTC

[VOTE] KIP-55: Secure quotas for authenticated users

I would like to initiate the vote for KIP-55.

The KIP details are here: KIP-55: Secure quotas for authenticated users
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users>
.

The JIRA  KAFKA-3492  <https://issues.apache.org/jira/browse/KAFKA-3492>has
a draft PR here: https://github.com/apache/kafka/pull/1256.

Thank you...


Regards,

Rajini

Re: [DISCUSS] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
The PR for this KIP is ready for review. JIRA is
https://issues.apache.org/jira/browse/KAFKA-3492, PR is
https://github.com/apache/kafka/pull/1753.

Thanks,

Rajini

On Tue, Aug 9, 2016 at 1:06 PM, Rajini Sivaram <rajinisivaram@googlemail.com
> wrote:

> Hi Tom,
>
> Have updated the KIP wiki. Will submit a PR later this week.
>
> Regards,
>
> Rajini
>
> On Tue, Aug 9, 2016 at 12:30 PM, Tom Crayford <tc...@heroku.com>
> wrote:
>
>> Seeing as voting passed on this, can somebody with access update the wiki?
>>
>> Is there code for this KIP in a PR somewhere that needs merging?
>>
>> Thanks
>> Tom Crayford
>> Heroku Kafka
>>
>> On Friday, 1 July 2016, Rajini Sivaram <ra...@googlemail.com>
>> wrote:
>>
>> > Thank you, Jun.
>> >
>> > Hi all,
>> >
>> > Please let me know if you have any comments or suggestions on the
>> updated
>> > KIP. If there are no objections, I will initiate voting next week.
>> >
>> > Thank you...
>> >
>> >
>> > On Thu, Jun 30, 2016 at 10:37 PM, Jun Rao <jun@confluent.io
>> <javascript:;>>
>> > wrote:
>> >
>> > > Rajini,
>> > >
>> > > The latest wiki looks good to me. Perhaps you want to ask other
>> people to
>> > > also take a look and then we can start the voting.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > > On Tue, Jun 28, 2016 at 6:27 AM, Rajini Sivaram <
>> > > rajinisivaram@googlemail.com <javascript:;>> wrote:
>> > >
>> > > > Jun,
>> > > >
>> > > > Thank you for the review. I have changed all default property
>> configs
>> > to
>> > > be
>> > > > stored with the node name <default>. So the defaults are
>> > > > /config/clients/<default> for default client-id quota,
>> > > > /config/users/<default> for default user quota and
>> > > > /config/users/<default/clients/<default> for default <user,
>> client-id>
>> > > > quota. Hope that makes sense.
>> > > >
>> > > > On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <jun@confluent.io
>> > <javascript:;>> wrote:
>> > > >
>> > > > > Rajini,
>> > > > >
>> > > > > Thanks for the update. Looks good to me. My only comment is that
>> > > > > instead of /config/users/<default>/clients,
>> > > > > would it be better to represent it as
>> > > > > /config/users/<default>/clients/<default>
>> > > > > so that it's more consistent?
>> > > > >
>> > > > > Jun
>> > > > >
>> > > > >
>> > > > > On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
>> > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
>> > > > >
>> > > > > > Jun,
>> > > > > >
>> > > > > > Yes, I agree that it makes sense to retain the existing
>> semantics
>> > for
>> > > > > > client-id quotas for compatibility. Especially if we can provide
>> > the
>> > > > > option
>> > > > > > to enable secure client-id quotas for multi-user clusters as
>> well.
>> > > > > >
>> > > > > > I have updated the KIP - each of these levels can have defaults
>> as
>> > > well
>> > > > > as
>> > > > > > specific entries:
>> > > > > >
>> > > > > >    - /config/clients : Insecure <client-id> quotas with the same
>> > > > > semantics
>> > > > > >    as now
>> > > > > >    - /config/users: User quotas
>> > > > > >    - /config/users/userA/clients: <user, client-id> quotas for
>> > userA
>> > > > > >    - /config/users/<default>/clients: Default <user, client-id>
>> > > quotas
>> > > > > >
>> > > > > > Now it is fully flexible as well as compatible with the current
>> > > > > > implementation. I used /config/users/<default>/clients rather
>> than
>> > > > > > /config/users/clients since "clients" is a valid (unlikely, but
>> > still
>> > > > > > possible) user principal. I used <default>, but it could be
>> > anything
>> > > > that
>> > > > > > is a valid Zookeeper node name, but not a valid URL-encoded
>> name.
>> > > > > >
>> > > > > > Thank you,
>> > > > > >
>> > > > > > Rajini
>> > > > > >
>> > > > > > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <jun@confluent.io
>> > <javascript:;>> wrote:
>> > > > > >
>> > > > > > > Hi, Rajini,
>> > > > > > >
>> > > > > > > For the following statements, would it be better to allocate
>> the
>> > > > quota
>> > > > > to
>> > > > > > > all connections whose client-id is clientX? This way, existing
>> > > > > client-id
>> > > > > > > quotas are fully compatible in the new release whether the
>> > cluster
>> > > is
>> > > > > in
>> > > > > > a
>> > > > > > > single user or multi-user environment.
>> > > > > > >
>> > > > > > > 4. If client-id quota override is defined for clientX in
>> > > > > > > /config/clients/clientX, this quota is allocated for the sole
>> use
>> > > of
>> > > > > > > <userN,
>> > > > > > > clientX>
>> > > > > > > 5. If dynamic client-id default is configured in
>> /config/clients,
>> > > > this
>> > > > > > > default quota is allocated for the sole use of <userN,
>> clientX>
>> > > > > > > 6. If quota.producer.default is configured for the broker in
>> > > > > > > server.properties, this default quota is allocated for the
>> sole
>> > use
>> > > > of
>> > > > > > > <userN,
>> > > > > > > clientX>
>> > > > > > >
>> > > > > > > We can potentially add a default quota for both user and
>> client
>> > at
>> > > > path
>> > > > > > > /config/users/clients?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > >
>> > > > > > > Jun
>> > > > > > >
>> > > > > > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
>> > > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
>> > > > > > >
>> > > > > > > > Ismael, Jun,
>> > > > > > > >
>> > > > > > > > Thank you both for the feedback. Have updated the KIP to add
>> > > > dynamic
>> > > > > > > > default quotas for client-id with deprecation of existing
>> > static
>> > > > > > default
>> > > > > > > > properties.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <jun@confluent.io
>> > <javascript:;>>
>> > > > wrote:
>> > > > > > > >
>> > > > > > > > > Yes, for consistency, perhaps we can allow client-id
>> quota to
>> > > be
>> > > > > > > > configured
>> > > > > > > > > dynamically too and mark the static config in the broker
>> as
>> > > > > > deprecated.
>> > > > > > > > If
>> > > > > > > > > both are set, the dynamic one wins.
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > >
>> > > > > > > > > Jun
>> > > > > > > > >
>> > > > > > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <
>> > > ismael@juma.me.uk <javascript:;>>
>> > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
>> > > > > > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > It is actually quite tempting to do the same for
>> > client-id
>> > > > > quotas
>> > > > > > > as
>> > > > > > > > > > well,
>> > > > > > > > > > > but I suppose we can't break existing users who have
>> > > > configured
>> > > > > > > > > defaults
>> > > > > > > > > > in
>> > > > > > > > > > > server.properties and providing two ways of setting
>> > > client-id
>> > > > > > > > defaults
>> > > > > > > > > > > would be just too confusing.
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Using two different approaches for client-id versus user
>> > > quota
>> > > > > > > defaults
>> > > > > > > > > is
>> > > > > > > > > > also not great. We could deprecate the server.properties
>> > > > default
>> > > > > > > > configs
>> > > > > > > > > > for client-id quotas and remove them in the future. In
>> the
>> > > > > > meantime,
>> > > > > > > we
>> > > > > > > > > > would have to live with 2 level defaults.
>> > > > > > > > > >
>> > > > > > > > > > Jun, what are your thoughts on this?
>> > > > > > > > > >
>> > > > > > > > > > Ismael
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Regards,
>> > > > > > > >
>> > > > > > > > Rajini
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Regards,
>> > > > > >
>> > > > > > Rajini
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Rajini
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Rajini
>> >
>>
>
>
>
> --
> Regards,
>
> Rajini
>



-- 
Regards,

Rajini

Re: [DISCUSS] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Hi Tom,

Have updated the KIP wiki. Will submit a PR later this week.

Regards,

Rajini

On Tue, Aug 9, 2016 at 12:30 PM, Tom Crayford <tc...@heroku.com> wrote:

> Seeing as voting passed on this, can somebody with access update the wiki?
>
> Is there code for this KIP in a PR somewhere that needs merging?
>
> Thanks
> Tom Crayford
> Heroku Kafka
>
> On Friday, 1 July 2016, Rajini Sivaram <ra...@googlemail.com>
> wrote:
>
> > Thank you, Jun.
> >
> > Hi all,
> >
> > Please let me know if you have any comments or suggestions on the updated
> > KIP. If there are no objections, I will initiate voting next week.
> >
> > Thank you...
> >
> >
> > On Thu, Jun 30, 2016 at 10:37 PM, Jun Rao <jun@confluent.io
> <javascript:;>>
> > wrote:
> >
> > > Rajini,
> > >
> > > The latest wiki looks good to me. Perhaps you want to ask other people
> to
> > > also take a look and then we can start the voting.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 28, 2016 at 6:27 AM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > >
> > > > Jun,
> > > >
> > > > Thank you for the review. I have changed all default property configs
> > to
> > > be
> > > > stored with the node name <default>. So the defaults are
> > > > /config/clients/<default> for default client-id quota,
> > > > /config/users/<default> for default user quota and
> > > > /config/users/<default/clients/<default> for default <user,
> client-id>
> > > > quota. Hope that makes sense.
> > > >
> > > > On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <jun@confluent.io
> > <javascript:;>> wrote:
> > > >
> > > > > Rajini,
> > > > >
> > > > > Thanks for the update. Looks good to me. My only comment is that
> > > > > instead of /config/users/<default>/clients,
> > > > > would it be better to represent it as
> > > > > /config/users/<default>/clients/<default>
> > > > > so that it's more consistent?
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
> > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > > >
> > > > > > Jun,
> > > > > >
> > > > > > Yes, I agree that it makes sense to retain the existing semantics
> > for
> > > > > > client-id quotas for compatibility. Especially if we can provide
> > the
> > > > > option
> > > > > > to enable secure client-id quotas for multi-user clusters as
> well.
> > > > > >
> > > > > > I have updated the KIP - each of these levels can have defaults
> as
> > > well
> > > > > as
> > > > > > specific entries:
> > > > > >
> > > > > >    - /config/clients : Insecure <client-id> quotas with the same
> > > > > semantics
> > > > > >    as now
> > > > > >    - /config/users: User quotas
> > > > > >    - /config/users/userA/clients: <user, client-id> quotas for
> > userA
> > > > > >    - /config/users/<default>/clients: Default <user, client-id>
> > > quotas
> > > > > >
> > > > > > Now it is fully flexible as well as compatible with the current
> > > > > > implementation. I used /config/users/<default>/clients rather
> than
> > > > > > /config/users/clients since "clients" is a valid (unlikely, but
> > still
> > > > > > possible) user principal. I used <default>, but it could be
> > anything
> > > > that
> > > > > > is a valid Zookeeper node name, but not a valid URL-encoded name.
> > > > > >
> > > > > > Thank you,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > > > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <jun@confluent.io
> > <javascript:;>> wrote:
> > > > > >
> > > > > > > Hi, Rajini,
> > > > > > >
> > > > > > > For the following statements, would it be better to allocate
> the
> > > > quota
> > > > > to
> > > > > > > all connections whose client-id is clientX? This way, existing
> > > > > client-id
> > > > > > > quotas are fully compatible in the new release whether the
> > cluster
> > > is
> > > > > in
> > > > > > a
> > > > > > > single user or multi-user environment.
> > > > > > >
> > > > > > > 4. If client-id quota override is defined for clientX in
> > > > > > > /config/clients/clientX, this quota is allocated for the sole
> use
> > > of
> > > > > > > <userN,
> > > > > > > clientX>
> > > > > > > 5. If dynamic client-id default is configured in
> /config/clients,
> > > > this
> > > > > > > default quota is allocated for the sole use of <userN, clientX>
> > > > > > > 6. If quota.producer.default is configured for the broker in
> > > > > > > server.properties, this default quota is allocated for the sole
> > use
> > > > of
> > > > > > > <userN,
> > > > > > > clientX>
> > > > > > >
> > > > > > > We can potentially add a default quota for both user and client
> > at
> > > > path
> > > > > > > /config/users/clients?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > > > > >
> > > > > > > > Ismael, Jun,
> > > > > > > >
> > > > > > > > Thank you both for the feedback. Have updated the KIP to add
> > > > dynamic
> > > > > > > > default quotas for client-id with deprecation of existing
> > static
> > > > > > default
> > > > > > > > properties.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <jun@confluent.io
> > <javascript:;>>
> > > > wrote:
> > > > > > > >
> > > > > > > > > Yes, for consistency, perhaps we can allow client-id quota
> to
> > > be
> > > > > > > > configured
> > > > > > > > > dynamically too and mark the static config in the broker as
> > > > > > deprecated.
> > > > > > > > If
> > > > > > > > > both are set, the dynamic one wins.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <
> > > ismael@juma.me.uk <javascript:;>>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > > > > > > > >
> > > > > > > > > > > It is actually quite tempting to do the same for
> > client-id
> > > > > quotas
> > > > > > > as
> > > > > > > > > > well,
> > > > > > > > > > > but I suppose we can't break existing users who have
> > > > configured
> > > > > > > > > defaults
> > > > > > > > > > in
> > > > > > > > > > > server.properties and providing two ways of setting
> > > client-id
> > > > > > > > defaults
> > > > > > > > > > > would be just too confusing.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Using two different approaches for client-id versus user
> > > quota
> > > > > > > defaults
> > > > > > > > > is
> > > > > > > > > > also not great. We could deprecate the server.properties
> > > > default
> > > > > > > > configs
> > > > > > > > > > for client-id quotas and remove them in the future. In
> the
> > > > > > meantime,
> > > > > > > we
> > > > > > > > > > would have to live with 2 level defaults.
> > > > > > > > > >
> > > > > > > > > > Jun, what are your thoughts on this?
> > > > > > > > > >
> > > > > > > > > > Ismael
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Rajini
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [DISCUSS] KIP-55: Secure quotas for authenticated users

Posted by Tom Crayford <tc...@heroku.com>.
Seeing as voting passed on this, can somebody with access update the wiki?

Is there code for this KIP in a PR somewhere that needs merging?

Thanks
Tom Crayford
Heroku Kafka

On Friday, 1 July 2016, Rajini Sivaram <ra...@googlemail.com> wrote:

> Thank you, Jun.
>
> Hi all,
>
> Please let me know if you have any comments or suggestions on the updated
> KIP. If there are no objections, I will initiate voting next week.
>
> Thank you...
>
>
> On Thu, Jun 30, 2016 at 10:37 PM, Jun Rao <jun@confluent.io <javascript:;>>
> wrote:
>
> > Rajini,
> >
> > The latest wiki looks good to me. Perhaps you want to ask other people to
> > also take a look and then we can start the voting.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 28, 2016 at 6:27 AM, Rajini Sivaram <
> > rajinisivaram@googlemail.com <javascript:;>> wrote:
> >
> > > Jun,
> > >
> > > Thank you for the review. I have changed all default property configs
> to
> > be
> > > stored with the node name <default>. So the defaults are
> > > /config/clients/<default> for default client-id quota,
> > > /config/users/<default> for default user quota and
> > > /config/users/<default/clients/<default> for default <user, client-id>
> > > quota. Hope that makes sense.
> > >
> > > On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <jun@confluent.io
> <javascript:;>> wrote:
> > >
> > > > Rajini,
> > > >
> > > > Thanks for the update. Looks good to me. My only comment is that
> > > > instead of /config/users/<default>/clients,
> > > > would it be better to represent it as
> > > > /config/users/<default>/clients/<default>
> > > > so that it's more consistent?
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
> > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > >
> > > > > Jun,
> > > > >
> > > > > Yes, I agree that it makes sense to retain the existing semantics
> for
> > > > > client-id quotas for compatibility. Especially if we can provide
> the
> > > > option
> > > > > to enable secure client-id quotas for multi-user clusters as well.
> > > > >
> > > > > I have updated the KIP - each of these levels can have defaults as
> > well
> > > > as
> > > > > specific entries:
> > > > >
> > > > >    - /config/clients : Insecure <client-id> quotas with the same
> > > > semantics
> > > > >    as now
> > > > >    - /config/users: User quotas
> > > > >    - /config/users/userA/clients: <user, client-id> quotas for
> userA
> > > > >    - /config/users/<default>/clients: Default <user, client-id>
> > quotas
> > > > >
> > > > > Now it is fully flexible as well as compatible with the current
> > > > > implementation. I used /config/users/<default>/clients rather than
> > > > > /config/users/clients since "clients" is a valid (unlikely, but
> still
> > > > > possible) user principal. I used <default>, but it could be
> anything
> > > that
> > > > > is a valid Zookeeper node name, but not a valid URL-encoded name.
> > > > >
> > > > > Thank you,
> > > > >
> > > > > Rajini
> > > > >
> > > > > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <jun@confluent.io
> <javascript:;>> wrote:
> > > > >
> > > > > > Hi, Rajini,
> > > > > >
> > > > > > For the following statements, would it be better to allocate the
> > > quota
> > > > to
> > > > > > all connections whose client-id is clientX? This way, existing
> > > > client-id
> > > > > > quotas are fully compatible in the new release whether the
> cluster
> > is
> > > > in
> > > > > a
> > > > > > single user or multi-user environment.
> > > > > >
> > > > > > 4. If client-id quota override is defined for clientX in
> > > > > > /config/clients/clientX, this quota is allocated for the sole use
> > of
> > > > > > <userN,
> > > > > > clientX>
> > > > > > 5. If dynamic client-id default is configured in /config/clients,
> > > this
> > > > > > default quota is allocated for the sole use of <userN, clientX>
> > > > > > 6. If quota.producer.default is configured for the broker in
> > > > > > server.properties, this default quota is allocated for the sole
> use
> > > of
> > > > > > <userN,
> > > > > > clientX>
> > > > > >
> > > > > > We can potentially add a default quota for both user and client
> at
> > > path
> > > > > > /config/users/clients?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > > > >
> > > > > > > Ismael, Jun,
> > > > > > >
> > > > > > > Thank you both for the feedback. Have updated the KIP to add
> > > dynamic
> > > > > > > default quotas for client-id with deprecation of existing
> static
> > > > > default
> > > > > > > properties.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <jun@confluent.io
> <javascript:;>>
> > > wrote:
> > > > > > >
> > > > > > > > Yes, for consistency, perhaps we can allow client-id quota to
> > be
> > > > > > > configured
> > > > > > > > dynamically too and mark the static config in the broker as
> > > > > deprecated.
> > > > > > > If
> > > > > > > > both are set, the dynamic one wins.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <
> > ismael@juma.me.uk <javascript:;>>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > > > > > rajinisivaram@googlemail.com <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > > > It is actually quite tempting to do the same for
> client-id
> > > > quotas
> > > > > > as
> > > > > > > > > well,
> > > > > > > > > > but I suppose we can't break existing users who have
> > > configured
> > > > > > > > defaults
> > > > > > > > > in
> > > > > > > > > > server.properties and providing two ways of setting
> > client-id
> > > > > > > defaults
> > > > > > > > > > would be just too confusing.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Using two different approaches for client-id versus user
> > quota
> > > > > > defaults
> > > > > > > > is
> > > > > > > > > also not great. We could deprecate the server.properties
> > > default
> > > > > > > configs
> > > > > > > > > for client-id quotas and remove them in the future. In the
> > > > > meantime,
> > > > > > we
> > > > > > > > > would have to live with 2 level defaults.
> > > > > > > > >
> > > > > > > > > Jun, what are your thoughts on this?
> > > > > > > > >
> > > > > > > > > Ismael
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Regards,
> > > > > > >
> > > > > > > Rajini
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Rajini
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Thank you, Jun.

Hi all,

Please let me know if you have any comments or suggestions on the updated
KIP. If there are no objections, I will initiate voting next week.

Thank you...


On Thu, Jun 30, 2016 at 10:37 PM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> The latest wiki looks good to me. Perhaps you want to ask other people to
> also take a look and then we can start the voting.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 28, 2016 at 6:27 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Jun,
> >
> > Thank you for the review. I have changed all default property configs to
> be
> > stored with the node name <default>. So the defaults are
> > /config/clients/<default> for default client-id quota,
> > /config/users/<default> for default user quota and
> > /config/users/<default/clients/<default> for default <user, client-id>
> > quota. Hope that makes sense.
> >
> > On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Rajini,
> > >
> > > Thanks for the update. Looks good to me. My only comment is that
> > > instead of /config/users/<default>/clients,
> > > would it be better to represent it as
> > > /config/users/<default>/clients/<default>
> > > so that it's more consistent?
> > >
> > > Jun
> > >
> > >
> > > On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > Jun,
> > > >
> > > > Yes, I agree that it makes sense to retain the existing semantics for
> > > > client-id quotas for compatibility. Especially if we can provide the
> > > option
> > > > to enable secure client-id quotas for multi-user clusters as well.
> > > >
> > > > I have updated the KIP - each of these levels can have defaults as
> well
> > > as
> > > > specific entries:
> > > >
> > > >    - /config/clients : Insecure <client-id> quotas with the same
> > > semantics
> > > >    as now
> > > >    - /config/users: User quotas
> > > >    - /config/users/userA/clients: <user, client-id> quotas for userA
> > > >    - /config/users/<default>/clients: Default <user, client-id>
> quotas
> > > >
> > > > Now it is fully flexible as well as compatible with the current
> > > > implementation. I used /config/users/<default>/clients rather than
> > > > /config/users/clients since "clients" is a valid (unlikely, but still
> > > > possible) user principal. I used <default>, but it could be anything
> > that
> > > > is a valid Zookeeper node name, but not a valid URL-encoded name.
> > > >
> > > > Thank you,
> > > >
> > > > Rajini
> > > >
> > > > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Rajini,
> > > > >
> > > > > For the following statements, would it be better to allocate the
> > quota
> > > to
> > > > > all connections whose client-id is clientX? This way, existing
> > > client-id
> > > > > quotas are fully compatible in the new release whether the cluster
> is
> > > in
> > > > a
> > > > > single user or multi-user environment.
> > > > >
> > > > > 4. If client-id quota override is defined for clientX in
> > > > > /config/clients/clientX, this quota is allocated for the sole use
> of
> > > > > <userN,
> > > > > clientX>
> > > > > 5. If dynamic client-id default is configured in /config/clients,
> > this
> > > > > default quota is allocated for the sole use of <userN, clientX>
> > > > > 6. If quota.producer.default is configured for the broker in
> > > > > server.properties, this default quota is allocated for the sole use
> > of
> > > > > <userN,
> > > > > clientX>
> > > > >
> > > > > We can potentially add a default quota for both user and client at
> > path
> > > > > /config/users/clients?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > > > > rajinisivaram@googlemail.com> wrote:
> > > > >
> > > > > > Ismael, Jun,
> > > > > >
> > > > > > Thank you both for the feedback. Have updated the KIP to add
> > dynamic
> > > > > > default quotas for client-id with deprecation of existing static
> > > > default
> > > > > > properties.
> > > > > >
> > > > > >
> > > > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > >
> > > > > > > Yes, for consistency, perhaps we can allow client-id quota to
> be
> > > > > > configured
> > > > > > > dynamically too and mark the static config in the broker as
> > > > deprecated.
> > > > > > If
> > > > > > > both are set, the dynamic one wins.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <
> ismael@juma.me.uk>
> > > > > wrote:
> > > > > > >
> > > > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > > >
> > > > > > > > > It is actually quite tempting to do the same for client-id
> > > quotas
> > > > > as
> > > > > > > > well,
> > > > > > > > > but I suppose we can't break existing users who have
> > configured
> > > > > > > defaults
> > > > > > > > in
> > > > > > > > > server.properties and providing two ways of setting
> client-id
> > > > > > defaults
> > > > > > > > > would be just too confusing.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Using two different approaches for client-id versus user
> quota
> > > > > defaults
> > > > > > > is
> > > > > > > > also not great. We could deprecate the server.properties
> > default
> > > > > > configs
> > > > > > > > for client-id quotas and remove them in the future. In the
> > > > meantime,
> > > > > we
> > > > > > > > would have to live with 2 level defaults.
> > > > > > > >
> > > > > > > > Jun, what are your thoughts on this?
> > > > > > > >
> > > > > > > > Ismael
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

The latest wiki looks good to me. Perhaps you want to ask other people to
also take a look and then we can start the voting.

Thanks,

Jun

On Tue, Jun 28, 2016 at 6:27 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Jun,
>
> Thank you for the review. I have changed all default property configs to be
> stored with the node name <default>. So the defaults are
> /config/clients/<default> for default client-id quota,
> /config/users/<default> for default user quota and
> /config/users/<default/clients/<default> for default <user, client-id>
> quota. Hope that makes sense.
>
> On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Rajini,
> >
> > Thanks for the update. Looks good to me. My only comment is that
> > instead of /config/users/<default>/clients,
> > would it be better to represent it as
> > /config/users/<default>/clients/<default>
> > so that it's more consistent?
> >
> > Jun
> >
> >
> > On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > Jun,
> > >
> > > Yes, I agree that it makes sense to retain the existing semantics for
> > > client-id quotas for compatibility. Especially if we can provide the
> > option
> > > to enable secure client-id quotas for multi-user clusters as well.
> > >
> > > I have updated the KIP - each of these levels can have defaults as well
> > as
> > > specific entries:
> > >
> > >    - /config/clients : Insecure <client-id> quotas with the same
> > semantics
> > >    as now
> > >    - /config/users: User quotas
> > >    - /config/users/userA/clients: <user, client-id> quotas for userA
> > >    - /config/users/<default>/clients: Default <user, client-id> quotas
> > >
> > > Now it is fully flexible as well as compatible with the current
> > > implementation. I used /config/users/<default>/clients rather than
> > > /config/users/clients since "clients" is a valid (unlikely, but still
> > > possible) user principal. I used <default>, but it could be anything
> that
> > > is a valid Zookeeper node name, but not a valid URL-encoded name.
> > >
> > > Thank you,
> > >
> > > Rajini
> > >
> > > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Rajini,
> > > >
> > > > For the following statements, would it be better to allocate the
> quota
> > to
> > > > all connections whose client-id is clientX? This way, existing
> > client-id
> > > > quotas are fully compatible in the new release whether the cluster is
> > in
> > > a
> > > > single user or multi-user environment.
> > > >
> > > > 4. If client-id quota override is defined for clientX in
> > > > /config/clients/clientX, this quota is allocated for the sole use of
> > > > <userN,
> > > > clientX>
> > > > 5. If dynamic client-id default is configured in /config/clients,
> this
> > > > default quota is allocated for the sole use of <userN, clientX>
> > > > 6. If quota.producer.default is configured for the broker in
> > > > server.properties, this default quota is allocated for the sole use
> of
> > > > <userN,
> > > > clientX>
> > > >
> > > > We can potentially add a default quota for both user and client at
> path
> > > > /config/users/clients?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > > > rajinisivaram@googlemail.com> wrote:
> > > >
> > > > > Ismael, Jun,
> > > > >
> > > > > Thank you both for the feedback. Have updated the KIP to add
> dynamic
> > > > > default quotas for client-id with deprecation of existing static
> > > default
> > > > > properties.
> > > > >
> > > > >
> > > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > >
> > > > > > Yes, for consistency, perhaps we can allow client-id quota to be
> > > > > configured
> > > > > > dynamically too and mark the static config in the broker as
> > > deprecated.
> > > > > If
> > > > > > both are set, the dynamic one wins.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk>
> > > > wrote:
> > > > > >
> > > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > >
> > > > > > > > It is actually quite tempting to do the same for client-id
> > quotas
> > > > as
> > > > > > > well,
> > > > > > > > but I suppose we can't break existing users who have
> configured
> > > > > > defaults
> > > > > > > in
> > > > > > > > server.properties and providing two ways of setting client-id
> > > > > defaults
> > > > > > > > would be just too confusing.
> > > > > > > >
> > > > > > >
> > > > > > > Using two different approaches for client-id versus user quota
> > > > defaults
> > > > > > is
> > > > > > > also not great. We could deprecate the server.properties
> default
> > > > > configs
> > > > > > > for client-id quotas and remove them in the future. In the
> > > meantime,
> > > > we
> > > > > > > would have to live with 2 level defaults.
> > > > > > >
> > > > > > > Jun, what are your thoughts on this?
> > > > > > >
> > > > > > > Ismael
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Rajini
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

Thank you for the review. I have changed all default property configs to be
stored with the node name <default>. So the defaults are
/config/clients/<default> for default client-id quota,
/config/users/<default> for default user quota and
/config/users/<default/clients/<default> for default <user, client-id>
quota. Hope that makes sense.

On Mon, Jun 27, 2016 at 10:25 PM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> Thanks for the update. Looks good to me. My only comment is that
> instead of /config/users/<default>/clients,
> would it be better to represent it as
> /config/users/<default>/clients/<default>
> so that it's more consistent?
>
> Jun
>
>
> On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Jun,
> >
> > Yes, I agree that it makes sense to retain the existing semantics for
> > client-id quotas for compatibility. Especially if we can provide the
> option
> > to enable secure client-id quotas for multi-user clusters as well.
> >
> > I have updated the KIP - each of these levels can have defaults as well
> as
> > specific entries:
> >
> >    - /config/clients : Insecure <client-id> quotas with the same
> semantics
> >    as now
> >    - /config/users: User quotas
> >    - /config/users/userA/clients: <user, client-id> quotas for userA
> >    - /config/users/<default>/clients: Default <user, client-id> quotas
> >
> > Now it is fully flexible as well as compatible with the current
> > implementation. I used /config/users/<default>/clients rather than
> > /config/users/clients since "clients" is a valid (unlikely, but still
> > possible) user principal. I used <default>, but it could be anything that
> > is a valid Zookeeper node name, but not a valid URL-encoded name.
> >
> > Thank you,
> >
> > Rajini
> >
> > On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Rajini,
> > >
> > > For the following statements, would it be better to allocate the quota
> to
> > > all connections whose client-id is clientX? This way, existing
> client-id
> > > quotas are fully compatible in the new release whether the cluster is
> in
> > a
> > > single user or multi-user environment.
> > >
> > > 4. If client-id quota override is defined for clientX in
> > > /config/clients/clientX, this quota is allocated for the sole use of
> > > <userN,
> > > clientX>
> > > 5. If dynamic client-id default is configured in /config/clients, this
> > > default quota is allocated for the sole use of <userN, clientX>
> > > 6. If quota.producer.default is configured for the broker in
> > > server.properties, this default quota is allocated for the sole use of
> > > <userN,
> > > clientX>
> > >
> > > We can potentially add a default quota for both user and client at path
> > > /config/users/clients?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > Ismael, Jun,
> > > >
> > > > Thank you both for the feedback. Have updated the KIP to add dynamic
> > > > default quotas for client-id with deprecation of existing static
> > default
> > > > properties.
> > > >
> > > >
> > > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Yes, for consistency, perhaps we can allow client-id quota to be
> > > > configured
> > > > > dynamically too and mark the static config in the broker as
> > deprecated.
> > > > If
> > > > > both are set, the dynamic one wins.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > >
> > > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > > rajinisivaram@googlemail.com> wrote:
> > > > > >
> > > > > > > It is actually quite tempting to do the same for client-id
> quotas
> > > as
> > > > > > well,
> > > > > > > but I suppose we can't break existing users who have configured
> > > > > defaults
> > > > > > in
> > > > > > > server.properties and providing two ways of setting client-id
> > > > defaults
> > > > > > > would be just too confusing.
> > > > > > >
> > > > > >
> > > > > > Using two different approaches for client-id versus user quota
> > > defaults
> > > > > is
> > > > > > also not great. We could deprecate the server.properties default
> > > > configs
> > > > > > for client-id quotas and remove them in the future. In the
> > meantime,
> > > we
> > > > > > would have to live with 2 level defaults.
> > > > > >
> > > > > > Jun, what are your thoughts on this?
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

Thanks for the update. Looks good to me. My only comment is that
instead of /config/users/<default>/clients,
would it be better to represent it as /config/users/<default>/clients/<default>
so that it's more consistent?

Jun


On Thu, Jun 23, 2016 at 2:16 PM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Jun,
>
> Yes, I agree that it makes sense to retain the existing semantics for
> client-id quotas for compatibility. Especially if we can provide the option
> to enable secure client-id quotas for multi-user clusters as well.
>
> I have updated the KIP - each of these levels can have defaults as well as
> specific entries:
>
>    - /config/clients : Insecure <client-id> quotas with the same semantics
>    as now
>    - /config/users: User quotas
>    - /config/users/userA/clients: <user, client-id> quotas for userA
>    - /config/users/<default>/clients: Default <user, client-id> quotas
>
> Now it is fully flexible as well as compatible with the current
> implementation. I used /config/users/<default>/clients rather than
> /config/users/clients since "clients" is a valid (unlikely, but still
> possible) user principal. I used <default>, but it could be anything that
> is a valid Zookeeper node name, but not a valid URL-encoded name.
>
> Thank you,
>
> Rajini
>
> On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Rajini,
> >
> > For the following statements, would it be better to allocate the quota to
> > all connections whose client-id is clientX? This way, existing client-id
> > quotas are fully compatible in the new release whether the cluster is in
> a
> > single user or multi-user environment.
> >
> > 4. If client-id quota override is defined for clientX in
> > /config/clients/clientX, this quota is allocated for the sole use of
> > <userN,
> > clientX>
> > 5. If dynamic client-id default is configured in /config/clients, this
> > default quota is allocated for the sole use of <userN, clientX>
> > 6. If quota.producer.default is configured for the broker in
> > server.properties, this default quota is allocated for the sole use of
> > <userN,
> > clientX>
> >
> > We can potentially add a default quota for both user and client at path
> > /config/users/clients?
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > Ismael, Jun,
> > >
> > > Thank you both for the feedback. Have updated the KIP to add dynamic
> > > default quotas for client-id with deprecation of existing static
> default
> > > properties.
> > >
> > >
> > > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Yes, for consistency, perhaps we can allow client-id quota to be
> > > configured
> > > > dynamically too and mark the static config in the broker as
> deprecated.
> > > If
> > > > both are set, the dynamic one wins.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > > rajinisivaram@googlemail.com> wrote:
> > > > >
> > > > > > It is actually quite tempting to do the same for client-id quotas
> > as
> > > > > well,
> > > > > > but I suppose we can't break existing users who have configured
> > > > defaults
> > > > > in
> > > > > > server.properties and providing two ways of setting client-id
> > > defaults
> > > > > > would be just too confusing.
> > > > > >
> > > > >
> > > > > Using two different approaches for client-id versus user quota
> > defaults
> > > > is
> > > > > also not great. We could deprecate the server.properties default
> > > configs
> > > > > for client-id quotas and remove them in the future. In the
> meantime,
> > we
> > > > > would have to live with 2 level defaults.
> > > > >
> > > > > Jun, what are your thoughts on this?
> > > > >
> > > > > Ismael
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

Yes, I agree that it makes sense to retain the existing semantics for
client-id quotas for compatibility. Especially if we can provide the option
to enable secure client-id quotas for multi-user clusters as well.

I have updated the KIP - each of these levels can have defaults as well as
specific entries:

   - /config/clients : Insecure <client-id> quotas with the same semantics
   as now
   - /config/users: User quotas
   - /config/users/userA/clients: <user, client-id> quotas for userA
   - /config/users/<default>/clients: Default <user, client-id> quotas

Now it is fully flexible as well as compatible with the current
implementation. I used /config/users/<default>/clients rather than
/config/users/clients since "clients" is a valid (unlikely, but still
possible) user principal. I used <default>, but it could be anything that
is a valid Zookeeper node name, but not a valid URL-encoded name.

Thank you,

Rajini

On Thu, Jun 23, 2016 at 3:43 PM, Jun Rao <ju...@confluent.io> wrote:

> Hi, Rajini,
>
> For the following statements, would it be better to allocate the quota to
> all connections whose client-id is clientX? This way, existing client-id
> quotas are fully compatible in the new release whether the cluster is in a
> single user or multi-user environment.
>
> 4. If client-id quota override is defined for clientX in
> /config/clients/clientX, this quota is allocated for the sole use of
> <userN,
> clientX>
> 5. If dynamic client-id default is configured in /config/clients, this
> default quota is allocated for the sole use of <userN, clientX>
> 6. If quota.producer.default is configured for the broker in
> server.properties, this default quota is allocated for the sole use of
> <userN,
> clientX>
>
> We can potentially add a default quota for both user and client at path
> /config/users/clients?
>
> Thanks,
>
> Jun
>
> On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Ismael, Jun,
> >
> > Thank you both for the feedback. Have updated the KIP to add dynamic
> > default quotas for client-id with deprecation of existing static default
> > properties.
> >
> >
> > On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Yes, for consistency, perhaps we can allow client-id quota to be
> > configured
> > > dynamically too and mark the static config in the broker as deprecated.
> > If
> > > both are set, the dynamic one wins.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk>
> wrote:
> > >
> > > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > > rajinisivaram@googlemail.com> wrote:
> > > >
> > > > > It is actually quite tempting to do the same for client-id quotas
> as
> > > > well,
> > > > > but I suppose we can't break existing users who have configured
> > > defaults
> > > > in
> > > > > server.properties and providing two ways of setting client-id
> > defaults
> > > > > would be just too confusing.
> > > > >
> > > >
> > > > Using two different approaches for client-id versus user quota
> defaults
> > > is
> > > > also not great. We could deprecate the server.properties default
> > configs
> > > > for client-id quotas and remove them in the future. In the meantime,
> we
> > > > would have to live with 2 level defaults.
> > > >
> > > > Jun, what are your thoughts on this?
> > > >
> > > > Ismael
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Hi, Rajini,

For the following statements, would it be better to allocate the quota to
all connections whose client-id is clientX? This way, existing client-id
quotas are fully compatible in the new release whether the cluster is in a
single user or multi-user environment.

4. If client-id quota override is defined for clientX in
/config/clients/clientX, this quota is allocated for the sole use of <userN,
clientX>
5. If dynamic client-id default is configured in /config/clients, this
default quota is allocated for the sole use of <userN, clientX>
6. If quota.producer.default is configured for the broker in
server.properties, this default quota is allocated for the sole use of <userN,
clientX>

We can potentially add a default quota for both user and client at path
/config/users/clients?

Thanks,

Jun

On Wed, Jun 22, 2016 at 3:01 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Ismael, Jun,
>
> Thank you both for the feedback. Have updated the KIP to add dynamic
> default quotas for client-id with deprecation of existing static default
> properties.
>
>
> On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Yes, for consistency, perhaps we can allow client-id quota to be
> configured
> > dynamically too and mark the static config in the broker as deprecated.
> If
> > both are set, the dynamic one wins.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > It is actually quite tempting to do the same for client-id quotas as
> > > well,
> > > > but I suppose we can't break existing users who have configured
> > defaults
> > > in
> > > > server.properties and providing two ways of setting client-id
> defaults
> > > > would be just too confusing.
> > > >
> > >
> > > Using two different approaches for client-id versus user quota defaults
> > is
> > > also not great. We could deprecate the server.properties default
> configs
> > > for client-id quotas and remove them in the future. In the meantime, we
> > > would have to live with 2 level defaults.
> > >
> > > Jun, what are your thoughts on this?
> > >
> > > Ismael
> > >
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Ismael, Jun,

Thank you both for the feedback. Have updated the KIP to add dynamic
default quotas for client-id with deprecation of existing static default
properties.


On Wed, Jun 22, 2016 at 12:50 AM, Jun Rao <ju...@confluent.io> wrote:

> Yes, for consistency, perhaps we can allow client-id quota to be configured
> dynamically too and mark the static config in the broker as deprecated. If
> both are set, the dynamic one wins.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > It is actually quite tempting to do the same for client-id quotas as
> > well,
> > > but I suppose we can't break existing users who have configured
> defaults
> > in
> > > server.properties and providing two ways of setting client-id defaults
> > > would be just too confusing.
> > >
> >
> > Using two different approaches for client-id versus user quota defaults
> is
> > also not great. We could deprecate the server.properties default configs
> > for client-id quotas and remove them in the future. In the meantime, we
> > would have to live with 2 level defaults.
> >
> > Jun, what are your thoughts on this?
> >
> > Ismael
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Yes, for consistency, perhaps we can allow client-id quota to be configured
dynamically too and mark the static config in the broker as deprecated. If
both are set, the dynamic one wins.

Thanks,

Jun

On Tue, Jun 21, 2016 at 3:56 AM, Ismael Juma <is...@juma.me.uk> wrote:

> On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > It is actually quite tempting to do the same for client-id quotas as
> well,
> > but I suppose we can't break existing users who have configured defaults
> in
> > server.properties and providing two ways of setting client-id defaults
> > would be just too confusing.
> >
>
> Using two different approaches for client-id versus user quota defaults is
> also not great. We could deprecate the server.properties default configs
> for client-id quotas and remove them in the future. In the meantime, we
> would have to live with 2 level defaults.
>
> Jun, what are your thoughts on this?
>
> Ismael
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Ismael Juma <is...@juma.me.uk>.
On Tue, Jun 21, 2016 at 12:50 PM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> It is actually quite tempting to do the same for client-id quotas as well,
> but I suppose we can't break existing users who have configured defaults in
> server.properties and providing two ways of setting client-id defaults
> would be just too confusing.
>

Using two different approaches for client-id versus user quota defaults is
also not great. We could deprecate the server.properties default configs
for client-id quotas and remove them in the future. In the meantime, we
would have to live with 2 level defaults.

Jun, what are your thoughts on this?

Ismael

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

I think that is a good idea. It keeps all user quotas in one place with an
intuitive hierarchy, avoids properties that are applied based on various
conditions and also enables dynamic updates. Dynamic updates obviously need
to get applied to all clients using the default, but I think that should be
ok.

It is actually quite tempting to do the same for client-id quotas as well,
but I suppose we can't break existing users who have configured defaults in
server.properties and providing two ways of setting client-id defaults
would be just too confusing.

I have updated the KIP to remove the default user quota properties and set
default user quotas in /config/users.

Thank you,

Rajini

On Tue, Jun 21, 2016 at 6:04 AM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> Another thing that's probably worth thinking through is whether it's better
> to make the default user quota dynamic as well. So, instead of adding
> quota.user.producer.default
> and quota.user.consumer.default in the broker config, another way is to set
> them using sth like the following.
>
> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> 'producer_byte_rate=10,consumer_byte_rate=20' --entity-name=* --entity-type
> users
>
> We may add other types of quotas in the future and we probably don't want
> to keep adding static configs.
>
> Thanks,
>
> Jun
>
>
>
> On Mon, Jun 20, 2016 at 2:32 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Gwen/Jay,
> >
> > Have you had a chance to look at the updated KIP? It will be good to get
> > your feedback as well before restarting vote on the updated KIP.
> >
> > If there are no objections, I will start the vote tomorrow.
> >
> >
> > On Fri, Jun 17, 2016 at 6:59 PM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > Thank you, Jun. I have removed user_principal from the KIP.
> > >
> > > On Fri, Jun 17, 2016 at 6:00 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > >> Rajini,
> > >>
> > >> 10. Yes, then we can probably leave out the user_principal field and
> > keep
> > >> the version to be 1.
> > >>
> > >> Other than that, the KIP looks good to me.
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Fri, Jun 17, 2016 at 3:29 AM, Rajini Sivaram <
> > >> rajinisivaram@googlemail.com> wrote:
> > >>
> > >> > Jun,
> > >> >
> > >> > 10. Since entity_type "users" is new, shouldn't the JSON for these
> > >> entities
> > >> > have version 1? I have moved "user_principal" out of the config in
> the
> > >> > samples and added to the <user, client> entries as well. But
> actually,
> > >> do
> > >> > we need to store the non-encoded principal at all? The node name is
> > >> > URL-encoded user principal, so it is fairly readable if you are
> > looking
> > >> in
> > >> > ZK and *kafka_configs.sh* will show the non-encoded principal by
> > >> decoding
> > >> > the name from the path (since it needs to do encoding anyway because
> > the
> > >> > names specified on the command line will be non-encoded principals,
> it
> > >> can
> > >> > do decoding too). Perhaps that is sufficient?
> > >> >
> > >> > 11. I liked the second approach since it looks neat and
> future-proof.
> > >> Have
> > >> > updated the KIP.
> > >> >
> > >> > 12. Yes, that is correct.
> > >> >
> > >> > Many thanks,
> > >> >
> > >> > Rajini
> > >> >
> > >> >
> > >> > On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:
> > >> >
> > >> > > Rajini,
> > >> > >
> > >> > > Thanks for the update. A few more questions/comments.
> > >> > >
> > >> > > 10. For the quota value stored in ZK, since we are adding an
> > optional
> > >> > > user_principal field in the json, we should bump the version from
> 1
> > >> to 2.
> > >> > > Also, user_principal is not really part of the config values. So,
> > >> perhaps
> > >> > > we should represent it as the following?
> > >> > > {
> > >> > >     "version":2,
> > >> > >     "config": {
> > >> > >         "producer_byte_rate":"1024",
> > >> > >         "consumer_byte_rate":"2048"
> > >> > >     },
> > >> > >     "user_principal" : "user1"
> > >> > > }
> > >> > >
> > >> > >  Also, we should store user_principal in the following json too,
> > >> right?
> > >> > > // Zookeeper persistence path
> /users/<encoded-user2>/clients/clientA
> > >> > > {
> > >> > >     "version":1,
> > >> > >     "config": {
> > >> > >         "producer_byte_rate":"10",
> > >> > >         "consumer_byte_rate":"30"
> > >> > >     }
> > >> > > }
> > >> > >
> > >> > > 11. For the change notification path, would it be better to change
> > it
> > >> to
> > >> > > something like the following and bump up version to 2?
> > >> > > // Change notification for quota of <user2, clientA>
> > >> > > {
> > >> > >     "version":2,
> > >> > >     [
> > >> > >       { "entity_type": "users",
> > >> > >         "entity_name": "user2"
> > >> > >       },
> > >> > >       { "entity_type": "clients",
> > >> > >         "entity_name": "clientA"
> > >> > >       }
> > >> > >     ]
> > >> > >  }
> > >> > >
> > >> > > Alternatively, we could change it to
> > >> > > // Change notification for quota of <user2, clientA>
> > >> > > {
> > >> > >     "version":2,
> > >> > >     "entity_path": "users/user2"
> > >> > > }
> > >> > >
> > >> > > {
> > >> > >     "version":2,
> > >> > >     "entity_path": "users/user2/clients/clientA"
> > >> > > }
> > >> > >
> > >> > > 12. Just to clarify on the meaning of remainder quota. If you have
> > >> quotas
> > >> > > like the following,
> > >> > >  <user1, client1> = 5
> > >> > >  <user1, client2> = 10
> > >> > >  <user1> = 12
> > >> > > it means that all connections with user1 whose client-id is
> neither
> > >> > client1
> > >> > > nor client2 will be sharing a quota of 12, right? In other words,
> > the
> > >> > quota
> > >> > > of <user1> doesn't include the quota for <user1, client1> and
> > <user1,
> > >> > > client2>.
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > >
> > >> > > On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
> > >> > > rajinisivaram@googlemail.com> wrote:
> > >> > >
> > >> > > > Jun,
> > >> > > >
> > >> > > > Actually, with quotas stored in different nodes in ZK, it is
> > better
> > >> to
> > >> > > > store remainder quota rather than total quota under
> /users/<user>
> > so
> > >> > that
> > >> > > > quota calculations are not dependent on the order of
> > notifications.
> > >> I
> > >> > > have
> > >> > > > updated the KIP to reflect that. So the quotas in ZK now always
> > >> reflect
> > >> > > the
> > >> > > > quota applied to a group of client connections and use the same
> > >> format
> > >> > as
> > >> > > > client-id quotas. But it is not hierarchical, making the
> > >> configuration
> > >> > > > simpler.
> > >> > > >
> > >> > > > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> > >> > > > rajinisivaram@googlemail.com> wrote:
> > >> > > >
> > >> > > > > Jun,
> > >> > > > >
> > >> > > > > Thank you for the review. I have updated the KIP:
> > >> > > > >
> > >> > > > >
> > >> > > > >    1. Added an overview section. Slightly reworded since it is
> > >> better
> > >> > > to
> > >> > > > >    treat user and client-id as different levels rather than
> the
> > >> same
> > >> > > > level.
> > >> > > > >    2. Yes, it is neater to store quota for each entity in a
> > >> different
> > >> > > > >    path in Zookeeper. I put clients under users rather than
> the
> > >> other
> > >> > > way
> > >> > > > >    round since that reflects the hierarchy and also keeps a
> > user's
> > >> > > quotas
> > >> > > > >    together under a single sub-tree. I had initially used a
> > single
> > >> > node
> > >> > > > to
> > >> > > > >    keep quotas and sub-quotas of a user together so that
> updates
> > >> are
> > >> > > > atomic
> > >> > > > >    since changes to sub-quotas also affect remainder quotas
> for
> > >> other
> > >> > > > clients.
> > >> > > > >    But I imagine, updates to configs are rare and it is not a
> > big
> > >> > > issue.
> > >> > > > >    3. I haven't modified the JSON for configuration change
> > >> > > notifications.
> > >> > > > >    The entity_name can now be a subpath that has both user and
> > >> > client.
> > >> > > > Have
> > >> > > > >    added an example to the KIP. The downside of keeping
> clients
> > >> under
> > >> > > > users in
> > >> > > > >    ZK in 2) is that the change notification for sub-quota has
> > >> > > entity_type
> > >> > > > >    "users". I could extend the JSON to include client
> > separately,
> > >> but
> > >> > > > since
> > >> > > > >    changes to a client sub-quota does impact other clients of
> > the
> > >> > user
> > >> > > > as well
> > >> > > > >    due to change in remainder quota, it may be ok as it is. Do
> > >> let me
> > >> > > > know if
> > >> > > > >    it looks confusing in the example.
> > >> > > > >    4. Agree, updated.
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io>
> > >> wrote:
> > >> > > > >
> > >> > > > >> Hi, Rajini,
> > >> > > > >>
> > >> > > > >> Thanks for the updated wiki. Overall, I like the new
> approach.
> > It
> > >> > > covers
> > >> > > > >> the common use cases now, is extensible, and is backward
> > >> > compatible. A
> > >> > > > few
> > >> > > > >> comments below.
> > >> > > > >>
> > >> > > > >> 1. It would be useful to describe a bit at the high level,
> how
> > >> the
> > >> > new
> > >> > > > >> approach works. I think this can be summarized as follows.
> > Quotas
> > >> > can
> > >> > > be
> > >> > > > >> set at user, client-id or <user, client-id> levels. For a
> given
> > >> > client
> > >> > > > >> connection, the most specific quota matching the connection
> > will
> > >> be
> > >> > > > >> applied. For example, if both a <user, client-id> and a
> <user>
> > >> quota
> > >> > > > match
> > >> > > > >> a connection, the <user, client-id> quota will be used. If
> more
> > >> > than 1
> > >> > > > >> quota at the same level (e.g., a quota on a user and another
> > >> quota
> > >> > on
> > >> > > a
> > >> > > > >> client-id) match the connection, the user level quota will be
> > >> used,
> > >> > > > i.e.,
> > >> > > > >> user quota takes precedence over client-id quota.
> > >> > > > >>
> > >> > > > >> 2. For the ZK data structure, would it be better to store
> > <user,
> > >> > > > >> client-id>
> > >> > > > >> quota as the following. Then the format of the value in each
> > >> path is
> > >> > > the
> > >> > > > >> same. The wiki also mentions that we want to include the
> > original
> > >> > user
> > >> > > > >> name
> > >> > > > >> in the ZK value. Could you describe that in an example?
> > >> > > > >> // Zookeeper persistence path
> > >> /clients/clientA/users/<encoded-user2>
> > >> > > > >> {
> > >> > > > >>     "version":1,
> > >> > > > >>     "config": {
> > >> > > > >>         "producer_byte_rate":"10",
> > >> > > > >>         "consumer_byte_rate":"20"
> > >> > > > >>     }
> > >> > > > >> }
> > >> > > > >>
> > >> > > > >> 3. Could you document the format change of the ZK value in
> > >> > > > >> /config/changes/config_change_xxx, if any?
> > >> > > > >>
> > >> > > > >> 4. For the config command, could we specify the sub-quota
> like
> > >> the
> > >> > > > >> following, instead of in the config value? This seems more
> > >> > intuitive.
> > >> > > > >>
> > >> > > > >> bin/kafka-configs  --zookeeper localhost:2181 --alter
> > >> --add-config
> > >> > > > >> 'producer_byte_rate=1024,consumer_byte_rate=2048'
> --entity-name
> > >> > > > >> clientA --entity-type clients --entity-name user2
> --entity-type
> > >> > users
> > >> > > > >>
> > >> > > > >> Thanks,
> > >> > > > >>
> > >> > > > >> Jun
> > >> > > > >>
> > >> > > > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> > >> > > > >> rajinisivaram@googlemail.com> wrote:
> > >> > > > >>
> > >> > > > >> > Harsha,
> > >> > > > >> >
> > >> > > > >> > The sample configuration under
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> > >> > > > >> > shows
> > >> > > > >> > the Zookeeper data for different scenarios.
> > >> > > > >> >
> > >> > > > >> >    - *user1* (/users/user1 in ZK) has only user-level
> quotas
> > >> > > > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas
> > and
> > >> > > > >> sub-quotas
> > >> > > > >> >    for clients *clientA* and *clientB*. Other client-ids of
> > >> > *user2*
> > >> > > > >> share
> > >> > > > >> >    the remaining quota of *user2*.
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io>
> > >> wrote:
> > >> > > > >> >
> > >> > > > >> > > Rajini,
> > >> > > > >> > >           How does sub-quotas works in case of
> > authenticated
> > >> > > users.
> > >> > > > >> > >           Where are we maintaining the relation between
> > users
> > >> > and
> > >> > > > >> their
> > >> > > > >> > >           client Ids. Can you add an example of zk data
> > under
> > >> > > > /users.
> > >> > > > >> > > Thanks,
> > >> > > > >> > > Harsha
> > >> > > > >> > >
> > >> > > > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > >> > > > >> > > > I have updated KIP-55 to reflect the changes from the
> > >> > > discussions
> > >> > > > in
> > >> > > > >> > the
> > >> > > > >> > > > voting thread (
> > >> > > > >> > > >
> > >> > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
> > >> > > ).
> > >> > > > >> > > >
> > >> > > > >> > > > Jun/Gwen,
> > >> > > > >> > > >
> > >> > > > >> > > > Existing client-id quotas will be used as default
> > client-id
> > >> > > quotas
> > >> > > > >> for
> > >> > > > >> > > > users when no user quotas are configured - i.e.,
> default
> > >> user
> > >> > > > quota
> > >> > > > >> is
> > >> > > > >> > > > unlimited and no user-specific quota override is
> > specified.
> > >> > This
> > >> > > > >> > enables
> > >> > > > >> > > > user rate limits to be configured for ANONYMOUS if
> > required
> > >> > in a
> > >> > > > >> > cluster
> > >> > > > >> > > > that has both PLAINTEXT and SSL/SASL. By default,
> without
> > >> any
> > >> > > user
> > >> > > > >> rate
> > >> > > > >> > > > limits set, rate limits for client-ids will apply,
> > >> retaining
> > >> > the
> > >> > > > >> > current
> > >> > > > >> > > > client-id quota configuration for single-user clusters.
> > >> > > > >> > > >
> > >> > > > >> > > > Zookeeper will have two paths /clients with client-id
> > >> quotas
> > >> > > that
> > >> > > > >> apply
> > >> > > > >> > > > only when user quota is unlimited similar to now. And
> > >> /users
> > >> > > which
> > >> > > > >> > > > persists
> > >> > > > >> > > > user quotas for any user including ANONYMOUS.
> > >> > > > >> > > >
> > >> > > > >> > > > Comments and feedback are appreciated.
> > >> > > > >> > > >
> > >> > > > >> > > > Regards,
> > >> > > > >> > > >
> > >> > > > >> > > > Rajini
> > >> > > > >> > > >
> > >> > > > >> > > >
> > >> > > > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > >> > > > >> > > > <rajinisivaram@googlemail.com
> > >> > > > >> > > > > wrote:
> > >> > > > >> > > >
> > >> > > > >> > > > > Jun,
> > >> > > > >> > > > >
> > >> > > > >> > > > > Oops, sorry, I hadn't realized that the last note was
> > on
> > >> the
> > >> > > > >> discuss
> > >> > > > >> > > > > thread. Thank you for pointing it out. I have sent
> > >> another
> > >> > > note
> > >> > > > >> for
> > >> > > > >> > > voting.
> > >> > > > >> > > > >
> > >> > > > >> > > > >
> > >> > > > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <
> > >> jun@confluent.io>
> > >> > > > wrote:
> > >> > > > >> > > > >
> > >> > > > >> > > > >> Rajini,
> > >> > > > >> > > > >>
> > >> > > > >> > > > >> Perhaps it will be clearer if you start the voting
> in
> > a
> > >> new
> > >> > > > >> thread
> > >> > > > >> > > (with
> > >> > > > >> > > > >> VOTE in the subject).
> > >> > > > >> > > > >>
> > >> > > > >> > > > >> Thanks,
> > >> > > > >> > > > >>
> > >> > > > >> > > > >> Jun
> > >> > > > >> > > > >>
> > >> > > > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > >> > > > >> > > > >> rajinisivaram@googlemail.com
> > >> > > > >> > > > >> > wrote:
> > >> > > > >> > > > >>
> > >> > > > >> > > > >> > I would like to initiate the vote for KIP-55.
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> > The KIP details are here: KIP-55: Secure quotas
> for
> > >> > > > >> authenticated
> > >> > > > >> > > users
> > >> > > > >> > > > >> > <
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >>
> > >> > > > >> > >
> > >> > > > >> >
> > >> > > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > >> > > > >> > > > >> > >
> > >> > > > >> > > > >> > .
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> > The JIRA  KAFKA-3492  <
> > >> > > > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> > >> > > > >> > > > >> > >has
> > >> > > > >> > > > >> > a draft PR here:
> > >> > https://github.com/apache/kafka/pull/1256
> > >> > > .
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> > Thank you...
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> > Regards,
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >> > Rajini
> > >> > > > >> > > > >> >
> > >> > > > >> > > > >>
> > >> > > > >> > > > >
> > >> > > > >> > > > >
> > >> > > > >> > > > >
> > >> > > > >> > > > > --
> > >> > > > >> > > > > Regards,
> > >> > > > >> > > > >
> > >> > > > >> > > > > Rajini
> > >> > > > >> > > > >
> > >> > > > >> > > >
> > >> > > > >> > > >
> > >> > > > >> > > >
> > >> > > > >> > > > --
> > >> > > > >> > > > Regards,
> > >> > > > >> > > >
> > >> > > > >> > > > Rajini
> > >> > > > >> > >
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > --
> > >> > > > >> > Regards,
> > >> > > > >> >
> > >> > > > >> > Rajini
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Regards,
> > >> > > > >
> > >> > > > > Rajini
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Regards,
> > >> > > >
> > >> > > > Rajini
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Regards,
> > >> >
> > >> > Rajini
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

Another thing that's probably worth thinking through is whether it's better
to make the default user quota dynamic as well. So, instead of adding
quota.user.producer.default
and quota.user.consumer.default in the broker config, another way is to set
them using sth like the following.

bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
'producer_byte_rate=10,consumer_byte_rate=20' --entity-name=* --entity-type
users

We may add other types of quotas in the future and we probably don't want
to keep adding static configs.

Thanks,

Jun



On Mon, Jun 20, 2016 at 2:32 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Gwen/Jay,
>
> Have you had a chance to look at the updated KIP? It will be good to get
> your feedback as well before restarting vote on the updated KIP.
>
> If there are no objections, I will start the vote tomorrow.
>
>
> On Fri, Jun 17, 2016 at 6:59 PM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Thank you, Jun. I have removed user_principal from the KIP.
> >
> > On Fri, Jun 17, 2016 at 6:00 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Rajini,
> >>
> >> 10. Yes, then we can probably leave out the user_principal field and
> keep
> >> the version to be 1.
> >>
> >> Other than that, the KIP looks good to me.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Fri, Jun 17, 2016 at 3:29 AM, Rajini Sivaram <
> >> rajinisivaram@googlemail.com> wrote:
> >>
> >> > Jun,
> >> >
> >> > 10. Since entity_type "users" is new, shouldn't the JSON for these
> >> entities
> >> > have version 1? I have moved "user_principal" out of the config in the
> >> > samples and added to the <user, client> entries as well. But actually,
> >> do
> >> > we need to store the non-encoded principal at all? The node name is
> >> > URL-encoded user principal, so it is fairly readable if you are
> looking
> >> in
> >> > ZK and *kafka_configs.sh* will show the non-encoded principal by
> >> decoding
> >> > the name from the path (since it needs to do encoding anyway because
> the
> >> > names specified on the command line will be non-encoded principals, it
> >> can
> >> > do decoding too). Perhaps that is sufficient?
> >> >
> >> > 11. I liked the second approach since it looks neat and future-proof.
> >> Have
> >> > updated the KIP.
> >> >
> >> > 12. Yes, that is correct.
> >> >
> >> > Many thanks,
> >> >
> >> > Rajini
> >> >
> >> >
> >> > On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:
> >> >
> >> > > Rajini,
> >> > >
> >> > > Thanks for the update. A few more questions/comments.
> >> > >
> >> > > 10. For the quota value stored in ZK, since we are adding an
> optional
> >> > > user_principal field in the json, we should bump the version from 1
> >> to 2.
> >> > > Also, user_principal is not really part of the config values. So,
> >> perhaps
> >> > > we should represent it as the following?
> >> > > {
> >> > >     "version":2,
> >> > >     "config": {
> >> > >         "producer_byte_rate":"1024",
> >> > >         "consumer_byte_rate":"2048"
> >> > >     },
> >> > >     "user_principal" : "user1"
> >> > > }
> >> > >
> >> > >  Also, we should store user_principal in the following json too,
> >> right?
> >> > > // Zookeeper persistence path /users/<encoded-user2>/clients/clientA
> >> > > {
> >> > >     "version":1,
> >> > >     "config": {
> >> > >         "producer_byte_rate":"10",
> >> > >         "consumer_byte_rate":"30"
> >> > >     }
> >> > > }
> >> > >
> >> > > 11. For the change notification path, would it be better to change
> it
> >> to
> >> > > something like the following and bump up version to 2?
> >> > > // Change notification for quota of <user2, clientA>
> >> > > {
> >> > >     "version":2,
> >> > >     [
> >> > >       { "entity_type": "users",
> >> > >         "entity_name": "user2"
> >> > >       },
> >> > >       { "entity_type": "clients",
> >> > >         "entity_name": "clientA"
> >> > >       }
> >> > >     ]
> >> > >  }
> >> > >
> >> > > Alternatively, we could change it to
> >> > > // Change notification for quota of <user2, clientA>
> >> > > {
> >> > >     "version":2,
> >> > >     "entity_path": "users/user2"
> >> > > }
> >> > >
> >> > > {
> >> > >     "version":2,
> >> > >     "entity_path": "users/user2/clients/clientA"
> >> > > }
> >> > >
> >> > > 12. Just to clarify on the meaning of remainder quota. If you have
> >> quotas
> >> > > like the following,
> >> > >  <user1, client1> = 5
> >> > >  <user1, client2> = 10
> >> > >  <user1> = 12
> >> > > it means that all connections with user1 whose client-id is neither
> >> > client1
> >> > > nor client2 will be sharing a quota of 12, right? In other words,
> the
> >> > quota
> >> > > of <user1> doesn't include the quota for <user1, client1> and
> <user1,
> >> > > client2>.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
> >> > > rajinisivaram@googlemail.com> wrote:
> >> > >
> >> > > > Jun,
> >> > > >
> >> > > > Actually, with quotas stored in different nodes in ZK, it is
> better
> >> to
> >> > > > store remainder quota rather than total quota under /users/<user>
> so
> >> > that
> >> > > > quota calculations are not dependent on the order of
> notifications.
> >> I
> >> > > have
> >> > > > updated the KIP to reflect that. So the quotas in ZK now always
> >> reflect
> >> > > the
> >> > > > quota applied to a group of client connections and use the same
> >> format
> >> > as
> >> > > > client-id quotas. But it is not hierarchical, making the
> >> configuration
> >> > > > simpler.
> >> > > >
> >> > > > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> >> > > > rajinisivaram@googlemail.com> wrote:
> >> > > >
> >> > > > > Jun,
> >> > > > >
> >> > > > > Thank you for the review. I have updated the KIP:
> >> > > > >
> >> > > > >
> >> > > > >    1. Added an overview section. Slightly reworded since it is
> >> better
> >> > > to
> >> > > > >    treat user and client-id as different levels rather than the
> >> same
> >> > > > level.
> >> > > > >    2. Yes, it is neater to store quota for each entity in a
> >> different
> >> > > > >    path in Zookeeper. I put clients under users rather than the
> >> other
> >> > > way
> >> > > > >    round since that reflects the hierarchy and also keeps a
> user's
> >> > > quotas
> >> > > > >    together under a single sub-tree. I had initially used a
> single
> >> > node
> >> > > > to
> >> > > > >    keep quotas and sub-quotas of a user together so that updates
> >> are
> >> > > > atomic
> >> > > > >    since changes to sub-quotas also affect remainder quotas for
> >> other
> >> > > > clients.
> >> > > > >    But I imagine, updates to configs are rare and it is not a
> big
> >> > > issue.
> >> > > > >    3. I haven't modified the JSON for configuration change
> >> > > notifications.
> >> > > > >    The entity_name can now be a subpath that has both user and
> >> > client.
> >> > > > Have
> >> > > > >    added an example to the KIP. The downside of keeping clients
> >> under
> >> > > > users in
> >> > > > >    ZK in 2) is that the change notification for sub-quota has
> >> > > entity_type
> >> > > > >    "users". I could extend the JSON to include client
> separately,
> >> but
> >> > > > since
> >> > > > >    changes to a client sub-quota does impact other clients of
> the
> >> > user
> >> > > > as well
> >> > > > >    due to change in remainder quota, it may be ok as it is. Do
> >> let me
> >> > > > know if
> >> > > > >    it looks confusing in the example.
> >> > > > >    4. Agree, updated.
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io>
> >> wrote:
> >> > > > >
> >> > > > >> Hi, Rajini,
> >> > > > >>
> >> > > > >> Thanks for the updated wiki. Overall, I like the new approach.
> It
> >> > > covers
> >> > > > >> the common use cases now, is extensible, and is backward
> >> > compatible. A
> >> > > > few
> >> > > > >> comments below.
> >> > > > >>
> >> > > > >> 1. It would be useful to describe a bit at the high level, how
> >> the
> >> > new
> >> > > > >> approach works. I think this can be summarized as follows.
> Quotas
> >> > can
> >> > > be
> >> > > > >> set at user, client-id or <user, client-id> levels. For a given
> >> > client
> >> > > > >> connection, the most specific quota matching the connection
> will
> >> be
> >> > > > >> applied. For example, if both a <user, client-id> and a <user>
> >> quota
> >> > > > match
> >> > > > >> a connection, the <user, client-id> quota will be used. If more
> >> > than 1
> >> > > > >> quota at the same level (e.g., a quota on a user and another
> >> quota
> >> > on
> >> > > a
> >> > > > >> client-id) match the connection, the user level quota will be
> >> used,
> >> > > > i.e.,
> >> > > > >> user quota takes precedence over client-id quota.
> >> > > > >>
> >> > > > >> 2. For the ZK data structure, would it be better to store
> <user,
> >> > > > >> client-id>
> >> > > > >> quota as the following. Then the format of the value in each
> >> path is
> >> > > the
> >> > > > >> same. The wiki also mentions that we want to include the
> original
> >> > user
> >> > > > >> name
> >> > > > >> in the ZK value. Could you describe that in an example?
> >> > > > >> // Zookeeper persistence path
> >> /clients/clientA/users/<encoded-user2>
> >> > > > >> {
> >> > > > >>     "version":1,
> >> > > > >>     "config": {
> >> > > > >>         "producer_byte_rate":"10",
> >> > > > >>         "consumer_byte_rate":"20"
> >> > > > >>     }
> >> > > > >> }
> >> > > > >>
> >> > > > >> 3. Could you document the format change of the ZK value in
> >> > > > >> /config/changes/config_change_xxx, if any?
> >> > > > >>
> >> > > > >> 4. For the config command, could we specify the sub-quota like
> >> the
> >> > > > >> following, instead of in the config value? This seems more
> >> > intuitive.
> >> > > > >>
> >> > > > >> bin/kafka-configs  --zookeeper localhost:2181 --alter
> >> --add-config
> >> > > > >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> >> > > > >> clientA --entity-type clients --entity-name user2 --entity-type
> >> > users
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >>
> >> > > > >> Jun
> >> > > > >>
> >> > > > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> >> > > > >> rajinisivaram@googlemail.com> wrote:
> >> > > > >>
> >> > > > >> > Harsha,
> >> > > > >> >
> >> > > > >> > The sample configuration under
> >> > > > >> >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> >> > > > >> > shows
> >> > > > >> > the Zookeeper data for different scenarios.
> >> > > > >> >
> >> > > > >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> >> > > > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas
> and
> >> > > > >> sub-quotas
> >> > > > >> >    for clients *clientA* and *clientB*. Other client-ids of
> >> > *user2*
> >> > > > >> share
> >> > > > >> >    the remaining quota of *user2*.
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io>
> >> wrote:
> >> > > > >> >
> >> > > > >> > > Rajini,
> >> > > > >> > >           How does sub-quotas works in case of
> authenticated
> >> > > users.
> >> > > > >> > >           Where are we maintaining the relation between
> users
> >> > and
> >> > > > >> their
> >> > > > >> > >           client Ids. Can you add an example of zk data
> under
> >> > > > /users.
> >> > > > >> > > Thanks,
> >> > > > >> > > Harsha
> >> > > > >> > >
> >> > > > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> >> > > > >> > > > I have updated KIP-55 to reflect the changes from the
> >> > > discussions
> >> > > > in
> >> > > > >> > the
> >> > > > >> > > > voting thread (
> >> > > > >> > > >
> >> > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
> >> > > ).
> >> > > > >> > > >
> >> > > > >> > > > Jun/Gwen,
> >> > > > >> > > >
> >> > > > >> > > > Existing client-id quotas will be used as default
> client-id
> >> > > quotas
> >> > > > >> for
> >> > > > >> > > > users when no user quotas are configured - i.e., default
> >> user
> >> > > > quota
> >> > > > >> is
> >> > > > >> > > > unlimited and no user-specific quota override is
> specified.
> >> > This
> >> > > > >> > enables
> >> > > > >> > > > user rate limits to be configured for ANONYMOUS if
> required
> >> > in a
> >> > > > >> > cluster
> >> > > > >> > > > that has both PLAINTEXT and SSL/SASL. By default, without
> >> any
> >> > > user
> >> > > > >> rate
> >> > > > >> > > > limits set, rate limits for client-ids will apply,
> >> retaining
> >> > the
> >> > > > >> > current
> >> > > > >> > > > client-id quota configuration for single-user clusters.
> >> > > > >> > > >
> >> > > > >> > > > Zookeeper will have two paths /clients with client-id
> >> quotas
> >> > > that
> >> > > > >> apply
> >> > > > >> > > > only when user quota is unlimited similar to now. And
> >> /users
> >> > > which
> >> > > > >> > > > persists
> >> > > > >> > > > user quotas for any user including ANONYMOUS.
> >> > > > >> > > >
> >> > > > >> > > > Comments and feedback are appreciated.
> >> > > > >> > > >
> >> > > > >> > > > Regards,
> >> > > > >> > > >
> >> > > > >> > > > Rajini
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> >> > > > >> > > > <rajinisivaram@googlemail.com
> >> > > > >> > > > > wrote:
> >> > > > >> > > >
> >> > > > >> > > > > Jun,
> >> > > > >> > > > >
> >> > > > >> > > > > Oops, sorry, I hadn't realized that the last note was
> on
> >> the
> >> > > > >> discuss
> >> > > > >> > > > > thread. Thank you for pointing it out. I have sent
> >> another
> >> > > note
> >> > > > >> for
> >> > > > >> > > voting.
> >> > > > >> > > > >
> >> > > > >> > > > >
> >> > > > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <
> >> jun@confluent.io>
> >> > > > wrote:
> >> > > > >> > > > >
> >> > > > >> > > > >> Rajini,
> >> > > > >> > > > >>
> >> > > > >> > > > >> Perhaps it will be clearer if you start the voting in
> a
> >> new
> >> > > > >> thread
> >> > > > >> > > (with
> >> > > > >> > > > >> VOTE in the subject).
> >> > > > >> > > > >>
> >> > > > >> > > > >> Thanks,
> >> > > > >> > > > >>
> >> > > > >> > > > >> Jun
> >> > > > >> > > > >>
> >> > > > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> >> > > > >> > > > >> rajinisivaram@googlemail.com
> >> > > > >> > > > >> > wrote:
> >> > > > >> > > > >>
> >> > > > >> > > > >> > I would like to initiate the vote for KIP-55.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> >> > > > >> authenticated
> >> > > > >> > > users
> >> > > > >> > > > >> > <
> >> > > > >> > > > >> >
> >> > > > >> > > > >>
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> >> > > > >> > > > >> > >
> >> > > > >> > > > >> > .
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > The JIRA  KAFKA-3492  <
> >> > > > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> >> > > > >> > > > >> > >has
> >> > > > >> > > > >> > a draft PR here:
> >> > https://github.com/apache/kafka/pull/1256
> >> > > .
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Thank you...
> >> > > > >> > > > >> >
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Regards,
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Rajini
> >> > > > >> > > > >> >
> >> > > > >> > > > >>
> >> > > > >> > > > >
> >> > > > >> > > > >
> >> > > > >> > > > >
> >> > > > >> > > > > --
> >> > > > >> > > > > Regards,
> >> > > > >> > > > >
> >> > > > >> > > > > Rajini
> >> > > > >> > > > >
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > --
> >> > > > >> > > > Regards,
> >> > > > >> > > >
> >> > > > >> > > > Rajini
> >> > > > >> > >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > --
> >> > > > >> > Regards,
> >> > > > >> >
> >> > > > >> > Rajini
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Regards,
> >> > > > >
> >> > > > > Rajini
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Regards,
> >> > > >
> >> > > > Rajini
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> >
> >> > Rajini
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Gwen/Jay,

Have you had a chance to look at the updated KIP? It will be good to get
your feedback as well before restarting vote on the updated KIP.

If there are no objections, I will start the vote tomorrow.


On Fri, Jun 17, 2016 at 6:59 PM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Thank you, Jun. I have removed user_principal from the KIP.
>
> On Fri, Jun 17, 2016 at 6:00 PM, Jun Rao <ju...@confluent.io> wrote:
>
>> Rajini,
>>
>> 10. Yes, then we can probably leave out the user_principal field and keep
>> the version to be 1.
>>
>> Other than that, the KIP looks good to me.
>>
>> Thanks,
>>
>> Jun
>>
>> On Fri, Jun 17, 2016 at 3:29 AM, Rajini Sivaram <
>> rajinisivaram@googlemail.com> wrote:
>>
>> > Jun,
>> >
>> > 10. Since entity_type "users" is new, shouldn't the JSON for these
>> entities
>> > have version 1? I have moved "user_principal" out of the config in the
>> > samples and added to the <user, client> entries as well. But actually,
>> do
>> > we need to store the non-encoded principal at all? The node name is
>> > URL-encoded user principal, so it is fairly readable if you are looking
>> in
>> > ZK and *kafka_configs.sh* will show the non-encoded principal by
>> decoding
>> > the name from the path (since it needs to do encoding anyway because the
>> > names specified on the command line will be non-encoded principals, it
>> can
>> > do decoding too). Perhaps that is sufficient?
>> >
>> > 11. I liked the second approach since it looks neat and future-proof.
>> Have
>> > updated the KIP.
>> >
>> > 12. Yes, that is correct.
>> >
>> > Many thanks,
>> >
>> > Rajini
>> >
>> >
>> > On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:
>> >
>> > > Rajini,
>> > >
>> > > Thanks for the update. A few more questions/comments.
>> > >
>> > > 10. For the quota value stored in ZK, since we are adding an optional
>> > > user_principal field in the json, we should bump the version from 1
>> to 2.
>> > > Also, user_principal is not really part of the config values. So,
>> perhaps
>> > > we should represent it as the following?
>> > > {
>> > >     "version":2,
>> > >     "config": {
>> > >         "producer_byte_rate":"1024",
>> > >         "consumer_byte_rate":"2048"
>> > >     },
>> > >     "user_principal" : "user1"
>> > > }
>> > >
>> > >  Also, we should store user_principal in the following json too,
>> right?
>> > > // Zookeeper persistence path /users/<encoded-user2>/clients/clientA
>> > > {
>> > >     "version":1,
>> > >     "config": {
>> > >         "producer_byte_rate":"10",
>> > >         "consumer_byte_rate":"30"
>> > >     }
>> > > }
>> > >
>> > > 11. For the change notification path, would it be better to change it
>> to
>> > > something like the following and bump up version to 2?
>> > > // Change notification for quota of <user2, clientA>
>> > > {
>> > >     "version":2,
>> > >     [
>> > >       { "entity_type": "users",
>> > >         "entity_name": "user2"
>> > >       },
>> > >       { "entity_type": "clients",
>> > >         "entity_name": "clientA"
>> > >       }
>> > >     ]
>> > >  }
>> > >
>> > > Alternatively, we could change it to
>> > > // Change notification for quota of <user2, clientA>
>> > > {
>> > >     "version":2,
>> > >     "entity_path": "users/user2"
>> > > }
>> > >
>> > > {
>> > >     "version":2,
>> > >     "entity_path": "users/user2/clients/clientA"
>> > > }
>> > >
>> > > 12. Just to clarify on the meaning of remainder quota. If you have
>> quotas
>> > > like the following,
>> > >  <user1, client1> = 5
>> > >  <user1, client2> = 10
>> > >  <user1> = 12
>> > > it means that all connections with user1 whose client-id is neither
>> > client1
>> > > nor client2 will be sharing a quota of 12, right? In other words, the
>> > quota
>> > > of <user1> doesn't include the quota for <user1, client1> and <user1,
>> > > client2>.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
>> > > rajinisivaram@googlemail.com> wrote:
>> > >
>> > > > Jun,
>> > > >
>> > > > Actually, with quotas stored in different nodes in ZK, it is better
>> to
>> > > > store remainder quota rather than total quota under /users/<user> so
>> > that
>> > > > quota calculations are not dependent on the order of notifications.
>> I
>> > > have
>> > > > updated the KIP to reflect that. So the quotas in ZK now always
>> reflect
>> > > the
>> > > > quota applied to a group of client connections and use the same
>> format
>> > as
>> > > > client-id quotas. But it is not hierarchical, making the
>> configuration
>> > > > simpler.
>> > > >
>> > > > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
>> > > > rajinisivaram@googlemail.com> wrote:
>> > > >
>> > > > > Jun,
>> > > > >
>> > > > > Thank you for the review. I have updated the KIP:
>> > > > >
>> > > > >
>> > > > >    1. Added an overview section. Slightly reworded since it is
>> better
>> > > to
>> > > > >    treat user and client-id as different levels rather than the
>> same
>> > > > level.
>> > > > >    2. Yes, it is neater to store quota for each entity in a
>> different
>> > > > >    path in Zookeeper. I put clients under users rather than the
>> other
>> > > way
>> > > > >    round since that reflects the hierarchy and also keeps a user's
>> > > quotas
>> > > > >    together under a single sub-tree. I had initially used a single
>> > node
>> > > > to
>> > > > >    keep quotas and sub-quotas of a user together so that updates
>> are
>> > > > atomic
>> > > > >    since changes to sub-quotas also affect remainder quotas for
>> other
>> > > > clients.
>> > > > >    But I imagine, updates to configs are rare and it is not a big
>> > > issue.
>> > > > >    3. I haven't modified the JSON for configuration change
>> > > notifications.
>> > > > >    The entity_name can now be a subpath that has both user and
>> > client.
>> > > > Have
>> > > > >    added an example to the KIP. The downside of keeping clients
>> under
>> > > > users in
>> > > > >    ZK in 2) is that the change notification for sub-quota has
>> > > entity_type
>> > > > >    "users". I could extend the JSON to include client separately,
>> but
>> > > > since
>> > > > >    changes to a client sub-quota does impact other clients of the
>> > user
>> > > > as well
>> > > > >    due to change in remainder quota, it may be ok as it is. Do
>> let me
>> > > > know if
>> > > > >    it looks confusing in the example.
>> > > > >    4. Agree, updated.
>> > > > >
>> > > > >
>> > > > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io>
>> wrote:
>> > > > >
>> > > > >> Hi, Rajini,
>> > > > >>
>> > > > >> Thanks for the updated wiki. Overall, I like the new approach. It
>> > > covers
>> > > > >> the common use cases now, is extensible, and is backward
>> > compatible. A
>> > > > few
>> > > > >> comments below.
>> > > > >>
>> > > > >> 1. It would be useful to describe a bit at the high level, how
>> the
>> > new
>> > > > >> approach works. I think this can be summarized as follows. Quotas
>> > can
>> > > be
>> > > > >> set at user, client-id or <user, client-id> levels. For a given
>> > client
>> > > > >> connection, the most specific quota matching the connection will
>> be
>> > > > >> applied. For example, if both a <user, client-id> and a <user>
>> quota
>> > > > match
>> > > > >> a connection, the <user, client-id> quota will be used. If more
>> > than 1
>> > > > >> quota at the same level (e.g., a quota on a user and another
>> quota
>> > on
>> > > a
>> > > > >> client-id) match the connection, the user level quota will be
>> used,
>> > > > i.e.,
>> > > > >> user quota takes precedence over client-id quota.
>> > > > >>
>> > > > >> 2. For the ZK data structure, would it be better to store <user,
>> > > > >> client-id>
>> > > > >> quota as the following. Then the format of the value in each
>> path is
>> > > the
>> > > > >> same. The wiki also mentions that we want to include the original
>> > user
>> > > > >> name
>> > > > >> in the ZK value. Could you describe that in an example?
>> > > > >> // Zookeeper persistence path
>> /clients/clientA/users/<encoded-user2>
>> > > > >> {
>> > > > >>     "version":1,
>> > > > >>     "config": {
>> > > > >>         "producer_byte_rate":"10",
>> > > > >>         "consumer_byte_rate":"20"
>> > > > >>     }
>> > > > >> }
>> > > > >>
>> > > > >> 3. Could you document the format change of the ZK value in
>> > > > >> /config/changes/config_change_xxx, if any?
>> > > > >>
>> > > > >> 4. For the config command, could we specify the sub-quota like
>> the
>> > > > >> following, instead of in the config value? This seems more
>> > intuitive.
>> > > > >>
>> > > > >> bin/kafka-configs  --zookeeper localhost:2181 --alter
>> --add-config
>> > > > >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
>> > > > >> clientA --entity-type clients --entity-name user2 --entity-type
>> > users
>> > > > >>
>> > > > >> Thanks,
>> > > > >>
>> > > > >> Jun
>> > > > >>
>> > > > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
>> > > > >> rajinisivaram@googlemail.com> wrote:
>> > > > >>
>> > > > >> > Harsha,
>> > > > >> >
>> > > > >> > The sample configuration under
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
>> > > > >> > shows
>> > > > >> > the Zookeeper data for different scenarios.
>> > > > >> >
>> > > > >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
>> > > > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
>> > > > >> sub-quotas
>> > > > >> >    for clients *clientA* and *clientB*. Other client-ids of
>> > *user2*
>> > > > >> share
>> > > > >> >    the remaining quota of *user2*.
>> > > > >> >
>> > > > >> >
>> > > > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io>
>> wrote:
>> > > > >> >
>> > > > >> > > Rajini,
>> > > > >> > >           How does sub-quotas works in case of authenticated
>> > > users.
>> > > > >> > >           Where are we maintaining the relation between users
>> > and
>> > > > >> their
>> > > > >> > >           client Ids. Can you add an example of zk data under
>> > > > /users.
>> > > > >> > > Thanks,
>> > > > >> > > Harsha
>> > > > >> > >
>> > > > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
>> > > > >> > > > I have updated KIP-55 to reflect the changes from the
>> > > discussions
>> > > > in
>> > > > >> > the
>> > > > >> > > > voting thread (
>> > > > >> > > >
>> > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
>> > > ).
>> > > > >> > > >
>> > > > >> > > > Jun/Gwen,
>> > > > >> > > >
>> > > > >> > > > Existing client-id quotas will be used as default client-id
>> > > quotas
>> > > > >> for
>> > > > >> > > > users when no user quotas are configured - i.e., default
>> user
>> > > > quota
>> > > > >> is
>> > > > >> > > > unlimited and no user-specific quota override is specified.
>> > This
>> > > > >> > enables
>> > > > >> > > > user rate limits to be configured for ANONYMOUS if required
>> > in a
>> > > > >> > cluster
>> > > > >> > > > that has both PLAINTEXT and SSL/SASL. By default, without
>> any
>> > > user
>> > > > >> rate
>> > > > >> > > > limits set, rate limits for client-ids will apply,
>> retaining
>> > the
>> > > > >> > current
>> > > > >> > > > client-id quota configuration for single-user clusters.
>> > > > >> > > >
>> > > > >> > > > Zookeeper will have two paths /clients with client-id
>> quotas
>> > > that
>> > > > >> apply
>> > > > >> > > > only when user quota is unlimited similar to now. And
>> /users
>> > > which
>> > > > >> > > > persists
>> > > > >> > > > user quotas for any user including ANONYMOUS.
>> > > > >> > > >
>> > > > >> > > > Comments and feedback are appreciated.
>> > > > >> > > >
>> > > > >> > > > Regards,
>> > > > >> > > >
>> > > > >> > > > Rajini
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
>> > > > >> > > > <rajinisivaram@googlemail.com
>> > > > >> > > > > wrote:
>> > > > >> > > >
>> > > > >> > > > > Jun,
>> > > > >> > > > >
>> > > > >> > > > > Oops, sorry, I hadn't realized that the last note was on
>> the
>> > > > >> discuss
>> > > > >> > > > > thread. Thank you for pointing it out. I have sent
>> another
>> > > note
>> > > > >> for
>> > > > >> > > voting.
>> > > > >> > > > >
>> > > > >> > > > >
>> > > > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <
>> jun@confluent.io>
>> > > > wrote:
>> > > > >> > > > >
>> > > > >> > > > >> Rajini,
>> > > > >> > > > >>
>> > > > >> > > > >> Perhaps it will be clearer if you start the voting in a
>> new
>> > > > >> thread
>> > > > >> > > (with
>> > > > >> > > > >> VOTE in the subject).
>> > > > >> > > > >>
>> > > > >> > > > >> Thanks,
>> > > > >> > > > >>
>> > > > >> > > > >> Jun
>> > > > >> > > > >>
>> > > > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
>> > > > >> > > > >> rajinisivaram@googlemail.com
>> > > > >> > > > >> > wrote:
>> > > > >> > > > >>
>> > > > >> > > > >> > I would like to initiate the vote for KIP-55.
>> > > > >> > > > >> >
>> > > > >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
>> > > > >> authenticated
>> > > > >> > > users
>> > > > >> > > > >> > <
>> > > > >> > > > >> >
>> > > > >> > > > >>
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
>> > > > >> > > > >> > >
>> > > > >> > > > >> > .
>> > > > >> > > > >> >
>> > > > >> > > > >> > The JIRA  KAFKA-3492  <
>> > > > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
>> > > > >> > > > >> > >has
>> > > > >> > > > >> > a draft PR here:
>> > https://github.com/apache/kafka/pull/1256
>> > > .
>> > > > >> > > > >> >
>> > > > >> > > > >> > Thank you...
>> > > > >> > > > >> >
>> > > > >> > > > >> >
>> > > > >> > > > >> > Regards,
>> > > > >> > > > >> >
>> > > > >> > > > >> > Rajini
>> > > > >> > > > >> >
>> > > > >> > > > >>
>> > > > >> > > > >
>> > > > >> > > > >
>> > > > >> > > > >
>> > > > >> > > > > --
>> > > > >> > > > > Regards,
>> > > > >> > > > >
>> > > > >> > > > > Rajini
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > --
>> > > > >> > > > Regards,
>> > > > >> > > >
>> > > > >> > > > Rajini
>> > > > >> > >
>> > > > >> >
>> > > > >> >
>> > > > >> >
>> > > > >> > --
>> > > > >> > Regards,
>> > > > >> >
>> > > > >> > Rajini
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > >
>> > > > > Rajini
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Rajini
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Rajini
>> >
>>
>
>
>
> --
> Regards,
>
> Rajini
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Thank you, Jun. I have removed user_principal from the KIP.

On Fri, Jun 17, 2016 at 6:00 PM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> 10. Yes, then we can probably leave out the user_principal field and keep
> the version to be 1.
>
> Other than that, the KIP looks good to me.
>
> Thanks,
>
> Jun
>
> On Fri, Jun 17, 2016 at 3:29 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Jun,
> >
> > 10. Since entity_type "users" is new, shouldn't the JSON for these
> entities
> > have version 1? I have moved "user_principal" out of the config in the
> > samples and added to the <user, client> entries as well. But actually, do
> > we need to store the non-encoded principal at all? The node name is
> > URL-encoded user principal, so it is fairly readable if you are looking
> in
> > ZK and *kafka_configs.sh* will show the non-encoded principal by decoding
> > the name from the path (since it needs to do encoding anyway because the
> > names specified on the command line will be non-encoded principals, it
> can
> > do decoding too). Perhaps that is sufficient?
> >
> > 11. I liked the second approach since it looks neat and future-proof.
> Have
> > updated the KIP.
> >
> > 12. Yes, that is correct.
> >
> > Many thanks,
> >
> > Rajini
> >
> >
> > On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Rajini,
> > >
> > > Thanks for the update. A few more questions/comments.
> > >
> > > 10. For the quota value stored in ZK, since we are adding an optional
> > > user_principal field in the json, we should bump the version from 1 to
> 2.
> > > Also, user_principal is not really part of the config values. So,
> perhaps
> > > we should represent it as the following?
> > > {
> > >     "version":2,
> > >     "config": {
> > >         "producer_byte_rate":"1024",
> > >         "consumer_byte_rate":"2048"
> > >     },
> > >     "user_principal" : "user1"
> > > }
> > >
> > >  Also, we should store user_principal in the following json too, right?
> > > // Zookeeper persistence path /users/<encoded-user2>/clients/clientA
> > > {
> > >     "version":1,
> > >     "config": {
> > >         "producer_byte_rate":"10",
> > >         "consumer_byte_rate":"30"
> > >     }
> > > }
> > >
> > > 11. For the change notification path, would it be better to change it
> to
> > > something like the following and bump up version to 2?
> > > // Change notification for quota of <user2, clientA>
> > > {
> > >     "version":2,
> > >     [
> > >       { "entity_type": "users",
> > >         "entity_name": "user2"
> > >       },
> > >       { "entity_type": "clients",
> > >         "entity_name": "clientA"
> > >       }
> > >     ]
> > >  }
> > >
> > > Alternatively, we could change it to
> > > // Change notification for quota of <user2, clientA>
> > > {
> > >     "version":2,
> > >     "entity_path": "users/user2"
> > > }
> > >
> > > {
> > >     "version":2,
> > >     "entity_path": "users/user2/clients/clientA"
> > > }
> > >
> > > 12. Just to clarify on the meaning of remainder quota. If you have
> quotas
> > > like the following,
> > >  <user1, client1> = 5
> > >  <user1, client2> = 10
> > >  <user1> = 12
> > > it means that all connections with user1 whose client-id is neither
> > client1
> > > nor client2 will be sharing a quota of 12, right? In other words, the
> > quota
> > > of <user1> doesn't include the quota for <user1, client1> and <user1,
> > > client2>.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > Jun,
> > > >
> > > > Actually, with quotas stored in different nodes in ZK, it is better
> to
> > > > store remainder quota rather than total quota under /users/<user> so
> > that
> > > > quota calculations are not dependent on the order of notifications. I
> > > have
> > > > updated the KIP to reflect that. So the quotas in ZK now always
> reflect
> > > the
> > > > quota applied to a group of client connections and use the same
> format
> > as
> > > > client-id quotas. But it is not hierarchical, making the
> configuration
> > > > simpler.
> > > >
> > > > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> > > > rajinisivaram@googlemail.com> wrote:
> > > >
> > > > > Jun,
> > > > >
> > > > > Thank you for the review. I have updated the KIP:
> > > > >
> > > > >
> > > > >    1. Added an overview section. Slightly reworded since it is
> better
> > > to
> > > > >    treat user and client-id as different levels rather than the
> same
> > > > level.
> > > > >    2. Yes, it is neater to store quota for each entity in a
> different
> > > > >    path in Zookeeper. I put clients under users rather than the
> other
> > > way
> > > > >    round since that reflects the hierarchy and also keeps a user's
> > > quotas
> > > > >    together under a single sub-tree. I had initially used a single
> > node
> > > > to
> > > > >    keep quotas and sub-quotas of a user together so that updates
> are
> > > > atomic
> > > > >    since changes to sub-quotas also affect remainder quotas for
> other
> > > > clients.
> > > > >    But I imagine, updates to configs are rare and it is not a big
> > > issue.
> > > > >    3. I haven't modified the JSON for configuration change
> > > notifications.
> > > > >    The entity_name can now be a subpath that has both user and
> > client.
> > > > Have
> > > > >    added an example to the KIP. The downside of keeping clients
> under
> > > > users in
> > > > >    ZK in 2) is that the change notification for sub-quota has
> > > entity_type
> > > > >    "users". I could extend the JSON to include client separately,
> but
> > > > since
> > > > >    changes to a client sub-quota does impact other clients of the
> > user
> > > > as well
> > > > >    due to change in remainder quota, it may be ok as it is. Do let
> me
> > > > know if
> > > > >    it looks confusing in the example.
> > > > >    4. Agree, updated.
> > > > >
> > > > >
> > > > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > >
> > > > >> Hi, Rajini,
> > > > >>
> > > > >> Thanks for the updated wiki. Overall, I like the new approach. It
> > > covers
> > > > >> the common use cases now, is extensible, and is backward
> > compatible. A
> > > > few
> > > > >> comments below.
> > > > >>
> > > > >> 1. It would be useful to describe a bit at the high level, how the
> > new
> > > > >> approach works. I think this can be summarized as follows. Quotas
> > can
> > > be
> > > > >> set at user, client-id or <user, client-id> levels. For a given
> > client
> > > > >> connection, the most specific quota matching the connection will
> be
> > > > >> applied. For example, if both a <user, client-id> and a <user>
> quota
> > > > match
> > > > >> a connection, the <user, client-id> quota will be used. If more
> > than 1
> > > > >> quota at the same level (e.g., a quota on a user and another quota
> > on
> > > a
> > > > >> client-id) match the connection, the user level quota will be
> used,
> > > > i.e.,
> > > > >> user quota takes precedence over client-id quota.
> > > > >>
> > > > >> 2. For the ZK data structure, would it be better to store <user,
> > > > >> client-id>
> > > > >> quota as the following. Then the format of the value in each path
> is
> > > the
> > > > >> same. The wiki also mentions that we want to include the original
> > user
> > > > >> name
> > > > >> in the ZK value. Could you describe that in an example?
> > > > >> // Zookeeper persistence path
> /clients/clientA/users/<encoded-user2>
> > > > >> {
> > > > >>     "version":1,
> > > > >>     "config": {
> > > > >>         "producer_byte_rate":"10",
> > > > >>         "consumer_byte_rate":"20"
> > > > >>     }
> > > > >> }
> > > > >>
> > > > >> 3. Could you document the format change of the ZK value in
> > > > >> /config/changes/config_change_xxx, if any?
> > > > >>
> > > > >> 4. For the config command, could we specify the sub-quota like the
> > > > >> following, instead of in the config value? This seems more
> > intuitive.
> > > > >>
> > > > >> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> > > > >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> > > > >> clientA --entity-type clients --entity-name user2 --entity-type
> > users
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> > > > >> rajinisivaram@googlemail.com> wrote:
> > > > >>
> > > > >> > Harsha,
> > > > >> >
> > > > >> > The sample configuration under
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> > > > >> > shows
> > > > >> > the Zookeeper data for different scenarios.
> > > > >> >
> > > > >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> > > > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
> > > > >> sub-quotas
> > > > >> >    for clients *clientA* and *clientB*. Other client-ids of
> > *user2*
> > > > >> share
> > > > >> >    the remaining quota of *user2*.
> > > > >> >
> > > > >> >
> > > > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io>
> wrote:
> > > > >> >
> > > > >> > > Rajini,
> > > > >> > >           How does sub-quotas works in case of authenticated
> > > users.
> > > > >> > >           Where are we maintaining the relation between users
> > and
> > > > >> their
> > > > >> > >           client Ids. Can you add an example of zk data under
> > > > /users.
> > > > >> > > Thanks,
> > > > >> > > Harsha
> > > > >> > >
> > > > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > > > >> > > > I have updated KIP-55 to reflect the changes from the
> > > discussions
> > > > in
> > > > >> > the
> > > > >> > > > voting thread (
> > > > >> > > >
> > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
> > > ).
> > > > >> > > >
> > > > >> > > > Jun/Gwen,
> > > > >> > > >
> > > > >> > > > Existing client-id quotas will be used as default client-id
> > > quotas
> > > > >> for
> > > > >> > > > users when no user quotas are configured - i.e., default
> user
> > > > quota
> > > > >> is
> > > > >> > > > unlimited and no user-specific quota override is specified.
> > This
> > > > >> > enables
> > > > >> > > > user rate limits to be configured for ANONYMOUS if required
> > in a
> > > > >> > cluster
> > > > >> > > > that has both PLAINTEXT and SSL/SASL. By default, without
> any
> > > user
> > > > >> rate
> > > > >> > > > limits set, rate limits for client-ids will apply, retaining
> > the
> > > > >> > current
> > > > >> > > > client-id quota configuration for single-user clusters.
> > > > >> > > >
> > > > >> > > > Zookeeper will have two paths /clients with client-id quotas
> > > that
> > > > >> apply
> > > > >> > > > only when user quota is unlimited similar to now. And /users
> > > which
> > > > >> > > > persists
> > > > >> > > > user quotas for any user including ANONYMOUS.
> > > > >> > > >
> > > > >> > > > Comments and feedback are appreciated.
> > > > >> > > >
> > > > >> > > > Regards,
> > > > >> > > >
> > > > >> > > > Rajini
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > > > >> > > > <rajinisivaram@googlemail.com
> > > > >> > > > > wrote:
> > > > >> > > >
> > > > >> > > > > Jun,
> > > > >> > > > >
> > > > >> > > > > Oops, sorry, I hadn't realized that the last note was on
> the
> > > > >> discuss
> > > > >> > > > > thread. Thank you for pointing it out. I have sent another
> > > note
> > > > >> for
> > > > >> > > voting.
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <jun@confluent.io
> >
> > > > wrote:
> > > > >> > > > >
> > > > >> > > > >> Rajini,
> > > > >> > > > >>
> > > > >> > > > >> Perhaps it will be clearer if you start the voting in a
> new
> > > > >> thread
> > > > >> > > (with
> > > > >> > > > >> VOTE in the subject).
> > > > >> > > > >>
> > > > >> > > > >> Thanks,
> > > > >> > > > >>
> > > > >> > > > >> Jun
> > > > >> > > > >>
> > > > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > > > >> > > > >> rajinisivaram@googlemail.com
> > > > >> > > > >> > wrote:
> > > > >> > > > >>
> > > > >> > > > >> > I would like to initiate the vote for KIP-55.
> > > > >> > > > >> >
> > > > >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> > > > >> authenticated
> > > > >> > > users
> > > > >> > > > >> > <
> > > > >> > > > >> >
> > > > >> > > > >>
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > > > >> > > > >> > >
> > > > >> > > > >> > .
> > > > >> > > > >> >
> > > > >> > > > >> > The JIRA  KAFKA-3492  <
> > > > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> > > > >> > > > >> > >has
> > > > >> > > > >> > a draft PR here:
> > https://github.com/apache/kafka/pull/1256
> > > .
> > > > >> > > > >> >
> > > > >> > > > >> > Thank you...
> > > > >> > > > >> >
> > > > >> > > > >> >
> > > > >> > > > >> > Regards,
> > > > >> > > > >> >
> > > > >> > > > >> > Rajini
> > > > >> > > > >> >
> > > > >> > > > >>
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > Regards,
> > > > >> > > > >
> > > > >> > > > > Rajini
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Regards,
> > > > >> > > >
> > > > >> > > > Rajini
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Regards,
> > > > >> >
> > > > >> > Rajini
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Rajini
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

10. Yes, then we can probably leave out the user_principal field and keep
the version to be 1.

Other than that, the KIP looks good to me.

Thanks,

Jun

On Fri, Jun 17, 2016 at 3:29 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Jun,
>
> 10. Since entity_type "users" is new, shouldn't the JSON for these entities
> have version 1? I have moved "user_principal" out of the config in the
> samples and added to the <user, client> entries as well. But actually, do
> we need to store the non-encoded principal at all? The node name is
> URL-encoded user principal, so it is fairly readable if you are looking in
> ZK and *kafka_configs.sh* will show the non-encoded principal by decoding
> the name from the path (since it needs to do encoding anyway because the
> names specified on the command line will be non-encoded principals, it can
> do decoding too). Perhaps that is sufficient?
>
> 11. I liked the second approach since it looks neat and future-proof. Have
> updated the KIP.
>
> 12. Yes, that is correct.
>
> Many thanks,
>
> Rajini
>
>
> On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Rajini,
> >
> > Thanks for the update. A few more questions/comments.
> >
> > 10. For the quota value stored in ZK, since we are adding an optional
> > user_principal field in the json, we should bump the version from 1 to 2.
> > Also, user_principal is not really part of the config values. So, perhaps
> > we should represent it as the following?
> > {
> >     "version":2,
> >     "config": {
> >         "producer_byte_rate":"1024",
> >         "consumer_byte_rate":"2048"
> >     },
> >     "user_principal" : "user1"
> > }
> >
> >  Also, we should store user_principal in the following json too, right?
> > // Zookeeper persistence path /users/<encoded-user2>/clients/clientA
> > {
> >     "version":1,
> >     "config": {
> >         "producer_byte_rate":"10",
> >         "consumer_byte_rate":"30"
> >     }
> > }
> >
> > 11. For the change notification path, would it be better to change it to
> > something like the following and bump up version to 2?
> > // Change notification for quota of <user2, clientA>
> > {
> >     "version":2,
> >     [
> >       { "entity_type": "users",
> >         "entity_name": "user2"
> >       },
> >       { "entity_type": "clients",
> >         "entity_name": "clientA"
> >       }
> >     ]
> >  }
> >
> > Alternatively, we could change it to
> > // Change notification for quota of <user2, clientA>
> > {
> >     "version":2,
> >     "entity_path": "users/user2"
> > }
> >
> > {
> >     "version":2,
> >     "entity_path": "users/user2/clients/clientA"
> > }
> >
> > 12. Just to clarify on the meaning of remainder quota. If you have quotas
> > like the following,
> >  <user1, client1> = 5
> >  <user1, client2> = 10
> >  <user1> = 12
> > it means that all connections with user1 whose client-id is neither
> client1
> > nor client2 will be sharing a quota of 12, right? In other words, the
> quota
> > of <user1> doesn't include the quota for <user1, client1> and <user1,
> > client2>.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > Jun,
> > >
> > > Actually, with quotas stored in different nodes in ZK, it is better to
> > > store remainder quota rather than total quota under /users/<user> so
> that
> > > quota calculations are not dependent on the order of notifications. I
> > have
> > > updated the KIP to reflect that. So the quotas in ZK now always reflect
> > the
> > > quota applied to a group of client connections and use the same format
> as
> > > client-id quotas. But it is not hierarchical, making the configuration
> > > simpler.
> > >
> > > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > Jun,
> > > >
> > > > Thank you for the review. I have updated the KIP:
> > > >
> > > >
> > > >    1. Added an overview section. Slightly reworded since it is better
> > to
> > > >    treat user and client-id as different levels rather than the same
> > > level.
> > > >    2. Yes, it is neater to store quota for each entity in a different
> > > >    path in Zookeeper. I put clients under users rather than the other
> > way
> > > >    round since that reflects the hierarchy and also keeps a user's
> > quotas
> > > >    together under a single sub-tree. I had initially used a single
> node
> > > to
> > > >    keep quotas and sub-quotas of a user together so that updates are
> > > atomic
> > > >    since changes to sub-quotas also affect remainder quotas for other
> > > clients.
> > > >    But I imagine, updates to configs are rare and it is not a big
> > issue.
> > > >    3. I haven't modified the JSON for configuration change
> > notifications.
> > > >    The entity_name can now be a subpath that has both user and
> client.
> > > Have
> > > >    added an example to the KIP. The downside of keeping clients under
> > > users in
> > > >    ZK in 2) is that the change notification for sub-quota has
> > entity_type
> > > >    "users". I could extend the JSON to include client separately, but
> > > since
> > > >    changes to a client sub-quota does impact other clients of the
> user
> > > as well
> > > >    due to change in remainder quota, it may be ok as it is. Do let me
> > > know if
> > > >    it looks confusing in the example.
> > > >    4. Agree, updated.
> > > >
> > > >
> > > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > >> Hi, Rajini,
> > > >>
> > > >> Thanks for the updated wiki. Overall, I like the new approach. It
> > covers
> > > >> the common use cases now, is extensible, and is backward
> compatible. A
> > > few
> > > >> comments below.
> > > >>
> > > >> 1. It would be useful to describe a bit at the high level, how the
> new
> > > >> approach works. I think this can be summarized as follows. Quotas
> can
> > be
> > > >> set at user, client-id or <user, client-id> levels. For a given
> client
> > > >> connection, the most specific quota matching the connection will be
> > > >> applied. For example, if both a <user, client-id> and a <user> quota
> > > match
> > > >> a connection, the <user, client-id> quota will be used. If more
> than 1
> > > >> quota at the same level (e.g., a quota on a user and another quota
> on
> > a
> > > >> client-id) match the connection, the user level quota will be used,
> > > i.e.,
> > > >> user quota takes precedence over client-id quota.
> > > >>
> > > >> 2. For the ZK data structure, would it be better to store <user,
> > > >> client-id>
> > > >> quota as the following. Then the format of the value in each path is
> > the
> > > >> same. The wiki also mentions that we want to include the original
> user
> > > >> name
> > > >> in the ZK value. Could you describe that in an example?
> > > >> // Zookeeper persistence path /clients/clientA/users/<encoded-user2>
> > > >> {
> > > >>     "version":1,
> > > >>     "config": {
> > > >>         "producer_byte_rate":"10",
> > > >>         "consumer_byte_rate":"20"
> > > >>     }
> > > >> }
> > > >>
> > > >> 3. Could you document the format change of the ZK value in
> > > >> /config/changes/config_change_xxx, if any?
> > > >>
> > > >> 4. For the config command, could we specify the sub-quota like the
> > > >> following, instead of in the config value? This seems more
> intuitive.
> > > >>
> > > >> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> > > >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> > > >> clientA --entity-type clients --entity-name user2 --entity-type
> users
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> > > >> rajinisivaram@googlemail.com> wrote:
> > > >>
> > > >> > Harsha,
> > > >> >
> > > >> > The sample configuration under
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> > > >> > shows
> > > >> > the Zookeeper data for different scenarios.
> > > >> >
> > > >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> > > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
> > > >> sub-quotas
> > > >> >    for clients *clientA* and *clientB*. Other client-ids of
> *user2*
> > > >> share
> > > >> >    the remaining quota of *user2*.
> > > >> >
> > > >> >
> > > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
> > > >> >
> > > >> > > Rajini,
> > > >> > >           How does sub-quotas works in case of authenticated
> > users.
> > > >> > >           Where are we maintaining the relation between users
> and
> > > >> their
> > > >> > >           client Ids. Can you add an example of zk data under
> > > /users.
> > > >> > > Thanks,
> > > >> > > Harsha
> > > >> > >
> > > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > > >> > > > I have updated KIP-55 to reflect the changes from the
> > discussions
> > > in
> > > >> > the
> > > >> > > > voting thread (
> > > >> > > >
> https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
> > ).
> > > >> > > >
> > > >> > > > Jun/Gwen,
> > > >> > > >
> > > >> > > > Existing client-id quotas will be used as default client-id
> > quotas
> > > >> for
> > > >> > > > users when no user quotas are configured - i.e., default user
> > > quota
> > > >> is
> > > >> > > > unlimited and no user-specific quota override is specified.
> This
> > > >> > enables
> > > >> > > > user rate limits to be configured for ANONYMOUS if required
> in a
> > > >> > cluster
> > > >> > > > that has both PLAINTEXT and SSL/SASL. By default, without any
> > user
> > > >> rate
> > > >> > > > limits set, rate limits for client-ids will apply, retaining
> the
> > > >> > current
> > > >> > > > client-id quota configuration for single-user clusters.
> > > >> > > >
> > > >> > > > Zookeeper will have two paths /clients with client-id quotas
> > that
> > > >> apply
> > > >> > > > only when user quota is unlimited similar to now. And /users
> > which
> > > >> > > > persists
> > > >> > > > user quotas for any user including ANONYMOUS.
> > > >> > > >
> > > >> > > > Comments and feedback are appreciated.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > >
> > > >> > > > Rajini
> > > >> > > >
> > > >> > > >
> > > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > > >> > > > <rajinisivaram@googlemail.com
> > > >> > > > > wrote:
> > > >> > > >
> > > >> > > > > Jun,
> > > >> > > > >
> > > >> > > > > Oops, sorry, I hadn't realized that the last note was on the
> > > >> discuss
> > > >> > > > > thread. Thank you for pointing it out. I have sent another
> > note
> > > >> for
> > > >> > > voting.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io>
> > > wrote:
> > > >> > > > >
> > > >> > > > >> Rajini,
> > > >> > > > >>
> > > >> > > > >> Perhaps it will be clearer if you start the voting in a new
> > > >> thread
> > > >> > > (with
> > > >> > > > >> VOTE in the subject).
> > > >> > > > >>
> > > >> > > > >> Thanks,
> > > >> > > > >>
> > > >> > > > >> Jun
> > > >> > > > >>
> > > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > > >> > > > >> rajinisivaram@googlemail.com
> > > >> > > > >> > wrote:
> > > >> > > > >>
> > > >> > > > >> > I would like to initiate the vote for KIP-55.
> > > >> > > > >> >
> > > >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> > > >> authenticated
> > > >> > > users
> > > >> > > > >> > <
> > > >> > > > >> >
> > > >> > > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > > >> > > > >> > >
> > > >> > > > >> > .
> > > >> > > > >> >
> > > >> > > > >> > The JIRA  KAFKA-3492  <
> > > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> > > >> > > > >> > >has
> > > >> > > > >> > a draft PR here:
> https://github.com/apache/kafka/pull/1256
> > .
> > > >> > > > >> >
> > > >> > > > >> > Thank you...
> > > >> > > > >> >
> > > >> > > > >> >
> > > >> > > > >> > Regards,
> > > >> > > > >> >
> > > >> > > > >> > Rajini
> > > >> > > > >> >
> > > >> > > > >>
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > Regards,
> > > >> > > > >
> > > >> > > > > Rajini
> > > >> > > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > --
> > > >> > > > Regards,
> > > >> > > >
> > > >> > > > Rajini
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Regards,
> > > >> >
> > > >> > Rajini
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

10. Since entity_type "users" is new, shouldn't the JSON for these entities
have version 1? I have moved "user_principal" out of the config in the
samples and added to the <user, client> entries as well. But actually, do
we need to store the non-encoded principal at all? The node name is
URL-encoded user principal, so it is fairly readable if you are looking in
ZK and *kafka_configs.sh* will show the non-encoded principal by decoding
the name from the path (since it needs to do encoding anyway because the
names specified on the command line will be non-encoded principals, it can
do decoding too). Perhaps that is sufficient?

11. I liked the second approach since it looks neat and future-proof. Have
updated the KIP.

12. Yes, that is correct.

Many thanks,

Rajini


On Thu, Jun 16, 2016 at 11:36 PM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> Thanks for the update. A few more questions/comments.
>
> 10. For the quota value stored in ZK, since we are adding an optional
> user_principal field in the json, we should bump the version from 1 to 2.
> Also, user_principal is not really part of the config values. So, perhaps
> we should represent it as the following?
> {
>     "version":2,
>     "config": {
>         "producer_byte_rate":"1024",
>         "consumer_byte_rate":"2048"
>     },
>     "user_principal" : "user1"
> }
>
>  Also, we should store user_principal in the following json too, right?
> // Zookeeper persistence path /users/<encoded-user2>/clients/clientA
> {
>     "version":1,
>     "config": {
>         "producer_byte_rate":"10",
>         "consumer_byte_rate":"30"
>     }
> }
>
> 11. For the change notification path, would it be better to change it to
> something like the following and bump up version to 2?
> // Change notification for quota of <user2, clientA>
> {
>     "version":2,
>     [
>       { "entity_type": "users",
>         "entity_name": "user2"
>       },
>       { "entity_type": "clients",
>         "entity_name": "clientA"
>       }
>     ]
>  }
>
> Alternatively, we could change it to
> // Change notification for quota of <user2, clientA>
> {
>     "version":2,
>     "entity_path": "users/user2"
> }
>
> {
>     "version":2,
>     "entity_path": "users/user2/clients/clientA"
> }
>
> 12. Just to clarify on the meaning of remainder quota. If you have quotas
> like the following,
>  <user1, client1> = 5
>  <user1, client2> = 10
>  <user1> = 12
> it means that all connections with user1 whose client-id is neither client1
> nor client2 will be sharing a quota of 12, right? In other words, the quota
> of <user1> doesn't include the quota for <user1, client1> and <user1,
> client2>.
>
> Thanks,
>
> Jun
>
>
> On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Jun,
> >
> > Actually, with quotas stored in different nodes in ZK, it is better to
> > store remainder quota rather than total quota under /users/<user> so that
> > quota calculations are not dependent on the order of notifications. I
> have
> > updated the KIP to reflect that. So the quotas in ZK now always reflect
> the
> > quota applied to a group of client connections and use the same format as
> > client-id quotas. But it is not hierarchical, making the configuration
> > simpler.
> >
> > On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> > > Jun,
> > >
> > > Thank you for the review. I have updated the KIP:
> > >
> > >
> > >    1. Added an overview section. Slightly reworded since it is better
> to
> > >    treat user and client-id as different levels rather than the same
> > level.
> > >    2. Yes, it is neater to store quota for each entity in a different
> > >    path in Zookeeper. I put clients under users rather than the other
> way
> > >    round since that reflects the hierarchy and also keeps a user's
> quotas
> > >    together under a single sub-tree. I had initially used a single node
> > to
> > >    keep quotas and sub-quotas of a user together so that updates are
> > atomic
> > >    since changes to sub-quotas also affect remainder quotas for other
> > clients.
> > >    But I imagine, updates to configs are rare and it is not a big
> issue.
> > >    3. I haven't modified the JSON for configuration change
> notifications.
> > >    The entity_name can now be a subpath that has both user and client.
> > Have
> > >    added an example to the KIP. The downside of keeping clients under
> > users in
> > >    ZK in 2) is that the change notification for sub-quota has
> entity_type
> > >    "users". I could extend the JSON to include client separately, but
> > since
> > >    changes to a client sub-quota does impact other clients of the user
> > as well
> > >    due to change in remainder quota, it may be ok as it is. Do let me
> > know if
> > >    it looks confusing in the example.
> > >    4. Agree, updated.
> > >
> > >
> > > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > >> Hi, Rajini,
> > >>
> > >> Thanks for the updated wiki. Overall, I like the new approach. It
> covers
> > >> the common use cases now, is extensible, and is backward compatible. A
> > few
> > >> comments below.
> > >>
> > >> 1. It would be useful to describe a bit at the high level, how the new
> > >> approach works. I think this can be summarized as follows. Quotas can
> be
> > >> set at user, client-id or <user, client-id> levels. For a given client
> > >> connection, the most specific quota matching the connection will be
> > >> applied. For example, if both a <user, client-id> and a <user> quota
> > match
> > >> a connection, the <user, client-id> quota will be used. If more than 1
> > >> quota at the same level (e.g., a quota on a user and another quota on
> a
> > >> client-id) match the connection, the user level quota will be used,
> > i.e.,
> > >> user quota takes precedence over client-id quota.
> > >>
> > >> 2. For the ZK data structure, would it be better to store <user,
> > >> client-id>
> > >> quota as the following. Then the format of the value in each path is
> the
> > >> same. The wiki also mentions that we want to include the original user
> > >> name
> > >> in the ZK value. Could you describe that in an example?
> > >> // Zookeeper persistence path /clients/clientA/users/<encoded-user2>
> > >> {
> > >>     "version":1,
> > >>     "config": {
> > >>         "producer_byte_rate":"10",
> > >>         "consumer_byte_rate":"20"
> > >>     }
> > >> }
> > >>
> > >> 3. Could you document the format change of the ZK value in
> > >> /config/changes/config_change_xxx, if any?
> > >>
> > >> 4. For the config command, could we specify the sub-quota like the
> > >> following, instead of in the config value? This seems more intuitive.
> > >>
> > >> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> > >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> > >> clientA --entity-type clients --entity-name user2 --entity-type users
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> > >> rajinisivaram@googlemail.com> wrote:
> > >>
> > >> > Harsha,
> > >> >
> > >> > The sample configuration under
> > >> >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> > >> > shows
> > >> > the Zookeeper data for different scenarios.
> > >> >
> > >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> > >> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
> > >> sub-quotas
> > >> >    for clients *clientA* and *clientB*. Other client-ids of *user2*
> > >> share
> > >> >    the remaining quota of *user2*.
> > >> >
> > >> >
> > >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
> > >> >
> > >> > > Rajini,
> > >> > >           How does sub-quotas works in case of authenticated
> users.
> > >> > >           Where are we maintaining the relation between users and
> > >> their
> > >> > >           client Ids. Can you add an example of zk data under
> > /users.
> > >> > > Thanks,
> > >> > > Harsha
> > >> > >
> > >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > >> > > > I have updated KIP-55 to reflect the changes from the
> discussions
> > in
> > >> > the
> > >> > > > voting thread (
> > >> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html
> ).
> > >> > > >
> > >> > > > Jun/Gwen,
> > >> > > >
> > >> > > > Existing client-id quotas will be used as default client-id
> quotas
> > >> for
> > >> > > > users when no user quotas are configured - i.e., default user
> > quota
> > >> is
> > >> > > > unlimited and no user-specific quota override is specified. This
> > >> > enables
> > >> > > > user rate limits to be configured for ANONYMOUS if required in a
> > >> > cluster
> > >> > > > that has both PLAINTEXT and SSL/SASL. By default, without any
> user
> > >> rate
> > >> > > > limits set, rate limits for client-ids will apply, retaining the
> > >> > current
> > >> > > > client-id quota configuration for single-user clusters.
> > >> > > >
> > >> > > > Zookeeper will have two paths /clients with client-id quotas
> that
> > >> apply
> > >> > > > only when user quota is unlimited similar to now. And /users
> which
> > >> > > > persists
> > >> > > > user quotas for any user including ANONYMOUS.
> > >> > > >
> > >> > > > Comments and feedback are appreciated.
> > >> > > >
> > >> > > > Regards,
> > >> > > >
> > >> > > > Rajini
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > >> > > > <rajinisivaram@googlemail.com
> > >> > > > > wrote:
> > >> > > >
> > >> > > > > Jun,
> > >> > > > >
> > >> > > > > Oops, sorry, I hadn't realized that the last note was on the
> > >> discuss
> > >> > > > > thread. Thank you for pointing it out. I have sent another
> note
> > >> for
> > >> > > voting.
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > >> > > > >
> > >> > > > >> Rajini,
> > >> > > > >>
> > >> > > > >> Perhaps it will be clearer if you start the voting in a new
> > >> thread
> > >> > > (with
> > >> > > > >> VOTE in the subject).
> > >> > > > >>
> > >> > > > >> Thanks,
> > >> > > > >>
> > >> > > > >> Jun
> > >> > > > >>
> > >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > >> > > > >> rajinisivaram@googlemail.com
> > >> > > > >> > wrote:
> > >> > > > >>
> > >> > > > >> > I would like to initiate the vote for KIP-55.
> > >> > > > >> >
> > >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> > >> authenticated
> > >> > > users
> > >> > > > >> > <
> > >> > > > >> >
> > >> > > > >>
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > >> > > > >> > >
> > >> > > > >> > .
> > >> > > > >> >
> > >> > > > >> > The JIRA  KAFKA-3492  <
> > >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> > >> > > > >> > >has
> > >> > > > >> > a draft PR here: https://github.com/apache/kafka/pull/1256
> .
> > >> > > > >> >
> > >> > > > >> > Thank you...
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Regards,
> > >> > > > >> >
> > >> > > > >> > Rajini
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Regards,
> > >> > > > >
> > >> > > > > Rajini
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Regards,
> > >> > > >
> > >> > > > Rajini
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Regards,
> > >> >
> > >> > Rajini
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

Thanks for the update. A few more questions/comments.

10. For the quota value stored in ZK, since we are adding an optional
user_principal field in the json, we should bump the version from 1 to 2.
Also, user_principal is not really part of the config values. So, perhaps
we should represent it as the following?
{
    "version":2,
    "config": {
        "producer_byte_rate":"1024",
        "consumer_byte_rate":"2048"
    },
    "user_principal" : "user1"
}

 Also, we should store user_principal in the following json too, right?
// Zookeeper persistence path /users/<encoded-user2>/clients/clientA
{
    "version":1,
    "config": {
        "producer_byte_rate":"10",
        "consumer_byte_rate":"30"
    }
}

11. For the change notification path, would it be better to change it to
something like the following and bump up version to 2?
// Change notification for quota of <user2, clientA>
{
    "version":2,
    [
      { "entity_type": "users",
        "entity_name": "user2"
      },
      { "entity_type": "clients",
        "entity_name": "clientA"
      }
    ]
 }

Alternatively, we could change it to
// Change notification for quota of <user2, clientA>
{
    "version":2,
    "entity_path": "users/user2"
}

{
    "version":2,
    "entity_path": "users/user2/clients/clientA"
}

12. Just to clarify on the meaning of remainder quota. If you have quotas
like the following,
 <user1, client1> = 5
 <user1, client2> = 10
 <user1> = 12
it means that all connections with user1 whose client-id is neither client1
nor client2 will be sharing a quota of 12, right? In other words, the quota
of <user1> doesn't include the quota for <user1, client1> and <user1,
client2>.

Thanks,

Jun


On Thu, Jun 16, 2016 at 5:03 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Jun,
>
> Actually, with quotas stored in different nodes in ZK, it is better to
> store remainder quota rather than total quota under /users/<user> so that
> quota calculations are not dependent on the order of notifications. I have
> updated the KIP to reflect that. So the quotas in ZK now always reflect the
> quota applied to a group of client connections and use the same format as
> client-id quotas. But it is not hierarchical, making the configuration
> simpler.
>
> On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Jun,
> >
> > Thank you for the review. I have updated the KIP:
> >
> >
> >    1. Added an overview section. Slightly reworded since it is better to
> >    treat user and client-id as different levels rather than the same
> level.
> >    2. Yes, it is neater to store quota for each entity in a different
> >    path in Zookeeper. I put clients under users rather than the other way
> >    round since that reflects the hierarchy and also keeps a user's quotas
> >    together under a single sub-tree. I had initially used a single node
> to
> >    keep quotas and sub-quotas of a user together so that updates are
> atomic
> >    since changes to sub-quotas also affect remainder quotas for other
> clients.
> >    But I imagine, updates to configs are rare and it is not a big issue.
> >    3. I haven't modified the JSON for configuration change notifications.
> >    The entity_name can now be a subpath that has both user and client.
> Have
> >    added an example to the KIP. The downside of keeping clients under
> users in
> >    ZK in 2) is that the change notification for sub-quota has entity_type
> >    "users". I could extend the JSON to include client separately, but
> since
> >    changes to a client sub-quota does impact other clients of the user
> as well
> >    due to change in remainder quota, it may be ok as it is. Do let me
> know if
> >    it looks confusing in the example.
> >    4. Agree, updated.
> >
> >
> > On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Hi, Rajini,
> >>
> >> Thanks for the updated wiki. Overall, I like the new approach. It covers
> >> the common use cases now, is extensible, and is backward compatible. A
> few
> >> comments below.
> >>
> >> 1. It would be useful to describe a bit at the high level, how the new
> >> approach works. I think this can be summarized as follows. Quotas can be
> >> set at user, client-id or <user, client-id> levels. For a given client
> >> connection, the most specific quota matching the connection will be
> >> applied. For example, if both a <user, client-id> and a <user> quota
> match
> >> a connection, the <user, client-id> quota will be used. If more than 1
> >> quota at the same level (e.g., a quota on a user and another quota on a
> >> client-id) match the connection, the user level quota will be used,
> i.e.,
> >> user quota takes precedence over client-id quota.
> >>
> >> 2. For the ZK data structure, would it be better to store <user,
> >> client-id>
> >> quota as the following. Then the format of the value in each path is the
> >> same. The wiki also mentions that we want to include the original user
> >> name
> >> in the ZK value. Could you describe that in an example?
> >> // Zookeeper persistence path /clients/clientA/users/<encoded-user2>
> >> {
> >>     "version":1,
> >>     "config": {
> >>         "producer_byte_rate":"10",
> >>         "consumer_byte_rate":"20"
> >>     }
> >> }
> >>
> >> 3. Could you document the format change of the ZK value in
> >> /config/changes/config_change_xxx, if any?
> >>
> >> 4. For the config command, could we specify the sub-quota like the
> >> following, instead of in the config value? This seems more intuitive.
> >>
> >> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> >> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> >> clientA --entity-type clients --entity-name user2 --entity-type users
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> >> rajinisivaram@googlemail.com> wrote:
> >>
> >> > Harsha,
> >> >
> >> > The sample configuration under
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> >> > shows
> >> > the Zookeeper data for different scenarios.
> >> >
> >> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> >> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
> >> sub-quotas
> >> >    for clients *clientA* and *clientB*. Other client-ids of *user2*
> >> share
> >> >    the remaining quota of *user2*.
> >> >
> >> >
> >> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
> >> >
> >> > > Rajini,
> >> > >           How does sub-quotas works in case of authenticated users.
> >> > >           Where are we maintaining the relation between users and
> >> their
> >> > >           client Ids. Can you add an example of zk data under
> /users.
> >> > > Thanks,
> >> > > Harsha
> >> > >
> >> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> >> > > > I have updated KIP-55 to reflect the changes from the discussions
> in
> >> > the
> >> > > > voting thread (
> >> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
> >> > > >
> >> > > > Jun/Gwen,
> >> > > >
> >> > > > Existing client-id quotas will be used as default client-id quotas
> >> for
> >> > > > users when no user quotas are configured - i.e., default user
> quota
> >> is
> >> > > > unlimited and no user-specific quota override is specified. This
> >> > enables
> >> > > > user rate limits to be configured for ANONYMOUS if required in a
> >> > cluster
> >> > > > that has both PLAINTEXT and SSL/SASL. By default, without any user
> >> rate
> >> > > > limits set, rate limits for client-ids will apply, retaining the
> >> > current
> >> > > > client-id quota configuration for single-user clusters.
> >> > > >
> >> > > > Zookeeper will have two paths /clients with client-id quotas that
> >> apply
> >> > > > only when user quota is unlimited similar to now. And /users which
> >> > > > persists
> >> > > > user quotas for any user including ANONYMOUS.
> >> > > >
> >> > > > Comments and feedback are appreciated.
> >> > > >
> >> > > > Regards,
> >> > > >
> >> > > > Rajini
> >> > > >
> >> > > >
> >> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> >> > > > <rajinisivaram@googlemail.com
> >> > > > > wrote:
> >> > > >
> >> > > > > Jun,
> >> > > > >
> >> > > > > Oops, sorry, I hadn't realized that the last note was on the
> >> discuss
> >> > > > > thread. Thank you for pointing it out. I have sent another note
> >> for
> >> > > voting.
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io>
> wrote:
> >> > > > >
> >> > > > >> Rajini,
> >> > > > >>
> >> > > > >> Perhaps it will be clearer if you start the voting in a new
> >> thread
> >> > > (with
> >> > > > >> VOTE in the subject).
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >>
> >> > > > >> Jun
> >> > > > >>
> >> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> >> > > > >> rajinisivaram@googlemail.com
> >> > > > >> > wrote:
> >> > > > >>
> >> > > > >> > I would like to initiate the vote for KIP-55.
> >> > > > >> >
> >> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> >> authenticated
> >> > > users
> >> > > > >> > <
> >> > > > >> >
> >> > > > >>
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> >> > > > >> > >
> >> > > > >> > .
> >> > > > >> >
> >> > > > >> > The JIRA  KAFKA-3492  <
> >> > > https://issues.apache.org/jira/browse/KAFKA-3492
> >> > > > >> > >has
> >> > > > >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> >> > > > >> >
> >> > > > >> > Thank you...
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > Regards,
> >> > > > >> >
> >> > > > >> > Rajini
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Regards,
> >> > > > >
> >> > > > > Rajini
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Regards,
> >> > > >
> >> > > > Rajini
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> >
> >> > Rajini
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

Actually, with quotas stored in different nodes in ZK, it is better to
store remainder quota rather than total quota under /users/<user> so that
quota calculations are not dependent on the order of notifications. I have
updated the KIP to reflect that. So the quotas in ZK now always reflect the
quota applied to a group of client connections and use the same format as
client-id quotas. But it is not hierarchical, making the configuration
simpler.

On Thu, Jun 16, 2016 at 11:49 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Jun,
>
> Thank you for the review. I have updated the KIP:
>
>
>    1. Added an overview section. Slightly reworded since it is better to
>    treat user and client-id as different levels rather than the same level.
>    2. Yes, it is neater to store quota for each entity in a different
>    path in Zookeeper. I put clients under users rather than the other way
>    round since that reflects the hierarchy and also keeps a user's quotas
>    together under a single sub-tree. I had initially used a single node to
>    keep quotas and sub-quotas of a user together so that updates are atomic
>    since changes to sub-quotas also affect remainder quotas for other clients.
>    But I imagine, updates to configs are rare and it is not a big issue.
>    3. I haven't modified the JSON for configuration change notifications.
>    The entity_name can now be a subpath that has both user and client. Have
>    added an example to the KIP. The downside of keeping clients under users in
>    ZK in 2) is that the change notification for sub-quota has entity_type
>    "users". I could extend the JSON to include client separately, but since
>    changes to a client sub-quota does impact other clients of the user as well
>    due to change in remainder quota, it may be ok as it is. Do let me know if
>    it looks confusing in the example.
>    4. Agree, updated.
>
>
> On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io> wrote:
>
>> Hi, Rajini,
>>
>> Thanks for the updated wiki. Overall, I like the new approach. It covers
>> the common use cases now, is extensible, and is backward compatible. A few
>> comments below.
>>
>> 1. It would be useful to describe a bit at the high level, how the new
>> approach works. I think this can be summarized as follows. Quotas can be
>> set at user, client-id or <user, client-id> levels. For a given client
>> connection, the most specific quota matching the connection will be
>> applied. For example, if both a <user, client-id> and a <user> quota match
>> a connection, the <user, client-id> quota will be used. If more than 1
>> quota at the same level (e.g., a quota on a user and another quota on a
>> client-id) match the connection, the user level quota will be used, i.e.,
>> user quota takes precedence over client-id quota.
>>
>> 2. For the ZK data structure, would it be better to store <user,
>> client-id>
>> quota as the following. Then the format of the value in each path is the
>> same. The wiki also mentions that we want to include the original user
>> name
>> in the ZK value. Could you describe that in an example?
>> // Zookeeper persistence path /clients/clientA/users/<encoded-user2>
>> {
>>     "version":1,
>>     "config": {
>>         "producer_byte_rate":"10",
>>         "consumer_byte_rate":"20"
>>     }
>> }
>>
>> 3. Could you document the format change of the ZK value in
>> /config/changes/config_change_xxx, if any?
>>
>> 4. For the config command, could we specify the sub-quota like the
>> following, instead of in the config value? This seems more intuitive.
>>
>> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
>> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
>> clientA --entity-type clients --entity-name user2 --entity-type users
>>
>> Thanks,
>>
>> Jun
>>
>> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
>> rajinisivaram@googlemail.com> wrote:
>>
>> > Harsha,
>> >
>> > The sample configuration under
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
>> > shows
>> > the Zookeeper data for different scenarios.
>> >
>> >    - *user1* (/users/user1 in ZK) has only user-level quotas
>> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
>> sub-quotas
>> >    for clients *clientA* and *clientB*. Other client-ids of *user2*
>> share
>> >    the remaining quota of *user2*.
>> >
>> >
>> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
>> >
>> > > Rajini,
>> > >           How does sub-quotas works in case of authenticated users.
>> > >           Where are we maintaining the relation between users and
>> their
>> > >           client Ids. Can you add an example of zk data under /users.
>> > > Thanks,
>> > > Harsha
>> > >
>> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
>> > > > I have updated KIP-55 to reflect the changes from the discussions in
>> > the
>> > > > voting thread (
>> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
>> > > >
>> > > > Jun/Gwen,
>> > > >
>> > > > Existing client-id quotas will be used as default client-id quotas
>> for
>> > > > users when no user quotas are configured - i.e., default user quota
>> is
>> > > > unlimited and no user-specific quota override is specified. This
>> > enables
>> > > > user rate limits to be configured for ANONYMOUS if required in a
>> > cluster
>> > > > that has both PLAINTEXT and SSL/SASL. By default, without any user
>> rate
>> > > > limits set, rate limits for client-ids will apply, retaining the
>> > current
>> > > > client-id quota configuration for single-user clusters.
>> > > >
>> > > > Zookeeper will have two paths /clients with client-id quotas that
>> apply
>> > > > only when user quota is unlimited similar to now. And /users which
>> > > > persists
>> > > > user quotas for any user including ANONYMOUS.
>> > > >
>> > > > Comments and feedback are appreciated.
>> > > >
>> > > > Regards,
>> > > >
>> > > > Rajini
>> > > >
>> > > >
>> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
>> > > > <rajinisivaram@googlemail.com
>> > > > > wrote:
>> > > >
>> > > > > Jun,
>> > > > >
>> > > > > Oops, sorry, I hadn't realized that the last note was on the
>> discuss
>> > > > > thread. Thank you for pointing it out. I have sent another note
>> for
>> > > voting.
>> > > > >
>> > > > >
>> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
>> > > > >
>> > > > >> Rajini,
>> > > > >>
>> > > > >> Perhaps it will be clearer if you start the voting in a new
>> thread
>> > > (with
>> > > > >> VOTE in the subject).
>> > > > >>
>> > > > >> Thanks,
>> > > > >>
>> > > > >> Jun
>> > > > >>
>> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
>> > > > >> rajinisivaram@googlemail.com
>> > > > >> > wrote:
>> > > > >>
>> > > > >> > I would like to initiate the vote for KIP-55.
>> > > > >> >
>> > > > >> > The KIP details are here: KIP-55: Secure quotas for
>> authenticated
>> > > users
>> > > > >> > <
>> > > > >> >
>> > > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
>> > > > >> > >
>> > > > >> > .
>> > > > >> >
>> > > > >> > The JIRA  KAFKA-3492  <
>> > > https://issues.apache.org/jira/browse/KAFKA-3492
>> > > > >> > >has
>> > > > >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
>> > > > >> >
>> > > > >> > Thank you...
>> > > > >> >
>> > > > >> >
>> > > > >> > Regards,
>> > > > >> >
>> > > > >> > Rajini
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > >
>> > > > > Rajini
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Rajini
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Rajini
>> >
>>
>
>
>
> --
> Regards,
>
> Rajini
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

Thank you for the review. I have updated the KIP:


   1. Added an overview section. Slightly reworded since it is better to
   treat user and client-id as different levels rather than the same level.
   2. Yes, it is neater to store quota for each entity in a different path
   in Zookeeper. I put clients under users rather than the other way round
   since that reflects the hierarchy and also keeps a user's quotas together
   under a single sub-tree. I had initially used a single node to keep quotas
   and sub-quotas of a user together so that updates are atomic since changes
   to sub-quotas also affect remainder quotas for other clients. But I
   imagine, updates to configs are rare and it is not a big issue.
   3. I haven't modified the JSON for configuration change notifications.
   The entity_name can now be a subpath that has both user and client. Have
   added an example to the KIP. The downside of keeping clients under users in
   ZK in 2) is that the change notification for sub-quota has entity_type
   "users". I could extend the JSON to include client separately, but since
   changes to a client sub-quota does impact other clients of the user as well
   due to change in remainder quota, it may be ok as it is. Do let me know if
   it looks confusing in the example.
   4. Agree, updated.


On Wed, Jun 15, 2016 at 10:27 PM, Jun Rao <ju...@confluent.io> wrote:

> Hi, Rajini,
>
> Thanks for the updated wiki. Overall, I like the new approach. It covers
> the common use cases now, is extensible, and is backward compatible. A few
> comments below.
>
> 1. It would be useful to describe a bit at the high level, how the new
> approach works. I think this can be summarized as follows. Quotas can be
> set at user, client-id or <user, client-id> levels. For a given client
> connection, the most specific quota matching the connection will be
> applied. For example, if both a <user, client-id> and a <user> quota match
> a connection, the <user, client-id> quota will be used. If more than 1
> quota at the same level (e.g., a quota on a user and another quota on a
> client-id) match the connection, the user level quota will be used, i.e.,
> user quota takes precedence over client-id quota.
>
> 2. For the ZK data structure, would it be better to store <user, client-id>
> quota as the following. Then the format of the value in each path is the
> same. The wiki also mentions that we want to include the original user name
> in the ZK value. Could you describe that in an example?
> // Zookeeper persistence path /clients/clientA/users/<encoded-user2>
> {
>     "version":1,
>     "config": {
>         "producer_byte_rate":"10",
>         "consumer_byte_rate":"20"
>     }
> }
>
> 3. Could you document the format change of the ZK value in
> /config/changes/config_change_xxx, if any?
>
> 4. For the config command, could we specify the sub-quota like the
> following, instead of in the config value? This seems more intuitive.
>
> bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
> 'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
> clientA --entity-type clients --entity-name user2 --entity-type users
>
> Thanks,
>
> Jun
>
> On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
>
> > Harsha,
> >
> > The sample configuration under
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> > shows
> > the Zookeeper data for different scenarios.
> >
> >    - *user1* (/users/user1 in ZK) has only user-level quotas
> >    - *user2* (/users/user2 in ZK) defines user-level quotas and
> sub-quotas
> >    for clients *clientA* and *clientB*. Other client-ids of *user2* share
> >    the remaining quota of *user2*.
> >
> >
> > On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
> >
> > > Rajini,
> > >           How does sub-quotas works in case of authenticated users.
> > >           Where are we maintaining the relation between users and their
> > >           client Ids. Can you add an example of zk data under /users.
> > > Thanks,
> > > Harsha
> > >
> > > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > > > I have updated KIP-55 to reflect the changes from the discussions in
> > the
> > > > voting thread (
> > > > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
> > > >
> > > > Jun/Gwen,
> > > >
> > > > Existing client-id quotas will be used as default client-id quotas
> for
> > > > users when no user quotas are configured - i.e., default user quota
> is
> > > > unlimited and no user-specific quota override is specified. This
> > enables
> > > > user rate limits to be configured for ANONYMOUS if required in a
> > cluster
> > > > that has both PLAINTEXT and SSL/SASL. By default, without any user
> rate
> > > > limits set, rate limits for client-ids will apply, retaining the
> > current
> > > > client-id quota configuration for single-user clusters.
> > > >
> > > > Zookeeper will have two paths /clients with client-id quotas that
> apply
> > > > only when user quota is unlimited similar to now. And /users which
> > > > persists
> > > > user quotas for any user including ANONYMOUS.
> > > >
> > > > Comments and feedback are appreciated.
> > > >
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > > >
> > > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > > > <rajinisivaram@googlemail.com
> > > > > wrote:
> > > >
> > > > > Jun,
> > > > >
> > > > > Oops, sorry, I hadn't realized that the last note was on the
> discuss
> > > > > thread. Thank you for pointing it out. I have sent another note for
> > > voting.
> > > > >
> > > > >
> > > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > >> Rajini,
> > > > >>
> > > > >> Perhaps it will be clearer if you start the voting in a new thread
> > > (with
> > > > >> VOTE in the subject).
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > > > >> rajinisivaram@googlemail.com
> > > > >> > wrote:
> > > > >>
> > > > >> > I would like to initiate the vote for KIP-55.
> > > > >> >
> > > > >> > The KIP details are here: KIP-55: Secure quotas for
> authenticated
> > > users
> > > > >> > <
> > > > >> >
> > > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > > > >> > >
> > > > >> > .
> > > > >> >
> > > > >> > The JIRA  KAFKA-3492  <
> > > https://issues.apache.org/jira/browse/KAFKA-3492
> > > > >> > >has
> > > > >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> > > > >> >
> > > > >> > Thank you...
> > > > >> >
> > > > >> >
> > > > >> > Regards,
> > > > >> >
> > > > >> > Rajini
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Rajini
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Hi, Rajini,

Thanks for the updated wiki. Overall, I like the new approach. It covers
the common use cases now, is extensible, and is backward compatible. A few
comments below.

1. It would be useful to describe a bit at the high level, how the new
approach works. I think this can be summarized as follows. Quotas can be
set at user, client-id or <user, client-id> levels. For a given client
connection, the most specific quota matching the connection will be
applied. For example, if both a <user, client-id> and a <user> quota match
a connection, the <user, client-id> quota will be used. If more than 1
quota at the same level (e.g., a quota on a user and another quota on a
client-id) match the connection, the user level quota will be used, i.e.,
user quota takes precedence over client-id quota.

2. For the ZK data structure, would it be better to store <user, client-id>
quota as the following. Then the format of the value in each path is the
same. The wiki also mentions that we want to include the original user name
in the ZK value. Could you describe that in an example?
// Zookeeper persistence path /clients/clientA/users/<encoded-user2>
{
    "version":1,
    "config": {
        "producer_byte_rate":"10",
        "consumer_byte_rate":"20"
    }
}

3. Could you document the format change of the ZK value in
/config/changes/config_change_xxx, if any?

4. For the config command, could we specify the sub-quota like the
following, instead of in the config value? This seems more intuitive.

bin/kafka-configs  --zookeeper localhost:2181 --alter --add-config
'producer_byte_rate=1024,consumer_byte_rate=2048' --entity-name
clientA --entity-type clients --entity-name user2 --entity-type users

Thanks,

Jun

On Wed, Jun 15, 2016 at 10:35 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> Harsha,
>
> The sample configuration under
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
> shows
> the Zookeeper data for different scenarios.
>
>    - *user1* (/users/user1 in ZK) has only user-level quotas
>    - *user2* (/users/user2 in ZK) defines user-level quotas and sub-quotas
>    for clients *clientA* and *clientB*. Other client-ids of *user2* share
>    the remaining quota of *user2*.
>
>
> On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:
>
> > Rajini,
> >           How does sub-quotas works in case of authenticated users.
> >           Where are we maintaining the relation between users and their
> >           client Ids. Can you add an example of zk data under /users.
> > Thanks,
> > Harsha
> >
> > On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > > I have updated KIP-55 to reflect the changes from the discussions in
> the
> > > voting thread (
> > > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
> > >
> > > Jun/Gwen,
> > >
> > > Existing client-id quotas will be used as default client-id quotas for
> > > users when no user quotas are configured - i.e., default user quota is
> > > unlimited and no user-specific quota override is specified. This
> enables
> > > user rate limits to be configured for ANONYMOUS if required in a
> cluster
> > > that has both PLAINTEXT and SSL/SASL. By default, without any user rate
> > > limits set, rate limits for client-ids will apply, retaining the
> current
> > > client-id quota configuration for single-user clusters.
> > >
> > > Zookeeper will have two paths /clients with client-id quotas that apply
> > > only when user quota is unlimited similar to now. And /users which
> > > persists
> > > user quotas for any user including ANONYMOUS.
> > >
> > > Comments and feedback are appreciated.
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > >
> > > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > > <rajinisivaram@googlemail.com
> > > > wrote:
> > >
> > > > Jun,
> > > >
> > > > Oops, sorry, I hadn't realized that the last note was on the discuss
> > > > thread. Thank you for pointing it out. I have sent another note for
> > voting.
> > > >
> > > >
> > > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > >> Rajini,
> > > >>
> > > >> Perhaps it will be clearer if you start the voting in a new thread
> > (with
> > > >> VOTE in the subject).
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > > >> rajinisivaram@googlemail.com
> > > >> > wrote:
> > > >>
> > > >> > I would like to initiate the vote for KIP-55.
> > > >> >
> > > >> > The KIP details are here: KIP-55: Secure quotas for authenticated
> > users
> > > >> > <
> > > >> >
> > > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > > >> > >
> > > >> > .
> > > >> >
> > > >> > The JIRA  KAFKA-3492  <
> > https://issues.apache.org/jira/browse/KAFKA-3492
> > > >> > >has
> > > >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> > > >> >
> > > >> > Thank you...
> > > >> >
> > > >> >
> > > >> > Regards,
> > > >> >
> > > >> > Rajini
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> >
>
>
>
> --
> Regards,
>
> Rajini
>

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Harsha,

The sample configuration under
https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users#KIP-55:SecureQuotasforAuthenticatedUsers-QuotaConfiguration
shows
the Zookeeper data for different scenarios.

   - *user1* (/users/user1 in ZK) has only user-level quotas
   - *user2* (/users/user2 in ZK) defines user-level quotas and sub-quotas
   for clients *clientA* and *clientB*. Other client-ids of *user2* share
   the remaining quota of *user2*.


On Wed, Jun 15, 2016 at 5:30 PM, Harsha <ka...@harsha.io> wrote:

> Rajini,
>           How does sub-quotas works in case of authenticated users.
>           Where are we maintaining the relation between users and their
>           client Ids. Can you add an example of zk data under /users.
> Thanks,
> Harsha
>
> On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> > I have updated KIP-55 to reflect the changes from the discussions in the
> > voting thread (
> > https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
> >
> > Jun/Gwen,
> >
> > Existing client-id quotas will be used as default client-id quotas for
> > users when no user quotas are configured - i.e., default user quota is
> > unlimited and no user-specific quota override is specified. This enables
> > user rate limits to be configured for ANONYMOUS if required in a cluster
> > that has both PLAINTEXT and SSL/SASL. By default, without any user rate
> > limits set, rate limits for client-ids will apply, retaining the current
> > client-id quota configuration for single-user clusters.
> >
> > Zookeeper will have two paths /clients with client-id quotas that apply
> > only when user quota is unlimited similar to now. And /users which
> > persists
> > user quotas for any user including ANONYMOUS.
> >
> > Comments and feedback are appreciated.
> >
> > Regards,
> >
> > Rajini
> >
> >
> > On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> > <rajinisivaram@googlemail.com
> > > wrote:
> >
> > > Jun,
> > >
> > > Oops, sorry, I hadn't realized that the last note was on the discuss
> > > thread. Thank you for pointing it out. I have sent another note for
> voting.
> > >
> > >
> > > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > >> Rajini,
> > >>
> > >> Perhaps it will be clearer if you start the voting in a new thread
> (with
> > >> VOTE in the subject).
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> > >> rajinisivaram@googlemail.com
> > >> > wrote:
> > >>
> > >> > I would like to initiate the vote for KIP-55.
> > >> >
> > >> > The KIP details are here: KIP-55: Secure quotas for authenticated
> users
> > >> > <
> > >> >
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > >> > >
> > >> > .
> > >> >
> > >> > The JIRA  KAFKA-3492  <
> https://issues.apache.org/jira/browse/KAFKA-3492
> > >> > >has
> > >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> > >> >
> > >> > Thank you...
> > >> >
> > >> >
> > >> > Regards,
> > >> >
> > >> > Rajini
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Rajini
> > >
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Harsha <ka...@harsha.io>.
Rajini,
          How does sub-quotas works in case of authenticated users.
          Where are we maintaining the relation between users and their
          client Ids. Can you add an example of zk data under /users.
Thanks,
Harsha

On Mon, Jun 13, 2016, at 05:01 AM, Rajini Sivaram wrote:
> I have updated KIP-55 to reflect the changes from the discussions in the
> voting thread (
> https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).
> 
> Jun/Gwen,
> 
> Existing client-id quotas will be used as default client-id quotas for
> users when no user quotas are configured - i.e., default user quota is
> unlimited and no user-specific quota override is specified. This enables
> user rate limits to be configured for ANONYMOUS if required in a cluster
> that has both PLAINTEXT and SSL/SASL. By default, without any user rate
> limits set, rate limits for client-ids will apply, retaining the current
> client-id quota configuration for single-user clusters.
> 
> Zookeeper will have two paths /clients with client-id quotas that apply
> only when user quota is unlimited similar to now. And /users which
> persists
> user quotas for any user including ANONYMOUS.
> 
> Comments and feedback are appreciated.
> 
> Regards,
> 
> Rajini
> 
> 
> On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram
> <rajinisivaram@googlemail.com
> > wrote:
> 
> > Jun,
> >
> > Oops, sorry, I hadn't realized that the last note was on the discuss
> > thread. Thank you for pointing it out. I have sent another note for voting.
> >
> >
> > On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Rajini,
> >>
> >> Perhaps it will be clearer if you start the voting in a new thread (with
> >> VOTE in the subject).
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> >> rajinisivaram@googlemail.com
> >> > wrote:
> >>
> >> > I would like to initiate the vote for KIP-55.
> >> >
> >> > The KIP details are here: KIP-55: Secure quotas for authenticated users
> >> > <
> >> >
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> >> > >
> >> > .
> >> >
> >> > The JIRA  KAFKA-3492  <https://issues.apache.org/jira/browse/KAFKA-3492
> >> > >has
> >> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> >> >
> >> > Thank you...
> >> >
> >> >
> >> > Regards,
> >> >
> >> > Rajini
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> >
> > Rajini
> >
> 
> 
> 
> -- 
> Regards,
> 
> Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
I have updated KIP-55 to reflect the changes from the discussions in the
voting thread (
https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html).

Jun/Gwen,

Existing client-id quotas will be used as default client-id quotas for
users when no user quotas are configured - i.e., default user quota is
unlimited and no user-specific quota override is specified. This enables
user rate limits to be configured for ANONYMOUS if required in a cluster
that has both PLAINTEXT and SSL/SASL. By default, without any user rate
limits set, rate limits for client-ids will apply, retaining the current
client-id quota configuration for single-user clusters.

Zookeeper will have two paths /clients with client-id quotas that apply
only when user quota is unlimited similar to now. And /users which persists
user quotas for any user including ANONYMOUS.

Comments and feedback are appreciated.

Regards,

Rajini


On Wed, Jun 8, 2016 at 9:00 PM, Rajini Sivaram <rajinisivaram@googlemail.com
> wrote:

> Jun,
>
> Oops, sorry, I hadn't realized that the last note was on the discuss
> thread. Thank you for pointing it out. I have sent another note for voting.
>
>
> On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:
>
>> Rajini,
>>
>> Perhaps it will be clearer if you start the voting in a new thread (with
>> VOTE in the subject).
>>
>> Thanks,
>>
>> Jun
>>
>> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
>> rajinisivaram@googlemail.com
>> > wrote:
>>
>> > I would like to initiate the vote for KIP-55.
>> >
>> > The KIP details are here: KIP-55: Secure quotas for authenticated users
>> > <
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
>> > >
>> > .
>> >
>> > The JIRA  KAFKA-3492  <https://issues.apache.org/jira/browse/KAFKA-3492
>> > >has
>> > a draft PR here: https://github.com/apache/kafka/pull/1256.
>> >
>> > Thank you...
>> >
>> >
>> > Regards,
>> >
>> > Rajini
>> >
>>
>
>
>
> --
> Regards,
>
> Rajini
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Rajini Sivaram <ra...@googlemail.com>.
Jun,

Oops, sorry, I hadn't realized that the last note was on the discuss
thread. Thank you for pointing it out. I have sent another note for voting.


On Wed, Jun 8, 2016 at 4:30 PM, Jun Rao <ju...@confluent.io> wrote:

> Rajini,
>
> Perhaps it will be clearer if you start the voting in a new thread (with
> VOTE in the subject).
>
> Thanks,
>
> Jun
>
> On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <
> rajinisivaram@googlemail.com
> > wrote:
>
> > I would like to initiate the vote for KIP-55.
> >
> > The KIP details are here: KIP-55: Secure quotas for authenticated users
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> > >
> > .
> >
> > The JIRA  KAFKA-3492  <https://issues.apache.org/jira/browse/KAFKA-3492
> > >has
> > a draft PR here: https://github.com/apache/kafka/pull/1256.
> >
> > Thank you...
> >
> >
> > Regards,
> >
> > Rajini
> >
>



-- 
Regards,

Rajini

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Posted by Jun Rao <ju...@confluent.io>.
Rajini,

Perhaps it will be clearer if you start the voting in a new thread (with
VOTE in the subject).

Thanks,

Jun

On Tue, Jun 7, 2016 at 1:55 PM, Rajini Sivaram <rajinisivaram@googlemail.com
> wrote:

> I would like to initiate the vote for KIP-55.
>
> The KIP details are here: KIP-55: Secure quotas for authenticated users
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-55%3A+Secure+Quotas+for+Authenticated+Users
> >
> .
>
> The JIRA  KAFKA-3492  <https://issues.apache.org/jira/browse/KAFKA-3492
> >has
> a draft PR here: https://github.com/apache/kafka/pull/1256.
>
> Thank you...
>
>
> Regards,
>
> Rajini
>