You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Manikumar <ma...@gmail.com> on 2016/12/23 17:26:18 UTC
[VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi,
I would like to initiate the vote on KIP-48:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
Delegation+token+support+for+Kafka
Thanks,
Manikumar
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Jason,
As noticed by you, the current proposal does not support rotation of secret.
We also discussed about maintaining a list of secret keys. Other option
could
be using the controller to generate and rotate secret and distribute it to
all brokers.
I will update the possible alternatives to future work section of KIP.
On Wed, Feb 8, 2017 at 6:40 AM, Jason Gustafson <ja...@confluent.io> wrote:
> Looks like a great proposal! I noticed that key rotation is not included.
> That may be reasonable for the initial work, but it might be nice to share
> some thoughts on how that might work in the future. For example, I could
> imagine delegation.token.master.key could be a list, which would allow
> users to support both a new and old key at the same time while clients are
> upgrading keys.
>
> -Jason
>
> On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Read the KIP again and I think it looks good.
> >
> > +1 from me.
> >
> > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > > Hi, Mani,
> > >
> > > If a token expires, then every broker will potentially try to delete it
> > > around the same time, but only one will succeed. So, we will have to
> deal
> > > with failures in that case? Another way is to let just one broker (say,
> > the
> > > controller) deletes expired tokens.
> > >
> > > It would also be helpful for others to give feedback on this KIP.
> Rajini,
> > > Gwen, Ismael?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> > wrote:
> > >
> > >> Hi Jun,
> > >>
> > >> Please see the replies inline.
> > >>
> > >>
> > >> > >
> > >> > > Only one broker does the deletion. Broker updates the expiration
> in
> > its
> > >> > > local cache
> > >> > > and on zookeeper so other brokers also get notified and their
> cache
> > >> > > statuses are updated as well.
> > >> > >
> > >> > >
> > >> > Which broker does the deletion?
> > >> >
> > >>
> > >> Any broker can handle the create/expire/renew/describe delegationtoken
> > >> requests.
> > >> changes are propagated through zk notifications. Every broker is
> > >> responsible for
> > >> expiring the tokens. This check be can done during request handling
> time
> > >> and/or
> > >> during token authentication time.
> > >>
> > >>
> > >> >
> > >> >
> > >> > 110. The diagrams in the wiki still show MD5 digest. Could you
> change
> > it
> > >> to
> > >> > SCRAM?
> > >> >
> > >> >
> > >> Updated the diagram.
> > >>
> > >>
> > >>
> > >> Thanks,
> > >> Manikumar
> > >>
> > >>
> > >>
> > >>
> > >> >
> > >> >
> > >> > >
> > >> > > Thanks.
> > >> > > Manikumar
> > >> > >
> > >> > >
> > >> > > >
> > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > >> manikumar.reddy@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi,
> > >> > > > >
> > >> > > > > I would like to initiate the vote on KIP-48:
> > >> > > > >
> > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > >> > > > > Delegation+token+support+for+Kafka
> > >> > > > >
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Manikumar
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Roger Hoover <ro...@gmail.com>.
Thanks. I found the other discussion thread. Sorry for being behind on
this.
I'm interested in the future impersonation use cases. This seems to get us
closer.
+1 (non-binding)
On Wed, Feb 8, 2017 at 4:41 AM, Manikumar <ma...@gmail.com> wrote:
> Hi Roger,
>
> In the current proposal, we only allow a user to get delegation token for
> that user only.
> Anyone who gets that token can impersonate the user on the broker.
>
> Yes, In future we can extend the support to allow a user to acquire
> delegation tokens for
> other users.
>
> Pl refer discuss mail thread for impersonation related discussion.
>
> Thanks,
> Manikumar
>
> On Wed, Feb 8, 2017 at 8:37 AM, Roger Hoover <ro...@gmail.com>
> wrote:
>
> > Hi Jun,
> >
> > How does it allow impersonation at the connection level? Looking at the
> > KIP, the DelegationTokenRequest does not have an "Owner" field that can
> be
> > set. The owner field of the DelegationTokenResponse says it's the
> "Kakfa
> > Principal which requested the delegation token". For impersonation,
> don't
> > we need to be able to get tokens for other users besides the one making
> the
> > request?
> >
> > Thanks,
> >
> > Roger
> >
> > On Tue, Feb 7, 2017 at 6:45 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Roger,
> > >
> > > Just to clarify. This KIP already allows you to do impersonation at the
> > > connection level. Are you talking about impersonation at the request
> > level?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover <ro...@gmail.com>
> > > wrote:
> > >
> > > > Just wondering...how difficult would be it be to later add
> > impersonation
> > > (
> > > > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case
> would
> > > be
> > > > a
> > > > Kafka admin UI that would take action on the cluster on behalf
> > different
> > > > users. I suppose we could later add an "effectiveUserId" (in Unix
> > > > terminology) to the token details?
> > > >
> > > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > > >
> > > > > +1 from me as well.
> > > > >
> > > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <
> jason@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > Looks like a great proposal! I noticed that key rotation is not
> > > > included.
> > > > > > That may be reasonable for the initial work, but it might be nice
> > to
> > > > > share
> > > > > > some thoughts on how that might work in the future. For example,
> I
> > > > could
> > > > > > imagine delegation.token.master.key could be a list, which would
> > > allow
> > > > > > users to support both a new and old key at the same time while
> > > clients
> > > > > are
> > > > > > upgrading keys.
> > > > > >
> > > > > > -Jason
> > > > > >
> > > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io>
> > > > wrote:
> > > > > >
> > > > > > > Read the KIP again and I think it looks good.
> > > > > > >
> > > > > > > +1 from me.
> > > > > > >
> > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > > > Hi, Mani,
> > > > > > > >
> > > > > > > > If a token expires, then every broker will potentially try to
> > > > delete
> > > > > it
> > > > > > > > around the same time, but only one will succeed. So, we will
> > have
> > > > to
> > > > > > deal
> > > > > > > > with failures in that case? Another way is to let just one
> > broker
> > > > > (say,
> > > > > > > the
> > > > > > > > controller) deletes expired tokens.
> > > > > > > >
> > > > > > > > It would also be helpful for others to give feedback on this
> > KIP.
> > > > > > Rajini,
> > > > > > > > Gwen, Ismael?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <
> > > > manikumar.reddy@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Jun,
> > > > > > > >>
> > > > > > > >> Please see the replies inline.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> > >
> > > > > > > >> > > Only one broker does the deletion. Broker updates the
> > > > expiration
> > > > > > in
> > > > > > > its
> > > > > > > >> > > local cache
> > > > > > > >> > > and on zookeeper so other brokers also get notified and
> > > their
> > > > > > cache
> > > > > > > >> > > statuses are updated as well.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > Which broker does the deletion?
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >> Any broker can handle the create/expire/renew/describe
> > > > > delegationtoken
> > > > > > > >> requests.
> > > > > > > >> changes are propagated through zk notifications. Every
> broker
> > > is
> > > > > > > >> responsible for
> > > > > > > >> expiring the tokens. This check be can done during request
> > > > handling
> > > > > > time
> > > > > > > >> and/or
> > > > > > > >> during token authentication time.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could
> > you
> > > > > > change
> > > > > > > it
> > > > > > > >> to
> > > > > > > >> > SCRAM?
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> Updated the diagram.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Manikumar
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > >
> > > > > > > >> > > Thanks.
> > > > > > > >> > > Manikumar
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > > > > > >> manikumar.reddy@gmail.com>
> > > > > > > >> > > > wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > Hi,
> > > > > > > >> > > > >
> > > > > > > >> > > > > I would like to initiate the vote on KIP-48:
> > > > > > > >> > > > >
> > > > > > > >> > > > > https://cwiki.apache.org/
> > confluence/display/KAFKA/KIP-
> > > 48+
> > > > > > > >> > > > > Delegation+token+support+for+Kafka
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > Thanks,
> > > > > > > >> > > > > Manikumar
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Gwen Shapira
> > > > > > > Product Manager | Confluent
> > > > > > > 650.450.2760 | @gwenshap
> > > > > > > Follow us: Twitter | blog
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Roger,
In the current proposal, we only allow a user to get delegation token for
that user only.
Anyone who gets that token can impersonate the user on the broker.
Yes, In future we can extend the support to allow a user to acquire
delegation tokens for
other users.
Pl refer discuss mail thread for impersonation related discussion.
Thanks,
Manikumar
On Wed, Feb 8, 2017 at 8:37 AM, Roger Hoover <ro...@gmail.com> wrote:
> Hi Jun,
>
> How does it allow impersonation at the connection level? Looking at the
> KIP, the DelegationTokenRequest does not have an "Owner" field that can be
> set. The owner field of the DelegationTokenResponse says it's the "Kakfa
> Principal which requested the delegation token". For impersonation, don't
> we need to be able to get tokens for other users besides the one making the
> request?
>
> Thanks,
>
> Roger
>
> On Tue, Feb 7, 2017 at 6:45 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Roger,
> >
> > Just to clarify. This KIP already allows you to do impersonation at the
> > connection level. Are you talking about impersonation at the request
> level?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover <ro...@gmail.com>
> > wrote:
> >
> > > Just wondering...how difficult would be it be to later add
> impersonation
> > (
> > > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would
> > be
> > > a
> > > Kafka admin UI that would take action on the cluster on behalf
> different
> > > users. I suppose we could later add an "effectiveUserId" (in Unix
> > > terminology) to the token details?
> > >
> > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> > >
> > > > +1 from me as well.
> > > >
> > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <ja...@confluent.io>
> > > > wrote:
> > > >
> > > > > Looks like a great proposal! I noticed that key rotation is not
> > > included.
> > > > > That may be reasonable for the initial work, but it might be nice
> to
> > > > share
> > > > > some thoughts on how that might work in the future. For example, I
> > > could
> > > > > imagine delegation.token.master.key could be a list, which would
> > allow
> > > > > users to support both a new and old key at the same time while
> > clients
> > > > are
> > > > > upgrading keys.
> > > > >
> > > > > -Jason
> > > > >
> > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > > >
> > > > > > Read the KIP again and I think it looks good.
> > > > > >
> > > > > > +1 from me.
> > > > > >
> > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > > > Hi, Mani,
> > > > > > >
> > > > > > > If a token expires, then every broker will potentially try to
> > > delete
> > > > it
> > > > > > > around the same time, but only one will succeed. So, we will
> have
> > > to
> > > > > deal
> > > > > > > with failures in that case? Another way is to let just one
> broker
> > > > (say,
> > > > > > the
> > > > > > > controller) deletes expired tokens.
> > > > > > >
> > > > > > > It would also be helpful for others to give feedback on this
> KIP.
> > > > > Rajini,
> > > > > > > Gwen, Ismael?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <
> > > manikumar.reddy@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Jun,
> > > > > > >>
> > > > > > >> Please see the replies inline.
> > > > > > >>
> > > > > > >>
> > > > > > >> > >
> > > > > > >> > > Only one broker does the deletion. Broker updates the
> > > expiration
> > > > > in
> > > > > > its
> > > > > > >> > > local cache
> > > > > > >> > > and on zookeeper so other brokers also get notified and
> > their
> > > > > cache
> > > > > > >> > > statuses are updated as well.
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > Which broker does the deletion?
> > > > > > >> >
> > > > > > >>
> > > > > > >> Any broker can handle the create/expire/renew/describe
> > > > delegationtoken
> > > > > > >> requests.
> > > > > > >> changes are propagated through zk notifications. Every broker
> > is
> > > > > > >> responsible for
> > > > > > >> expiring the tokens. This check be can done during request
> > > handling
> > > > > time
> > > > > > >> and/or
> > > > > > >> during token authentication time.
> > > > > > >>
> > > > > > >>
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could
> you
> > > > > change
> > > > > > it
> > > > > > >> to
> > > > > > >> > SCRAM?
> > > > > > >> >
> > > > > > >> >
> > > > > > >> Updated the diagram.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Manikumar
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > > Thanks.
> > > > > > >> > > Manikumar
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > >
> > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > > > > >> manikumar.reddy@gmail.com>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Hi,
> > > > > > >> > > > >
> > > > > > >> > > > > I would like to initiate the vote on KIP-48:
> > > > > > >> > > > >
> > > > > > >> > > > > https://cwiki.apache.org/
> confluence/display/KAFKA/KIP-
> > 48+
> > > > > > >> > > > > Delegation+token+support+for+Kafka
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks,
> > > > > > >> > > > > Manikumar
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Gwen Shapira
> > > > > > Product Manager | Confluent
> > > > > > 650.450.2760 | @gwenshap
> > > > > > Follow us: Twitter | blog
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Roger Hoover <ro...@gmail.com>.
Hi Jun,
How does it allow impersonation at the connection level? Looking at the
KIP, the DelegationTokenRequest does not have an "Owner" field that can be
set. The owner field of the DelegationTokenResponse says it's the "Kakfa
Principal which requested the delegation token". For impersonation, don't
we need to be able to get tokens for other users besides the one making the
request?
Thanks,
Roger
On Tue, Feb 7, 2017 at 6:45 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Roger,
>
> Just to clarify. This KIP already allows you to do impersonation at the
> connection level. Are you talking about impersonation at the request level?
>
> Thanks,
>
> Jun
>
> On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover <ro...@gmail.com>
> wrote:
>
> > Just wondering...how difficult would be it be to later add impersonation
> (
> > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would
> be
> > a
> > Kafka admin UI that would take action on the cluster on behalf different
> > users. I suppose we could later add an "effectiveUserId" (in Unix
> > terminology) to the token details?
> >
> > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke <gh...@cloudera.com> wrote:
> >
> > > +1 from me as well.
> > >
> > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <ja...@confluent.io>
> > > wrote:
> > >
> > > > Looks like a great proposal! I noticed that key rotation is not
> > included.
> > > > That may be reasonable for the initial work, but it might be nice to
> > > share
> > > > some thoughts on how that might work in the future. For example, I
> > could
> > > > imagine delegation.token.master.key could be a list, which would
> allow
> > > > users to support both a new and old key at the same time while
> clients
> > > are
> > > > upgrading keys.
> > > >
> > > > -Jason
> > > >
> > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > > >
> > > > > Read the KIP again and I think it looks good.
> > > > >
> > > > > +1 from me.
> > > > >
> > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > > > Hi, Mani,
> > > > > >
> > > > > > If a token expires, then every broker will potentially try to
> > delete
> > > it
> > > > > > around the same time, but only one will succeed. So, we will have
> > to
> > > > deal
> > > > > > with failures in that case? Another way is to let just one broker
> > > (say,
> > > > > the
> > > > > > controller) deletes expired tokens.
> > > > > >
> > > > > > It would also be helpful for others to give feedback on this KIP.
> > > > Rajini,
> > > > > > Gwen, Ismael?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <
> > manikumar.reddy@gmail.com
> > > >
> > > > > wrote:
> > > > > >
> > > > > >> Hi Jun,
> > > > > >>
> > > > > >> Please see the replies inline.
> > > > > >>
> > > > > >>
> > > > > >> > >
> > > > > >> > > Only one broker does the deletion. Broker updates the
> > expiration
> > > > in
> > > > > its
> > > > > >> > > local cache
> > > > > >> > > and on zookeeper so other brokers also get notified and
> their
> > > > cache
> > > > > >> > > statuses are updated as well.
> > > > > >> > >
> > > > > >> > >
> > > > > >> > Which broker does the deletion?
> > > > > >> >
> > > > > >>
> > > > > >> Any broker can handle the create/expire/renew/describe
> > > delegationtoken
> > > > > >> requests.
> > > > > >> changes are propagated through zk notifications. Every broker
> is
> > > > > >> responsible for
> > > > > >> expiring the tokens. This check be can done during request
> > handling
> > > > time
> > > > > >> and/or
> > > > > >> during token authentication time.
> > > > > >>
> > > > > >>
> > > > > >> >
> > > > > >> >
> > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you
> > > > change
> > > > > it
> > > > > >> to
> > > > > >> > SCRAM?
> > > > > >> >
> > > > > >> >
> > > > > >> Updated the diagram.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Manikumar
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> >
> > > > > >> >
> > > > > >> > >
> > > > > >> > > Thanks.
> > > > > >> > > Manikumar
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > > > >> manikumar.reddy@gmail.com>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Hi,
> > > > > >> > > > >
> > > > > >> > > > > I would like to initiate the vote on KIP-48:
> > > > > >> > > > >
> > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 48+
> > > > > >> > > > > Delegation+token+support+for+Kafka
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > Thanks,
> > > > > >> > > > > Manikumar
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Gwen Shapira
> > > > > Product Manager | Confluent
> > > > > 650.450.2760 | @gwenshap
> > > > > Follow us: Twitter | blog
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Roger,
Just to clarify. This KIP already allows you to do impersonation at the
connection level. Are you talking about impersonation at the request level?
Thanks,
Jun
On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover <ro...@gmail.com> wrote:
> Just wondering...how difficult would be it be to later add impersonation (
> https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be
> a
> Kafka admin UI that would take action on the cluster on behalf different
> users. I suppose we could later add an "effectiveUserId" (in Unix
> terminology) to the token details?
>
> On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > +1 from me as well.
> >
> > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > Looks like a great proposal! I noticed that key rotation is not
> included.
> > > That may be reasonable for the initial work, but it might be nice to
> > share
> > > some thoughts on how that might work in the future. For example, I
> could
> > > imagine delegation.token.master.key could be a list, which would allow
> > > users to support both a new and old key at the same time while clients
> > are
> > > upgrading keys.
> > >
> > > -Jason
> > >
> > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > Read the KIP again and I think it looks good.
> > > >
> > > > +1 from me.
> > > >
> > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > > Hi, Mani,
> > > > >
> > > > > If a token expires, then every broker will potentially try to
> delete
> > it
> > > > > around the same time, but only one will succeed. So, we will have
> to
> > > deal
> > > > > with failures in that case? Another way is to let just one broker
> > (say,
> > > > the
> > > > > controller) deletes expired tokens.
> > > > >
> > > > > It would also be helpful for others to give feedback on this KIP.
> > > Rajini,
> > > > > Gwen, Ismael?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <
> manikumar.reddy@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > >> Hi Jun,
> > > > >>
> > > > >> Please see the replies inline.
> > > > >>
> > > > >>
> > > > >> > >
> > > > >> > > Only one broker does the deletion. Broker updates the
> expiration
> > > in
> > > > its
> > > > >> > > local cache
> > > > >> > > and on zookeeper so other brokers also get notified and their
> > > cache
> > > > >> > > statuses are updated as well.
> > > > >> > >
> > > > >> > >
> > > > >> > Which broker does the deletion?
> > > > >> >
> > > > >>
> > > > >> Any broker can handle the create/expire/renew/describe
> > delegationtoken
> > > > >> requests.
> > > > >> changes are propagated through zk notifications. Every broker is
> > > > >> responsible for
> > > > >> expiring the tokens. This check be can done during request
> handling
> > > time
> > > > >> and/or
> > > > >> during token authentication time.
> > > > >>
> > > > >>
> > > > >> >
> > > > >> >
> > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you
> > > change
> > > > it
> > > > >> to
> > > > >> > SCRAM?
> > > > >> >
> > > > >> >
> > > > >> Updated the diagram.
> > > > >>
> > > > >>
> > > > >>
> > > > >> Thanks,
> > > > >> Manikumar
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> >
> > > > >> >
> > > > >> > >
> > > > >> > > Thanks.
> > > > >> > > Manikumar
> > > > >> > >
> > > > >> > >
> > > > >> > > >
> > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > > >> manikumar.reddy@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Hi,
> > > > >> > > > >
> > > > >> > > > > I would like to initiate the vote on KIP-48:
> > > > >> > > > >
> > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > >> > > > > Delegation+token+support+for+Kafka
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > Manikumar
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Gwen Shapira
> > > > Product Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter | blog
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Roger Hoover <ro...@gmail.com>.
Just wondering...how difficult would be it be to later add impersonation (
https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be a
Kafka admin UI that would take action on the cluster on behalf different
users. I suppose we could later add an "effectiveUserId" (in Unix
terminology) to the token details?
On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke <gh...@cloudera.com> wrote:
> +1 from me as well.
>
> On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Looks like a great proposal! I noticed that key rotation is not included.
> > That may be reasonable for the initial work, but it might be nice to
> share
> > some thoughts on how that might work in the future. For example, I could
> > imagine delegation.token.master.key could be a list, which would allow
> > users to support both a new and old key at the same time while clients
> are
> > upgrading keys.
> >
> > -Jason
> >
> > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Read the KIP again and I think it looks good.
> > >
> > > +1 from me.
> > >
> > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > Hi, Mani,
> > > >
> > > > If a token expires, then every broker will potentially try to delete
> it
> > > > around the same time, but only one will succeed. So, we will have to
> > deal
> > > > with failures in that case? Another way is to let just one broker
> (say,
> > > the
> > > > controller) deletes expired tokens.
> > > >
> > > > It would also be helpful for others to give feedback on this KIP.
> > Rajini,
> > > > Gwen, Ismael?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <manikumar.reddy@gmail.com
> >
> > > wrote:
> > > >
> > > >> Hi Jun,
> > > >>
> > > >> Please see the replies inline.
> > > >>
> > > >>
> > > >> > >
> > > >> > > Only one broker does the deletion. Broker updates the expiration
> > in
> > > its
> > > >> > > local cache
> > > >> > > and on zookeeper so other brokers also get notified and their
> > cache
> > > >> > > statuses are updated as well.
> > > >> > >
> > > >> > >
> > > >> > Which broker does the deletion?
> > > >> >
> > > >>
> > > >> Any broker can handle the create/expire/renew/describe
> delegationtoken
> > > >> requests.
> > > >> changes are propagated through zk notifications. Every broker is
> > > >> responsible for
> > > >> expiring the tokens. This check be can done during request handling
> > time
> > > >> and/or
> > > >> during token authentication time.
> > > >>
> > > >>
> > > >> >
> > > >> >
> > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you
> > change
> > > it
> > > >> to
> > > >> > SCRAM?
> > > >> >
> > > >> >
> > > >> Updated the diagram.
> > > >>
> > > >>
> > > >>
> > > >> Thanks,
> > > >> Manikumar
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> >
> > > >> >
> > > >> > >
> > > >> > > Thanks.
> > > >> > > Manikumar
> > > >> > >
> > > >> > >
> > > >> > > >
> > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > >> manikumar.reddy@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi,
> > > >> > > > >
> > > >> > > > > I would like to initiate the vote on KIP-48:
> > > >> > > > >
> > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > >> > > > > Delegation+token+support+for+Kafka
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > Manikumar
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> > >
> > >
> > > --
> > > Gwen Shapira
> > > Product Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter | blog
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Grant Henke <gh...@cloudera.com>.
+1 from me as well.
On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson <ja...@confluent.io> wrote:
> Looks like a great proposal! I noticed that key rotation is not included.
> That may be reasonable for the initial work, but it might be nice to share
> some thoughts on how that might work in the future. For example, I could
> imagine delegation.token.master.key could be a list, which would allow
> users to support both a new and old key at the same time while clients are
> upgrading keys.
>
> -Jason
>
> On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Read the KIP again and I think it looks good.
> >
> > +1 from me.
> >
> > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > > Hi, Mani,
> > >
> > > If a token expires, then every broker will potentially try to delete it
> > > around the same time, but only one will succeed. So, we will have to
> deal
> > > with failures in that case? Another way is to let just one broker (say,
> > the
> > > controller) deletes expired tokens.
> > >
> > > It would also be helpful for others to give feedback on this KIP.
> Rajini,
> > > Gwen, Ismael?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> > wrote:
> > >
> > >> Hi Jun,
> > >>
> > >> Please see the replies inline.
> > >>
> > >>
> > >> > >
> > >> > > Only one broker does the deletion. Broker updates the expiration
> in
> > its
> > >> > > local cache
> > >> > > and on zookeeper so other brokers also get notified and their
> cache
> > >> > > statuses are updated as well.
> > >> > >
> > >> > >
> > >> > Which broker does the deletion?
> > >> >
> > >>
> > >> Any broker can handle the create/expire/renew/describe delegationtoken
> > >> requests.
> > >> changes are propagated through zk notifications. Every broker is
> > >> responsible for
> > >> expiring the tokens. This check be can done during request handling
> time
> > >> and/or
> > >> during token authentication time.
> > >>
> > >>
> > >> >
> > >> >
> > >> > 110. The diagrams in the wiki still show MD5 digest. Could you
> change
> > it
> > >> to
> > >> > SCRAM?
> > >> >
> > >> >
> > >> Updated the diagram.
> > >>
> > >>
> > >>
> > >> Thanks,
> > >> Manikumar
> > >>
> > >>
> > >>
> > >>
> > >> >
> > >> >
> > >> > >
> > >> > > Thanks.
> > >> > > Manikumar
> > >> > >
> > >> > >
> > >> > > >
> > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > >> manikumar.reddy@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi,
> > >> > > > >
> > >> > > > > I would like to initiate the vote on KIP-48:
> > >> > > > >
> > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > >> > > > > Delegation+token+support+for+Kafka
> > >> > > > >
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Manikumar
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jason Gustafson <ja...@confluent.io>.
Looks like a great proposal! I noticed that key rotation is not included.
That may be reasonable for the initial work, but it might be nice to share
some thoughts on how that might work in the future. For example, I could
imagine delegation.token.master.key could be a list, which would allow
users to support both a new and old key at the same time while clients are
upgrading keys.
-Jason
On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Read the KIP again and I think it looks good.
>
> +1 from me.
>
> On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> > Hi, Mani,
> >
> > If a token expires, then every broker will potentially try to delete it
> > around the same time, but only one will succeed. So, we will have to deal
> > with failures in that case? Another way is to let just one broker (say,
> the
> > controller) deletes expired tokens.
> >
> > It would also be helpful for others to give feedback on this KIP. Rajini,
> > Gwen, Ismael?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> wrote:
> >
> >> Hi Jun,
> >>
> >> Please see the replies inline.
> >>
> >>
> >> > >
> >> > > Only one broker does the deletion. Broker updates the expiration in
> its
> >> > > local cache
> >> > > and on zookeeper so other brokers also get notified and their cache
> >> > > statuses are updated as well.
> >> > >
> >> > >
> >> > Which broker does the deletion?
> >> >
> >>
> >> Any broker can handle the create/expire/renew/describe delegationtoken
> >> requests.
> >> changes are propagated through zk notifications. Every broker is
> >> responsible for
> >> expiring the tokens. This check be can done during request handling time
> >> and/or
> >> during token authentication time.
> >>
> >>
> >> >
> >> >
> >> > 110. The diagrams in the wiki still show MD5 digest. Could you change
> it
> >> to
> >> > SCRAM?
> >> >
> >> >
> >> Updated the diagram.
> >>
> >>
> >>
> >> Thanks,
> >> Manikumar
> >>
> >>
> >>
> >>
> >> >
> >> >
> >> > >
> >> > > Thanks.
> >> > > Manikumar
> >> > >
> >> > >
> >> > > >
> >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> >> manikumar.reddy@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > I would like to initiate the vote on KIP-48:
> >> > > > >
> >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> >> > > > > Delegation+token+support+for+Kafka
> >> > > > >
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Manikumar
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Gwen Shapira <gw...@confluent.io>.
Read the KIP again and I think it looks good.
+1 from me.
On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Mani,
>
> If a token expires, then every broker will potentially try to delete it
> around the same time, but only one will succeed. So, we will have to deal
> with failures in that case? Another way is to let just one broker (say, the
> controller) deletes expired tokens.
>
> It would also be helpful for others to give feedback on this KIP. Rajini,
> Gwen, Ismael?
>
> Thanks,
>
> Jun
>
>
> On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com> wrote:
>
>> Hi Jun,
>>
>> Please see the replies inline.
>>
>>
>> > >
>> > > Only one broker does the deletion. Broker updates the expiration in its
>> > > local cache
>> > > and on zookeeper so other brokers also get notified and their cache
>> > > statuses are updated as well.
>> > >
>> > >
>> > Which broker does the deletion?
>> >
>>
>> Any broker can handle the create/expire/renew/describe delegationtoken
>> requests.
>> changes are propagated through zk notifications. Every broker is
>> responsible for
>> expiring the tokens. This check be can done during request handling time
>> and/or
>> during token authentication time.
>>
>>
>> >
>> >
>> > 110. The diagrams in the wiki still show MD5 digest. Could you change it
>> to
>> > SCRAM?
>> >
>> >
>> Updated the diagram.
>>
>>
>>
>> Thanks,
>> Manikumar
>>
>>
>>
>>
>> >
>> >
>> > >
>> > > Thanks.
>> > > Manikumar
>> > >
>> > >
>> > > >
>> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
>> manikumar.reddy@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I would like to initiate the vote on KIP-48:
>> > > > >
>> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
>> > > > > Delegation+token+support+for+Kafka
>> > > > >
>> > > > >
>> > > > > Thanks,
>> > > > > Manikumar
>> > > > >
>> > > >
>> > >
>> >
>>
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Thank you all for your votes and feedback.
The vote has passed with 4 binding votes(Gwen, Jun, Grant, Harsha) and 2
non-binding votes(Roger, Dong Lin).
I have updated the relevant wiki pages.
Thanks
Manikumar
On Tue, Feb 14, 2017 at 12:02 AM, Dong Lin <li...@gmail.com> wrote:
> +1 (non-binding)
>
> On Mon, Feb 13, 2017 at 10:21 AM, Harsha Chintalapani <ma...@harsha.io>
> wrote:
>
> > +1.
> > -Harsha
> >
> > On Fri, Feb 10, 2017 at 11:12 PM Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Yes, owners and the renewers can always describe their own tokens.
> > Updated
> > > the KIP.
> > >
> > > On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Mani,
> > > >
> > > > Thanks for the update. Just a minor comment below. Otherwise, +1 from
> > me.
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > 116. Could you document the ACL rules associated with those new
> > > > requests?
> > > > > > For example, do we allow any one to create, delete, describe
> > > delegation
> > > > > > tokens?
> > > > > >
> > > > > >
> > > > > Currently we only allow a owner to create delegation token for that
> > > owner
> > > > > only.
> > > > > Any thing the owner has permission to do, delegation tokens should
> be
> > > > > allowed to do as well. We can also check renew and expire requests
> > are
> > > > > coming
> > > > > from owner or renewers of the token. So we may not need ACLs for
> > > > > create/renew/expire requests.
> > > > >
> > > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In
> > > future,
> > > > > when we extend
> > > > > the support to allow a user to acquire delegation tokens for other
> > > users,
> > > > > then we can enable
> > > > > CREATE/DELETE operations. Updated the KIP.
> > > > >
> > > > >
> > > > This sounds good. I guess the owner and the renewer can always
> describe
> > > > their own tokens?
> > > >
> > > > Jun
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Dong Lin <li...@gmail.com>.
+1 (non-binding)
On Mon, Feb 13, 2017 at 10:21 AM, Harsha Chintalapani <ma...@harsha.io>
wrote:
> +1.
> -Harsha
>
> On Fri, Feb 10, 2017 at 11:12 PM Manikumar <ma...@gmail.com>
> wrote:
>
> > Yes, owners and the renewers can always describe their own tokens.
> Updated
> > the KIP.
> >
> > On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Mani,
> > >
> > > Thanks for the update. Just a minor comment below. Otherwise, +1 from
> me.
> > >
> > >
> > > >
> > > > >
> > > > > 116. Could you document the ACL rules associated with those new
> > > requests?
> > > > > For example, do we allow any one to create, delete, describe
> > delegation
> > > > > tokens?
> > > > >
> > > > >
> > > > Currently we only allow a owner to create delegation token for that
> > owner
> > > > only.
> > > > Any thing the owner has permission to do, delegation tokens should be
> > > > allowed to do as well. We can also check renew and expire requests
> are
> > > > coming
> > > > from owner or renewers of the token. So we may not need ACLs for
> > > > create/renew/expire requests.
> > > >
> > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In
> > future,
> > > > when we extend
> > > > the support to allow a user to acquire delegation tokens for other
> > users,
> > > > then we can enable
> > > > CREATE/DELETE operations. Updated the KIP.
> > > >
> > > >
> > > This sounds good. I guess the owner and the renewer can always describe
> > > their own tokens?
> > >
> > > Jun
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Harsha Chintalapani <ma...@harsha.io>.
+1.
-Harsha
On Fri, Feb 10, 2017 at 11:12 PM Manikumar <ma...@gmail.com>
wrote:
> Yes, owners and the renewers can always describe their own tokens. Updated
> the KIP.
>
> On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Mani,
> >
> > Thanks for the update. Just a minor comment below. Otherwise, +1 from me.
> >
> >
> > >
> > > >
> > > > 116. Could you document the ACL rules associated with those new
> > requests?
> > > > For example, do we allow any one to create, delete, describe
> delegation
> > > > tokens?
> > > >
> > > >
> > > Currently we only allow a owner to create delegation token for that
> owner
> > > only.
> > > Any thing the owner has permission to do, delegation tokens should be
> > > allowed to do as well. We can also check renew and expire requests are
> > > coming
> > > from owner or renewers of the token. So we may not need ACLs for
> > > create/renew/expire requests.
> > >
> > > For describe, we can add DESCRIBE operation on TOKEN Resource. In
> future,
> > > when we extend
> > > the support to allow a user to acquire delegation tokens for other
> users,
> > > then we can enable
> > > CREATE/DELETE operations. Updated the KIP.
> > >
> > >
> > This sounds good. I guess the owner and the renewer can always describe
> > their own tokens?
> >
> > Jun
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Yes, owners and the renewers can always describe their own tokens. Updated
the KIP.
On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Mani,
>
> Thanks for the update. Just a minor comment below. Otherwise, +1 from me.
>
>
> >
> > >
> > > 116. Could you document the ACL rules associated with those new
> requests?
> > > For example, do we allow any one to create, delete, describe delegation
> > > tokens?
> > >
> > >
> > Currently we only allow a owner to create delegation token for that owner
> > only.
> > Any thing the owner has permission to do, delegation tokens should be
> > allowed to do as well. We can also check renew and expire requests are
> > coming
> > from owner or renewers of the token. So we may not need ACLs for
> > create/renew/expire requests.
> >
> > For describe, we can add DESCRIBE operation on TOKEN Resource. In future,
> > when we extend
> > the support to allow a user to acquire delegation tokens for other users,
> > then we can enable
> > CREATE/DELETE operations. Updated the KIP.
> >
> >
> This sounds good. I guess the owner and the renewer can always describe
> their own tokens?
>
> Jun
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Mani,
Thanks for the update. Just a minor comment below. Otherwise, +1 from me.
>
> >
> > 116. Could you document the ACL rules associated with those new requests?
> > For example, do we allow any one to create, delete, describe delegation
> > tokens?
> >
> >
> Currently we only allow a owner to create delegation token for that owner
> only.
> Any thing the owner has permission to do, delegation tokens should be
> allowed to do as well. We can also check renew and expire requests are
> coming
> from owner or renewers of the token. So we may not need ACLs for
> create/renew/expire requests.
>
> For describe, we can add DESCRIBE operation on TOKEN Resource. In future,
> when we extend
> the support to allow a user to acquire delegation tokens for other users,
> then we can enable
> CREATE/DELETE operations. Updated the KIP.
>
>
This sounds good. I guess the owner and the renewer can always describe
their own tokens?
Jun
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Jun,
Thanks for the detailed review. Pl see the replies inline
> 101.2/101.3. Could we just remove owner and renewer from
> DelegationTokenResponse if we don't have a use case?
>
>
Removed owner/renewer fileds from DelegationTokenResponse. Updated the KIP.
> 111. ExpireTokenResponse: Should we return the new expiration time in the
> response?
>
>
Yes. expiration time is also included in RenewToken response .
> 112. DescribeTokenRequest: A common use case is probably to see a list of
> tokens associated with a particular owner. Would it be useful to include a
> list of owners in the request? We can use the same convention in other
> requests such that if the list is set to null (i.e., length is -1), return
> all tokens.
>
>
Agree. Updated the KIP.
> 113. delegation.token.master.key: It seems that needs to be the same across
> brokers? Perhaps we can mention that in the wiki.
>
Updated the KIP.
>
> 114. Could we document the procedure of manually rotating the secret? Does
> one have to do sth like: expire all existing tokens, rotate the secret, and
> generate new tokens?
>
>
Thanks for the suggestion. Updated the suggested steps. During rotation
process, already
connected clients may continue to work. But any new connection requests and
renew/expire
requests with old tokens can fail.
115. Could we also include in the command line the ability to describe
> tokens?
>
Updated the KIP.
>
> 116. Could you document the ACL rules associated with those new requests?
> For example, do we allow any one to create, delete, describe delegation
> tokens?
>
>
Currently we only allow a owner to create delegation token for that owner
only.
Any thing the owner has permission to do, delegation tokens should be
allowed to do as well. We can also check renew and expire requests are
coming
from owner or renewers of the token. So we may not need ACLs for
create/renew/expire requests.
For describe, we can add DESCRIBE operation on TOKEN Resource. In future,
when we extend
the support to allow a user to acquire delegation tokens for other users,
then we can enable
CREATE/DELETE operations. Updated the KIP.
Thanks,
Manikumar
> On Wed, Feb 8, 2017 at 1:35 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi Jun,
> >
> >
> > > If a token expires, then every broker will potentially try to delete it
> > > around the same time, but only one will succeed. So, we will have to
> deal
> > > with failures in that case? Another way is to let just one broker (say,
> > the
> > > controller) deletes expired tokens.
> > >
> > >
> > Agree, we can run the token expiry check thread as part of controller
> > broker.
> > WIll update the KIP.
> >
> >
> > Thanks,
> > Manikumar
> >
> >
> > >
> > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hi Jun,
> > > >
> > > > Please see the replies inline.
> > > >
> > > >
> > > > > >
> > > > > > Only one broker does the deletion. Broker updates the expiration
> in
> > > its
> > > > > > local cache
> > > > > > and on zookeeper so other brokers also get notified and their
> cache
> > > > > > statuses are updated as well.
> > > > > >
> > > > > >
> > > > > Which broker does the deletion?
> > > > >
> > > >
> > > > Any broker can handle the create/expire/renew/describe
> delegationtoken
> > > > requests.
> > > > changes are propagated through zk notifications. Every broker is
> > > > responsible for
> > > > expiring the tokens. This check be can done during request handling
> > time
> > > > and/or
> > > > during token authentication time.
> > > >
> > > >
> > > > >
> > > > >
> > > > > 110. The diagrams in the wiki still show MD5 digest. Could you
> change
> > > it
> > > > to
> > > > > SCRAM?
> > > > >
> > > > >
> > > > Updated the diagram.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > >
> > > > > > Thanks.
> > > > > > Manikumar
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > > manikumar.reddy@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I would like to initiate the vote on KIP-48:
> > > > > > > >
> > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > > > > > Delegation+token+support+for+Kafka
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Manikumar
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Mani,
Thanks for the responses. A few more comments.
101.2/101.3. Could we just remove owner and renewer from
DelegationTokenResponse if we don't have a use case?
111. ExpireTokenResponse: Should we return the new expiration time in the
response?
112. DescribeTokenRequest: A common use case is probably to see a list of
tokens associated with a particular owner. Would it be useful to include a
list of owners in the request? We can use the same convention in other
requests such that if the list is set to null (i.e., length is -1), return
all tokens.
113. delegation.token.master.key: It seems that needs to be the same across
brokers? Perhaps we can mention that in the wiki.
114. Could we document the procedure of manually rotating the secret? Does
one have to do sth like: expire all existing tokens, rotate the secret, and
generate new tokens?
115. Could we also include in the command line the ability to describe
tokens?
116. Could you document the ACL rules associated with those new requests?
For example, do we allow any one to create, delete, describe delegation
tokens?
Thanks,
Jun
On Wed, Feb 8, 2017 at 1:35 AM, Manikumar <ma...@gmail.com> wrote:
> Hi Jun,
>
>
> > If a token expires, then every broker will potentially try to delete it
> > around the same time, but only one will succeed. So, we will have to deal
> > with failures in that case? Another way is to let just one broker (say,
> the
> > controller) deletes expired tokens.
> >
> >
> Agree, we can run the token expiry check thread as part of controller
> broker.
> WIll update the KIP.
>
>
> Thanks,
> Manikumar
>
>
> >
> > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hi Jun,
> > >
> > > Please see the replies inline.
> > >
> > >
> > > > >
> > > > > Only one broker does the deletion. Broker updates the expiration in
> > its
> > > > > local cache
> > > > > and on zookeeper so other brokers also get notified and their cache
> > > > > statuses are updated as well.
> > > > >
> > > > >
> > > > Which broker does the deletion?
> > > >
> > >
> > > Any broker can handle the create/expire/renew/describe delegationtoken
> > > requests.
> > > changes are propagated through zk notifications. Every broker is
> > > responsible for
> > > expiring the tokens. This check be can done during request handling
> time
> > > and/or
> > > during token authentication time.
> > >
> > >
> > > >
> > > >
> > > > 110. The diagrams in the wiki still show MD5 digest. Could you change
> > it
> > > to
> > > > SCRAM?
> > > >
> > > >
> > > Updated the diagram.
> > >
> > >
> > >
> > > Thanks,
> > > Manikumar
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > >
> > > > > Thanks.
> > > > > Manikumar
> > > > >
> > > > >
> > > > > >
> > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > > manikumar.reddy@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I would like to initiate the vote on KIP-48:
> > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > > > > Delegation+token+support+for+Kafka
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Manikumar
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Jun,
> If a token expires, then every broker will potentially try to delete it
> around the same time, but only one will succeed. So, we will have to deal
> with failures in that case? Another way is to let just one broker (say, the
> controller) deletes expired tokens.
>
>
Agree, we can run the token expiry check thread as part of controller
broker.
WIll update the KIP.
Thanks,
Manikumar
>
> On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi Jun,
> >
> > Please see the replies inline.
> >
> >
> > > >
> > > > Only one broker does the deletion. Broker updates the expiration in
> its
> > > > local cache
> > > > and on zookeeper so other brokers also get notified and their cache
> > > > statuses are updated as well.
> > > >
> > > >
> > > Which broker does the deletion?
> > >
> >
> > Any broker can handle the create/expire/renew/describe delegationtoken
> > requests.
> > changes are propagated through zk notifications. Every broker is
> > responsible for
> > expiring the tokens. This check be can done during request handling time
> > and/or
> > during token authentication time.
> >
> >
> > >
> > >
> > > 110. The diagrams in the wiki still show MD5 digest. Could you change
> it
> > to
> > > SCRAM?
> > >
> > >
> > Updated the diagram.
> >
> >
> >
> > Thanks,
> > Manikumar
> >
> >
> >
> >
> > >
> > >
> > > >
> > > > Thanks.
> > > > Manikumar
> > > >
> > > >
> > > > >
> > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> > manikumar.reddy@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I would like to initiate the vote on KIP-48:
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > > > Delegation+token+support+for+Kafka
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Manikumar
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Mani,
If a token expires, then every broker will potentially try to delete it
around the same time, but only one will succeed. So, we will have to deal
with failures in that case? Another way is to let just one broker (say, the
controller) deletes expired tokens.
It would also be helpful for others to give feedback on this KIP. Rajini,
Gwen, Ismael?
Thanks,
Jun
On Sun, Feb 5, 2017 at 9:54 AM, Manikumar <ma...@gmail.com> wrote:
> Hi Jun,
>
> Please see the replies inline.
>
>
> > >
> > > Only one broker does the deletion. Broker updates the expiration in its
> > > local cache
> > > and on zookeeper so other brokers also get notified and their cache
> > > statuses are updated as well.
> > >
> > >
> > Which broker does the deletion?
> >
>
> Any broker can handle the create/expire/renew/describe delegationtoken
> requests.
> changes are propagated through zk notifications. Every broker is
> responsible for
> expiring the tokens. This check be can done during request handling time
> and/or
> during token authentication time.
>
>
> >
> >
> > 110. The diagrams in the wiki still show MD5 digest. Could you change it
> to
> > SCRAM?
> >
> >
> Updated the diagram.
>
>
>
> Thanks,
> Manikumar
>
>
>
>
> >
> >
> > >
> > > Thanks.
> > > Manikumar
> > >
> > >
> > > >
> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <
> manikumar.reddy@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I would like to initiate the vote on KIP-48:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > > Delegation+token+support+for+Kafka
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Jun,
Please see the replies inline.
> >
> > Only one broker does the deletion. Broker updates the expiration in its
> > local cache
> > and on zookeeper so other brokers also get notified and their cache
> > statuses are updated as well.
> >
> >
> Which broker does the deletion?
>
Any broker can handle the create/expire/renew/describe delegationtoken
requests.
changes are propagated through zk notifications. Every broker is
responsible for
expiring the tokens. This check be can done during request handling time
and/or
during token authentication time.
>
>
> 110. The diagrams in the wiki still show MD5 digest. Could you change it to
> SCRAM?
>
>
Updated the diagram.
Thanks,
Manikumar
>
>
> >
> > Thanks.
> > Manikumar
> >
> >
> > >
> > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I would like to initiate the vote on KIP-48:
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > > Delegation+token+support+for+Kafka
> > > >
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Mani,
Sorry for the late response. A couple of more comments below.
> 107.4 How is token deletion handled? Does every broker delete the same
> > token independently or only one broker does the deletion?
> >
>
> Only one broker does the deletion. Broker updates the expiration in its
> local cache
> and on zookeeper so other brokers also get notified and their cache
> statuses are updated as well.
>
>
Which broker does the deletion?
110. The diagrams in the wiki still show MD5 digest. Could you change it to
SCRAM?
Thanks,
Jun
>
> Thanks.
> Manikumar
>
>
> >
> > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to initiate the vote on KIP-48:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > > Delegation+token+support+for+Kafka
> > >
> > >
> > > Thanks,
> > > Manikumar
> > >
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Jun,
Thanks for the review. Sorry for the delayed response. Please see the
replies inline.
> 101. DelegationTokenRequest/DelegationTokenResponse
> 101.1 I am wondering if renewer should be principalType + name to match
> what we have in kafka-acls tool?
>
Yes, renewer should be principalType + name. Will Update the KIP.
> 101.2 Do we need to return owner in DelegationTokenResponse?
> 101.3 Do we need to return renewer in DelegationTokenResponse?
>
>
The idea is to return complete token details. If there is no use, then we
can omit them.
@Ashish Since you are implementing this part of the KIP, Pl reply if
you have any concerns.
> 102. RenewDelegationTokenRequest/RenewDelegationTokenResponse
> 102.1 In RenewDelegationTokenRequest, is the renewal time bounded
> by delegation.token.expiry.time.sec ?
>
renewal time is bounded by delegation.token.max.lifetime.ms.
delegation.token.expiry.time.sec is used as default renew period. It will
be used to compute
expiryDate for first time and used if renew period is not provided in
RenewDelegationTokenRequest
token expiryDate = min(currentTime + renewPeriod, maxDate)
102.2 Do we need to include TokenDetails in RenewDelegationTokenResponse?
>
>
After renewal we are only updating token expiry time. So we may not need to
return
complete TokenDetails. Will Update the KIP.
> 103. Do we need a DescribeDelegationTokenRequest and a corresponding tool
> to list/describe existing delegation tokens?
>
>
Yes, we can add token describe request and tool. will update the KIP
> 104. KafkaConfig: Do we need a new config to disable/enable delegation
> tokens?
>
>
Currently we need masterKey (delegation.token.master.key config property)
to generate
delegation tokens. This masterKey needs to be configured with all the
brokers. If this config
property is not set, then brokers will disable the delegation token
support.
105. Could we be more consistent with the naming in configs, the request
> protocol fields and ZK structure with respect to time
> (e.g. delegation.token.max.lifetime.sec => delegation.token.max.lifetime.
> ms
> )?
>
>
All the time related configs/fileds changed to ms.
> 106. Using SASL SCRAM for delegation token
> 106.1 What should be set for username? Does it matter?
>
We will provide "tokenID(sequence number)" as username. This will uniquely
identifies the token.
> 106.2 Could you provide a bit more details on how to distinguish it from
> regular SASL SCRAM?
>
>
High level approach is given below:
On Client Side:
1. add a config property "tokenauth=true" to jass config. ScramLoginModule
loads this property if exists.
2. add this property to optional extension attributes of Scram
ClientFirstMessage.
On Server Side:
1. Update CredentialProvider class to handle tokenCredentialCache & add
TokenCredentialCallback
2. read the ClientFirstMessage
if tokenauth=true exists then
do token validations
get credentials from tokenCredentialCache
else
get credentials from regular CredentialCache
3. Continue Scram handshake
> 107. ZK structure
> 107.1 Could you provide more details on what SCRAM credentials are stored
> in ZK path /tokenauth/tokens/<tokenUID>?
>
We will store similar to regular scram credentials. These will get
generated during token creation.
Here we will use password=HMAC, iterations=4096 to generate credentials.
{
"version":1,
"owner" : "owner",
"renewer" : "renewer",
"issueDate" : "issueDate",
"tokenID" : "sequence-number",
//Store SCRAM credentials as per KIP-84
"credentials": {
"SCRAM-SHA-512" :
"salt=<Salt>,stored_key=<StoredKey>,server_key=<ServerKey>,iterations=<Iterations>",
"SCRAM-SHA-256" :
"salt=<Salt>,stored_key=<StoredKey>,server_key=<ServerKey>,iterations=<Iterations>
}
};
107.2 Do we need to somehow normalize tokeUID before using it
> in /tokenauth/tokens/<tokenUID> (e.g., what happens if tokenUUID includes
> '/')?
>
We will use a a sequence number as tokenUID. Will update the terminology
tokenUID => tokenID.
> 107.3 How does each broker know about changes in tokens? Does each broker
> register a watcher for /tokenauth/tokens?
>
Yes, the changes are updated through zk notifications.
> 107.4 How is token deletion handled? Does every broker delete the same
> token independently or only one broker does the deletion?
>
Only one broker does the deletion. Broker updates the expiration in its
local cache
and on zookeeper so other brokers also get notified and their cache
statuses are updated as well.
Thanks.
Manikumar
>
> On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I would like to initiate the vote on KIP-48:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> > Delegation+token+support+for+Kafka
> >
> >
> > Thanks,
> > Manikumar
> >
>
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Hi, Mani
Thanks for the proposal. Looks good at the high level. A few comments
below.
101. DelegationTokenRequest/DelegationTokenResponse
101.1 I am wondering if renewer should be principalType + name to match
what we have in kafka-acls tool?
101.2 Do we need to return owner in DelegationTokenResponse?
101.3 Do we need to return renewer in DelegationTokenResponse?
102. RenewDelegationTokenRequest/RenewDelegationTokenResponse
102.1 In RenewDelegationTokenRequest, is the renewal time bounded
by delegation.token.expiry.time.sec ?
102.2 Do we need to include TokenDetails in RenewDelegationTokenResponse?
103. Do we need a DescribeDelegationTokenRequest and a corresponding tool
to list/describe existing delegation tokens?
104. KafkaConfig: Do we need a new config to disable/enable delegation
tokens?
105. Could we be more consistent with the naming in configs, the request
protocol fields and ZK structure with respect to time
(e.g. delegation.token.max.lifetime.sec => delegation.token.max.lifetime.ms
)?
106. Using SASL SCRAM for delegation token
106.1 What should be set for username? Does it matter?
106.2 Could you provide a bit more details on how to distinguish it from
regular SASL SCRAM?
107. ZK structure
107.1 Could you provide more details on what SCRAM credentials are stored
in ZK path /tokenauth/tokens/<tokenUID>?
107.2 Do we need to somehow normalize tokeUID before using it
in /tokenauth/tokens/<tokenUID> (e.g., what happens if tokenUUID includes
'/')?
107.3 How does each broker know about changes in tokens? Does each broker
register a watcher for /tokenauth/tokens?
107.4 How is token deletion handled? Does every broker delete the same
token independently or only one broker does the deletion?
Thanks,
Jun
On Fri, Dec 23, 2016 at 9:26 AM, Manikumar <ma...@gmail.com>
wrote:
> Hi,
>
> I would like to initiate the vote on KIP-48:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+
> Delegation+token+support+for+Kafka
>
>
> Thanks,
> Manikumar
>