You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Harsha <ka...@harsha.io> on 2016/06/09 16:48:49 UTC
Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication
mechanism
Jun & Ismael,
Unfortunately I couldn't attend the KIP meeting
when delegation tokens discussed. Appreciate if
you can update the thread if you have any
further questions.
Thanks,
Harsha
On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> It seems that the links to images in the KIP are broken.
>
> Liquan
>
> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> brahmbhatt.parth@gmail.com> wrote:
>
> > 110. What does getDelegationTokenAs mean?
> > In the current proposal we only allow a user to get delegation token for
> > the identity that it authenticated as using another mechanism, i.e. A user
> > that authenticate using a keytab for principal user1@EXAMPLE.COM will get
> > delegation tokens for that user only. In future I think we will have to
> > extend support such that we allow some set of users (
> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to acquire
> > delegation tokens on behalf of other users whose identity they have
> > verified independently. Kafka brokers will have ACLs to control which
> > users are allowed to impersonate other users and get tokens on behalf of
> > them. Overall Impersonation is a whole different problem in my opinion and
> > I think we can tackle it in separate KIP.
> >
> > 111. What's the typical rate of getting and renewing delegation tokens?
> > Typically this should be very very low, 1 request per minute is a
> > relatively high estimate. However it depends on the token expiration. I am
> > less worried about the extra load it puts on controller vs the added
> > complexity and the value it offers.
> >
> > Thanks
> > Parth
> >
> >
> >
> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Thanks Rajini. It would probably require a separate KIP as it will
> > > introduce user visible changes. We could also update KIP-48 to have this
> > > information, but it seems cleaner to do it separately. We can discuss
> > that
> > > in the KIP call today.
> > >
> > > Ismael
> > >
> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > > > Ismael,
> > > >
> > > > I have created a JIRA (
> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > for adding SCRAM as a SASL mechanism. Would that need another KIP? If
> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets
> > > reviewed
> > > > when the PR is ready?
> > > >
> > > > Thank you,
> > > >
> > > > Rajini
> > > >
> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > > >
> > > > > Gwen had independently mentioned this as a SASL mechanism that might
> > be
> > > > > useful for Kafka and I have been meaning to check it in more detail.
> > > Good
> > > > > to know that you are willing to contribute an implementation. Maybe
> > we
> > > > > should file a separate JIRA for this?
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > > > rajinisivaram@googlemail.com> wrote:
> > > > >
> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is a
> > > better
> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in SCRAM
> > > > > > SaslServer or SaslClient, but I will be happy to add support in
> > Kafka
> > > > > since
> > > > > > it would be a useful mechanism to support anyway.
> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol for
> > > > > > SCRAM-SHA-256.
> > > > > >
> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <ju...@confluent.io> wrote:
> > > > > >
> > > > > > > Parth,
> > > > > > >
> > > > > > > Thanks for the explanation. A couple of more questions.
> > > > > > >
> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > > >
> > > > > > > 111. What's the typical rate of getting and renewing delegation
> > > > tokens?
> > > > > > > That may have an impact on whether they should be directed to the
> > > > > > > controller.
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Jun,
> > > > > > > >
> > > > > > > > Thanks for reviewing.
> > > > > > > >
> > > > > > > > * We could add a Cluster action to add acls on who can request
> > > > > > delegation
> > > > > > > > tokens. I don't see the use case for that yet but down the line
> > > > when
> > > > > we
> > > > > > > > start supporting getDelegationTokenAs it will be necessary.
> > > > > > > > * Yes we recommend tokens to be only used/distributed over
> > secure
> > > > > > > channels.
> > > > > > > > * Depending on what design we end up choosing Invalidation will
> > > be
> > > > > > > > responsibility of every broker or controller.
> > > > > > > > * I am not sure if I documented somewhere that invalidation
> > will
> > > > > > directly
> > > > > > > > go through zookeeper but that is not the intent. Invalidation
> > > will
> > > > > > either
> > > > > > > > be request based or due to expiration. No direct zookeeper
> > > > > interaction
> > > > > > > from
> > > > > > > > any client.
> > > > > > > > * "Broker also stores the DelegationToken without the hmac in
> > the
> > > > > > > > zookeeper." : Sorry about the confusion. The sole purpose of
> > > > > zookeeper
> > > > > > in
> > > > > > > > this design is as distribution channel for tokens between all
> > > > brokers
> > > > > > > and a
> > > > > > > > layer that ensures only tokens that were generated by making a
> > > > > request
> > > > > > > to a
> > > > > > > > broker will be accepted (more on this in second paragraph). The
> > > > token
> > > > > > > > consists of few elements (owner, renewer, uuid , expiration,
> > > hmac)
> > > > ,
> > > > > > one
> > > > > > > of
> > > > > > > > which is the finally generated hmac but hmac it self is
> > derivable
> > > > if
> > > > > > you
> > > > > > > > have all the other elements of the token + secret key to
> > generate
> > > > > hmac.
> > > > > > > > Given zookeeper does not provide SSL support we do not want the
> > > > > entire
> > > > > > > > token to be wire transferred to zookeeper as that will be an
> > > > insecure
> > > > > > > wire
> > > > > > > > transfer. Instead we only store all the other elements of a
> > > > > delegation
> > > > > > > > tokens. Brokers can read these elements and because they also
> > > have
> > > > > > access
> > > > > > > > to secret key they will be able to generate hmac on their end.
> > > > > > > >
> > > > > > > > One of the alternative proposed is to avoid zookeeper
> > > altogether. A
> > > > > > > Client
> > > > > > > > will call broker with required information (owner, renwer,
> > > > > expiration)
> > > > > > > and
> > > > > > > > get back (signed hmac, uuid). Broker won't store this in
> > > zookeeper.
> > > > > > From
> > > > > > > > this point a client can contact any broker with all the
> > > delegation
> > > > > > token
> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker will
> > > > > regenerate
> > > > > > > the
> > > > > > > > hmac and as long as it matches with hmac presented by client ,
> > > > broker
> > > > > > > will
> > > > > > > > allow the request to authenticate. Only problem with this
> > > approach
> > > > > is
> > > > > > if
> > > > > > > > the secret key is compromised any client can now generate
> > random
> > > > > tokens
> > > > > > > and
> > > > > > > > they will still be able to authenticate as any user they like.
> > > with
> > > > > > > > zookeeper we guarantee that only tokens acquired via a broker
> > > > (using
> > > > > > some
> > > > > > > > auth scheme other than delegation token) will be accepted. We
> > > need
> > > > to
> > > > > > > > discuss which proposal makes more sense and we can go over it
> > in
> > > > > > > tomorrow's
> > > > > > > > meeting.
> > > > > > > >
> > > > > > > > Also, can you forward the invite to me?
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Parth
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > > > > >
> > > > > > > > > 100. This potentially can be useful for Kafka Connect and
> > Kafka
> > > > > rest
> > > > > > > > proxy
> > > > > > > > > where a worker agent will need to run a task on behalf of a
> > > > client.
> > > > > > We
> > > > > > > > will
> > > > > > > > > likely need to change how those services use Kafka clients
> > > > > > > > > (producer/consumer). Instead of a shared client per worker,
> > we
> > > > will
> > > > > > > need
> > > > > > > > a
> > > > > > > > > client per user task since the authentication happens at the
> > > > > > connection
> > > > > > > > > level. For Kafka Connect, the renewer will be the workers.
> > So,
> > > we
> > > > > > > > probably
> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy, the
> > > > renewer
> > > > > > can
> > > > > > > > > probably just be the creator of the token.
> > > > > > > > >
> > > > > > > > > 101. Do we need new acl on who can request delegation tokens?
> > > > > > > > >
> > > > > > > > > 102. Do we recommend people to send delegation tokens in an
> > > > > encrypted
> > > > > > > > > channel?
> > > > > > > > >
> > > > > > > > > 103. Who is responsible for expiring tokens, every broker?
> > > > > > > > >
> > > > > > > > > 104. For invalidating tokens, would it be better to do it in
> > a
> > > > > > request
> > > > > > > > > instead of going to ZK directly?
> > > > > > > > >
> > > > > > > > > 105. The terminology of client in the wiki sometimes refers
> > to
> > > > the
> > > > > > end
> > > > > > > > > client and some other times refers to the client using the
> > > > > delegation
> > > > > > > > > tokens. It would be useful to distinguish between the two.
> > > > > > > > >
> > > > > > > > > 106. Could you explain the sentence "Broker also stores the
> > > > > > > > DelegationToken
> > > > > > > > > without the hmac in the zookeeper." a bit more? I thought the
> > > > > > > delegation
> > > > > > > > > token is the hmac.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi, Harsha,
> > > > > > > > > >
> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss this in
> > > the
> > > > > > > meeting
> > > > > > > > > > tomorrow.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <ka...@harsha.io>
> > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> Hi All,
> > > > > > > > > >> Can we have a KIP meeting around this. The KIP
> > is
> > > > up
> > > > > > for
> > > > > > > > > >> sometime and if there are any questions lets
> > > > quickly
> > > > > > hash
> > > > > > > > out
> > > > > > > > > >> details.
> > > > > > > > > >>
> > > > > > > > > >> Thanks,
> > > > > > > > > >> Harsha
> > > > > > > > > >>
> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt wrote:
> > > > > > > > > >> > That is what the hadoop echo system uses so no good
> > reason
> > > > > > really.
> > > > > > > > We
> > > > > > > > > >> > could
> > > > > > > > > >> > change it to whatever is the newest recommended standard
> > > is.
> > > > > > > > > >> >
> > > > > > > > > >> > Thanks
> > > > > > > > > >> > Parth
> > > > > > > > > >> >
> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
> > > > > ismael@juma.me.uk
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >> >
> > > > > > > > > >> > > Hi Parth,
> > > > > > > > > >> > >
> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing this and
> > > may
> > > > > have
> > > > > > > > > >> additional
> > > > > > > > > >> > > questions later. The immediate question that came to
> > > mind
> > > > is
> > > > > > our
> > > > > > > > > >> choice of
> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE in
> > the
> > > > IANA
> > > > > > > > > Registry
> > > > > > > > > >> of
> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has been
> > > moved
> > > > > to
> > > > > > > > > Historic
> > > > > > > > > >> > > status:
> > > > > > > > > >> > >
> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > > > > > > > > >> > >
> > > > > > > > >
> > > > > >
> > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > > > > > > >> > >
> > > > > > > > > >> > > What is the reasoning behind that choice?
> > > > > > > > > >> > >
> > > > > > > > > >> > > Thanks,
> > > > > > > > > >> > > Ismael
> > > > > > > > > >> > >
> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira <
> > > > > > > gwen@confluent.io
> > > > > > > > >
> > > > > > > > > >> wrote:
> > > > > > > > > >> > >
> > > > > > > > > >> > > > Also comments inline :)
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > * I want to emphasize that even though delegation
> > > > tokens
> > > > > > > are a
> > > > > > > > > >> Hadoop
> > > > > > > > > >> > > > > innovation, I feel very strongly about not adding
> > > > > > dependency
> > > > > > > > on
> > > > > > > > > >> Hadoop
> > > > > > > > > >> > > > > when implementing delegation tokens for Kafka. The
> > > KIP
> > > > > > > doesn't
> > > > > > > > > >> imply
> > > > > > > > > >> > > > > such dependency, but if you can clarify...
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will read
> > the
> > > > KIP
> > > > > > and
> > > > > > > > > panic
> > > > > > > > > >> > > > three weeks before the next release...
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > * Can we get delegation token at any time after
> > > > > > > > authenticating?
> > > > > > > > > >> only
> > > > > > > > > >> > > > > immediately after?
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *As long as you are authenticated you can get
> > > > delegation
> > > > > > > > tokens.
> > > > > > > > > >> We
> > > > > > > > > >> > > need
> > > > > > > > > >> > > > to
> > > > > > > > > >> > > > > discuss if a client authenticated using delegation
> > > > > token,
> > > > > > > can
> > > > > > > > > also
> > > > > > > > > >> > > > acquire
> > > > > > > > > >> > > > > delegation token again or not. Also there is the
> > > > > question
> > > > > > of
> > > > > > > > do
> > > > > > > > > we
> > > > > > > > > >> > > allow
> > > > > > > > > >> > > > > anyone to acquire delegation token or we want
> > > specific
> > > > > > ACLs
> > > > > > > (I
> > > > > > > > > >> think
> > > > > > > > > >> > > its
> > > > > > > > > >> > > > an
> > > > > > > > > >> > > > > overkill.)*
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > I think we are debating two options: Either require
> > > > > Kerberos
> > > > > > > > auth
> > > > > > > > > >> for
> > > > > > > > > >> > > > renewal or require non-owners to renew.
> > > > > > > > > >> > > > I *think* the latter is simpler (it basically
> > require
> > > a
> > > > > "job
> > > > > > > > > master"
> > > > > > > > > >> > > > to take responsibility for the renewal, it will have
> > > its
> > > > > own
> > > > > > > > > >> identity
> > > > > > > > > >> > > > anyway and I think this is the correct design
> > pattern
> > > > > > anyway.
> > > > > > > > For
> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate renewals?),
> > but
> > > > it
> > > > > is
> > > > > > > > hard
> > > > > > > > > to
> > > > > > > > > >> > > > debate simplicity without looking at the code
> > changes
> > > > > > > required.
> > > > > > > > If
> > > > > > > > > >> you
> > > > > > > > > >> > > > have a draft of how the "require Kerberos" will look
> > > in
> > > > > > Kafka
> > > > > > > > > code,
> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > * My understanding is that tokens will propagate
> > via
> > > > ZK
> > > > > > but
> > > > > > > > > >> without
> > > > > > > > > >> > > > > additional changes to UpdateMetadata protocol,
> > > > correct?
> > > > > > > > Clients
> > > > > > > > > >> > > > > currently don't retry on SASL auth failure (IIRC),
> > > but
> > > > > > since
> > > > > > > > the
> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we will
> > > need
> > > > to
> > > > > > > > retry a
> > > > > > > > > >> bit
> > > > > > > > > >> > > > > to avoid clients failing auth due to timing
> > issues.
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *I am considering 2 alternatives right now. The
> > > > current
> > > > > > > > > documented
> > > > > > > > > >> > > > approach
> > > > > > > > > >> > > > > is zookeeper based and it does not require any
> > > changes
> > > > > to
> > > > > > > > > >> > > UpdateMetadata
> > > > > > > > > >> > > > > protocol. An alternative approach can remove
> > > zookeeper
> > > > > > > > > dependency
> > > > > > > > > >> as
> > > > > > > > > >> > > well
> > > > > > > > > >> > > > > but we can discuss that in KIP discussion call.*
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping Jun to
> > > > > > arrange a
> > > > > > > > > call?
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just the
> > > > > > controller
> > > > > > > > > issue
> > > > > > > > > >> the
> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared
> > secret.
> > > > Not
> > > > > > > sure
> > > > > > > > if
> > > > > > > > > >> we
> > > > > > > > > >> > > > > want to continue the discussion here or on the
> > > wiki. I
> > > > > > think
> > > > > > > > > that
> > > > > > > > > >> we
> > > > > > > > > >> > > > > can decouple the problem of "token distribution"
> > > from
> > > > > > > "shared
> > > > > > > > > >> secret
> > > > > > > > > >> > > > > distribution" and use the controller as the only
> > > token
> > > > > > > > generator
> > > > > > > > > >> to
> > > > > > > > > >> > > > > solve the second issue, while still using ZK async
> > > to
> > > > > > > > distribute
> > > > > > > > > >> > > > > tokens.
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *As mentioned in the previous Email I am fine with
> > > > that
> > > > > > > > approach
> > > > > > > > > >> as
> > > > > > > > > >> > > long
> > > > > > > > > >> > > > as
> > > > > > > > > >> > > > > we agree that the extra complexity of
> > > adding/updating
> > > > > APIS
> > > > > > > > adds
> > > > > > > > > >> enough
> > > > > > > > > >> > > > > value. The advantage with the controller approach
> > is
> > > > > > secret
> > > > > > > > > >> rotation
> > > > > > > > > >> > > can
> > > > > > > > > >> > > > be
> > > > > > > > > >> > > > > automated,frequent and would not require
> > > deployment. *
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Can you detail the extra complexity (or point me to
> > > the
> > > > > > email
> > > > > > > I
> > > > > > > > > >> > > > missed?) - which Apis are required?
> > > > > > > > > >> > > > As far as I can tell, clients can already find the
> > > > > > controller
> > > > > > > > from
> > > > > > > > > >> > > > metadata. I'm a bit more concerned about controller
> > > > load.
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos auth
> > for
> > > > > > > renwal, I
> > > > > > > > > >> think
> > > > > > > > > >> > > > > it mixes the transport layer the the request
> > content
> > > > in
> > > > > a
> > > > > > > > pretty
> > > > > > > > > >> ugly
> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is
> > > better.
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *I feel this is a necessary evil. While this will
> > > make
> > > > > the
> > > > > > > > kafka
> > > > > > > > > >> code
> > > > > > > > > >> > > > > pretty straight forward , forcing renewer to
> > > > non-owner
> > > > > > > pushes
> > > > > > > > > >> the code
> > > > > > > > > >> > > > > ugliness to client and makes it even harder to
> > > > > > integrate. *
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > As mentioned before, I don't think the "renewal by
> > > > other"
> > > > > > > > approach
> > > > > > > > > >> is
> > > > > > > > > >> > > > that ugly for the clients we expect to use
> > delegation
> > > > > tokens
> > > > > > > > since
> > > > > > > > > >> > > > they will have an app-master of some sort who
> > > requested
> > > > > the
> > > > > > > > token
> > > > > > > > > to
> > > > > > > > > >> > > > begin with.
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > The response for my question on how multiple
> > > > identities
> > > > > > will
> > > > > > > > be
> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we don't
> > > > > > > > authenticate
> > > > > > > > > >> each
> > > > > > > > > >> > > > > request, we authenticate connections.
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > *We authenticate connections, and only when they
> > are
> > > > > being
> > > > > > > > > >> established.
> > > > > > > > > >> > > > Let
> > > > > > > > > >> > > > > me try to phrase this as a question, in absence of
> > > > > > > delegation
> > > > > > > > > >> tokens if
> > > > > > > > > >> > > > we
> > > > > > > > > >> > > > > had to support the use case using user TGT's how
> > > would
> > > > > we
> > > > > > do
> > > > > > > > it?
> > > > > > > > > >> My
> > > > > > > > > >> > > point
> > > > > > > > > >> > > > > was it would be no different with delegation
> > tokens.
> > > > The
> > > > > > use
> > > > > > > > > case
> > > > > > > > > >> you
> > > > > > > > > >> > > are
> > > > > > > > > >> > > > > describing seems more like impersonation.*
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Yeah, I thought that one of the things that
> > delegation
> > > > > > tokens
> > > > > > > > > >> handled.
> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Gwen
> > > > > > > > > >> > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > Thanks
> > > > > > > > > >> > > > > Parth
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen Shapira <
> > > > > > > > > gwen@confluent.io
> > > > > > > > > >> >
> > > > > > > > > >> > > > wrote:
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> Few more comments:
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * The API / RequestResponse section doesn't seem
> > to
> > > > > have
> > > > > > > good
> > > > > > > > > >> > > > >> description of the changes to the Kafka Protocol.
> > > > > Sounds
> > > > > > > like
> > > > > > > > > >> you are
> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
> > > > > > RenewTokenRequest
> > > > > > > > (and
> > > > > > > > > >> > > > >> matching responses), without detailing the
> > contents
> > > > of
> > > > > > the
> > > > > > > > > >> requests
> > > > > > > > > >> > > > >> and responses? Or rather, you show the class
> > > > interface,
> > > > > > but
> > > > > > > > not
> > > > > > > > > >> the
> > > > > > > > > >> > > > >> underlying protocol. This could be seen as an
> > > > > > > implementation
> > > > > > > > > >> detail,
> > > > > > > > > >> > > > >> but since the binary protocol is what we provide
> > to
> > > > > > > non-Java
> > > > > > > > > >> clients,
> > > > > > > > > >> > > > >> we need to show the changes there.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> > > > > > > > delegationTokenRequestHandler?
> > > > > > > > > >> Is it
> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client? Its
> > > > > unclear...
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * I want to emphasize that even though delegation
> > > > > tokens
> > > > > > > are
> > > > > > > > a
> > > > > > > > > >> Hadoop
> > > > > > > > > >> > > > >> innovation, I feel very strongly about not adding
> > > > > > > dependency
> > > > > > > > on
> > > > > > > > > >> Hadoop
> > > > > > > > > >> > > > >> when implementing delegation tokens for Kafka.
> > The
> > > > KIP
> > > > > > > > doesn't
> > > > > > > > > >> imply
> > > > > > > > > >> > > > >> such dependency, but if you can clarify...
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * Can we get delegation token at any time after
> > > > > > > > authenticating?
> > > > > > > > > >> only
> > > > > > > > > >> > > > >> immediately after?
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * My understanding is that tokens will propagate
> > > via
> > > > ZK
> > > > > > but
> > > > > > > > > >> without
> > > > > > > > > >> > > > >> additional changes to UpdateMetadata protocol,
> > > > correct?
> > > > > > > > Clients
> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure
> > (IIRC),
> > > > but
> > > > > > > since
> > > > > > > > > the
> > > > > > > > > >> > > > >> tokens propagate between brokers asynch, we will
> > > need
> > > > > to
> > > > > > > > retry
> > > > > > > > > a
> > > > > > > > > >> bit
> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing
> > issues.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * Strongly agreeing on clients not touching ZK
> > > > directly
> > > > > > :)
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having just the
> > > > > > controller
> > > > > > > > > >> issue the
> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared
> > > secret.
> > > > > Not
> > > > > > > sure
> > > > > > > > > if
> > > > > > > > > >> we
> > > > > > > > > >> > > > >> want to continue the discussion here or on the
> > > wiki.
> > > > I
> > > > > > > think
> > > > > > > > > >> that we
> > > > > > > > > >> > > > >> can decouple the problem of "token distribution"
> > > from
> > > > > > > "shared
> > > > > > > > > >> secret
> > > > > > > > > >> > > > >> distribution" and use the controller as the only
> > > > token
> > > > > > > > > generator
> > > > > > > > > >> to
> > > > > > > > > >> > > > >> solve the second issue, while still using ZK
> > async
> > > to
> > > > > > > > > distribute
> > > > > > > > > >> > > > >> tokens.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite lifetime
> > of
> > > > > > tokens
> > > > > > > > (and
> > > > > > > > > >> hoped
> > > > > > > > > >> > > > >> to hear from others in the community) - but
> > having
> > > > the
> > > > > > > option
> > > > > > > > > and
> > > > > > > > > >> > > > >> documenting it as less secure, allows users to
> > > > > configure
> > > > > > > > their
> > > > > > > > > >> system
> > > > > > > > > >> > > > >> as the wish.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * While I like the idea of forcing kerberos auth
> > > for
> > > > > > > renwal,
> > > > > > > > I
> > > > > > > > > >> think
> > > > > > > > > >> > > > >> it mixes the transport layer the the request
> > > content
> > > > > in a
> > > > > > > > > pretty
> > > > > > > > > >> ugly
> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner is
> > > better.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> Things I'd still like to see:
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * More detailed explanation on what we plan to do
> > > for
> > > > > the
> > > > > > > > java
> > > > > > > > > >> clients
> > > > > > > > > >> > > > >> specifically - new configuration? new APIs?
> > > > > > > > > >> > > > >> The response for my question on how multiple
> > > > identities
> > > > > > > will
> > > > > > > > be
> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we
> > don't
> > > > > > > > authenticate
> > > > > > > > > >> each
> > > > > > > > > >> > > > >> request, we authenticate connections.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only used
> > in
> > > > the
> > > > > > > Hadoop
> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> > alternatives
> > > in
> > > > > > other
> > > > > > > > > >> ecosystems
> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether there
> > are
> > > > some
> > > > > > > > > >> advantages
> > > > > > > > > >> > > > >> there.
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> Gwen
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
> > > > > kafka@harsha.io
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >> > > > >> > Hi Gwen,
> > > > > > > > > >> > > > >> > Can you look at Parth's last reply.
> > > Does
> > > > > it
> > > > > > > > answer
> > > > > > > > > >> your
> > > > > > > > > >> > > > >> > concerns.
> > > > > > > > > >> > > > >> >
> > > > > > > > > >> > > > >> > Thanks,
> > > > > > > > > >> > > > >> > Harsha
> > > > > > > > > >> > > > >> >
> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
> > > brahmbhatt
> > > > > > wrote:
> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki already
> > has
> > > > > > details
> > > > > > > on
> > > > > > > > > >> token
> > > > > > > > > >> > > > >> >> expiration
> > > > > > > > > >> > > > >> >> under token acquisition process
> > > > > > > > > >> > > > >> >> <
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> > > > > > > > > >> > > > >> >.
> > > > > > > > > >> > > > >> >> Current proposal is that tokens will expire
> > > based
> > > > > on a
> > > > > > > > > server
> > > > > > > > > >> side
> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless
> > renewed.
> > > > > > Renewal
> > > > > > > > is
> > > > > > > > > >> only
> > > > > > > > > >> > > > allowed
> > > > > > > > > >> > > > >> >> until the max life time of token.
> > Alternatively
> > > we
> > > > > > could
> > > > > > > > > also
> > > > > > > > > >> make
> > > > > > > > > >> > > > that
> > > > > > > > > >> > > > >> >> an
> > > > > > > > > >> > > > >> >> optional param and the server side default can
> > > > serve
> > > > > > as
> > > > > > > > the
> > > > > > > > > >> upper
> > > > > > > > > >> > > > bound.
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >> To your second point it will be done exactly
> > the
> > > > > same
> > > > > > > way
> > > > > > > > we
> > > > > > > > > >> would
> > > > > > > > > >> > > > >> >> support
> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client will have
> > > to
> > > > > put
> > > > > > > the
> > > > > > > > > >> tokens it
> > > > > > > > > >> > > > >> wants
> > > > > > > > > >> > > > >> >> to use in the subject instance and call
> > > > > > produce/consume
> > > > > > > > > inside
> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to keep
> > > track
> > > > of
> > > > > > its
> > > > > > > > own
> > > > > > > > > >> > > > subject. I
> > > > > > > > > >> > > > >> >> will have to look at the code to see if we
> > > support
> > > > > > this
> > > > > > > > > >> feature
> > > > > > > > > >> > > right
> > > > > > > > > >> > > > >> now
> > > > > > > > > >> > > > >> >> but my understanding is delegation token
> > > shouldn't
> > > > > > need
> > > > > > > > any
> > > > > > > > > >> special
> > > > > > > > > >> > > > >> >> treatment as its just another type of
> > Credential
> > > > in
> > > > > > the
> > > > > > > > > >> subject.
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >> I would also like to know what is your opinion
> > > > about
> > > > > > > > > infinite
> > > > > > > > > >> > > renewal
> > > > > > > > > >> > > > >> (my
> > > > > > > > > >> > > > >> >> recommendation is to not support this), tokens
> > > > > > renewing
> > > > > > > > them
> > > > > > > > > >> > > self(my
> > > > > > > > > >> > > > >> >> recommendation is to not support this) and
> > most
> > > > > > > > importantly
> > > > > > > > > >> your
> > > > > > > > > >> > > > choice
> > > > > > > > > >> > > > >> >> between the alternatives listed on this thread
> > > > > > > > > >> > > > >> >> <
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> > > > > > > > > >> > > > >> >
> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2 minus
> > > > > > > controller
> > > > > > > > > >> > > > distributing
> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >> Thanks
> > > > > > > > > >> > > > >> >> Parth
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen Shapira <
> > > > > > > > > >> gwen@confluent.io>
> > > > > > > > > >> > > > wrote:
> > > > > > > > > >> > > > >> >>
> > > > > > > > > >> > > > >> >> > Harsha,
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I didn't
> > see
> > > > > your
> > > > > > > > design
> > > > > > > > > >> yet,
> > > > > > > > > >> > > > but in
> > > > > > > > > >> > > > >> >> > our proxy, we have a set of producers, which
> > > > will
> > > > > > > serve
> > > > > > > > > >> multiple
> > > > > > > > > >> > > > users
> > > > > > > > > >> > > > >> >> > going through the proxy. Since these users
> > > will
> > > > > have
> > > > > > > > > >> different
> > > > > > > > > >> > > > >> >> > privileges, they'll need to authenticate
> > > > > separately,
> > > > > > > and
> > > > > > > > > >> can't
> > > > > > > > > >> > > > share a
> > > > > > > > > >> > > > >> >> > token.
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >> >> > Am I missing anything?
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >> >> > Gwen
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha <
> > > > > > > kafka@harsha.io
> > > > > > > > >
> > > > > > > > > >> wrote:
> > > > > > > > > >> > > > >> >> > > Gwen,
> > > > > > > > > >> > > > >> >> > > On your second point. Can you
> > > > > describe
> > > > > > a
> > > > > > > > > >> usecase
> > > > > > > > > >> > > where
> > > > > > > > > >> > > > >> >> > > mutliple clients ended up
> > > sharing a
> > > > > > > > producer
> > > > > > > > > >> and
> > > > > > > > > >> > > even
> > > > > > > > > >> > > > if
> > > > > > > > > >> > > > >> they
> > > > > > > > > >> > > > >> >> > > do why can't they not use
> > single
> > > > > token
> > > > > > > that
> > > > > > > > > >> producer
> > > > > > > > > >> > > > >> >> > > captures. Why would we need
> > > > multiple
> > > > > > > > clients
> > > > > > > > > >> with
> > > > > > > > > >> > > > >> different
> > > > > > > > > >> > > > >> >> > > tokens sharing a single
> > instance
> > > of
> > > > > > > > producer.
> > > > > > > > > >> Also
> > > > > > > > > >> > > in
> > > > > > > > > >> > > > >> this
> > > > > > > > > >> > > > >> >> > > case other clients have access
> > > all
> > > > > the
> > > > > > > > tokens
> > > > > > > > > >> no?
> > > > > > > > > >> > > > >> >> > >
> > > > > > > > > >> > > > >> >> > > Thanks,
> > > > > > > > > >> > > > >> >> > > Harsha
> > > > > > > > > >> > > > >> >> > >
> > > > > > > > > >> > > > >> >> > >
> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, Gwen
> > > Shapira
> > > > > > > wrote:
> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> > > > > > > > > >> > > > >> >> > >>
> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in the
> > > wiki:
> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for delegation
> > > > > tokens?
> > > > > > > > > >> Renewal? How
> > > > > > > > > >> > > > do we
> > > > > > > > > >> > > > >> >> > >> revoke them?
> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation tokens
> > for
> > > > > > "do-as"
> > > > > > > > > (say,
> > > > > > > > > >> > > submit
> > > > > > > > > >> > > > >> Storm
> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a producer
> > > for
> > > > > > every
> > > > > > > > job
> > > > > > > > > >> (we
> > > > > > > > > >> > > can't
> > > > > > > > > >> > > > >> share
> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running on
> > same
> > > > > node),
> > > > > > > > since
> > > > > > > > > >> we
> > > > > > > > > >> > > only
> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is there a
> > > plan
> > > > > to
> > > > > > > > change
> > > > > > > > > >> this
> > > > > > > > > >> > > for
> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow
> > > multiple
> > > > > > users
> > > > > > > > > with
> > > > > > > > > >> > > > different
> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> > > > > > > > > >> > > > >> >> > >>
> > > > > > > > > >> > > > >> >> > >> Gwen
> > > > > > > > > >> > > > >> >> > >>
> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, parth
> > > > > brahmbhatt
> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com> wrote:
> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can
> > other
> > > > > > > committers
> > > > > > > > > >> review?
> > > > > > > > > >> > > > >> >> > >> >
> > > > > > > > > >> > > > >> >> > >> > Thanks
> > > > > > > > > >> > > > >> >> > >> > Parth
> > > > > > > > > >> > > > >> >> > >> >
> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM,
> > Harsha <
> > > > > > > > > >> kafka@harsha.io>
> > > > > > > > > >> > > > wrote:
> > > > > > > > > >> > > > >> >> > >> >
> > > > > > > > > >> > > > >> >> > >> >> Parth,
> > > > > > > > > >> > > > >> >> > >> >> Overall current design looks
> > > > good
> > > > > to
> > > > > > > > me. I
> > > > > > > > > >> am +1
> > > > > > > > > >> > > on
> > > > > > > > > >> > > > >> the
> > > > > > > > > >> > > > >> >> > KIP.
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as
> > well.
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 AM,
> > parth
> > > > > > > > brahmbhatt
> > > > > > > > > >> wrote:
> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of infinite
> > > > lifetime
> > > > > > > but I
> > > > > > > > > >> see the
> > > > > > > > > >> > > > >> Streaming
> > > > > > > > > >> > > > >> >> > use
> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use case I
> > > was
> > > > > > hoping
> > > > > > > > > >> there will
> > > > > > > > > >> > > > be
> > > > > > > > > >> > > > >> some
> > > > > > > > > >> > > > >> >> > notion
> > > > > > > > > >> > > > >> >> > >> >> > of
> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new
> > > > delegation
> > > > > > > tokens
> > > > > > > > > at
> > > > > > > > > >> fixed
> > > > > > > > > >> > > > >> interval
> > > > > > > > > >> > > > >> >> > and
> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that is
> > not
> > > > the
> > > > > > case
> > > > > > > > for
> > > > > > > > > >> we can
> > > > > > > > > >> > > > >> discuss
> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing them self
> > > and
> > > > > the
> > > > > > > > > >> security
> > > > > > > > > >> > > > >> implications
> > > > > > > > > >> > > > >> >> > of the
> > > > > > > > > >> > > > >> >> > >> >> > same.
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch
> > tokens
> > > > > from
> > > > > > > > > >> zookeeper,
> > > > > > > > > >> > > > >> overall I
> > > > > > > > > >> > > > >> >> > think
> > > > > > > > > >> > > > >> >> > >> >> > its
> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely on our
> > > > > metadata
> > > > > > > > store
> > > > > > > > > >> and I
> > > > > > > > > >> > > > >> think we
> > > > > > > > > >> > > > >> >> > are
> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with all
> > the
> > > > > KIP-4
> > > > > > > > > >> improvements.
> > > > > > > > > >> > > > I
> > > > > > > > > >> > > > >> chose
> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the client
> > > > will
> > > > > > > still
> > > > > > > > > >> just talk
> > > > > > > > > >> > > > to
> > > > > > > > > >> > > > >> >> > broker , its
> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use zookeeper
> > > > which
> > > > > we
> > > > > > > > > >> already do
> > > > > > > > > >> > > > for a
> > > > > > > > > >> > > > >> lot
> > > > > > > > > >> > > > >> >> > of
> > > > > > > > > >> > > > >> >> > >> >> > other
> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development + and
> > > the
> > > > > > > ability
> > > > > > > > > so
> > > > > > > > > >> > > tokens
> > > > > > > > > >> > > > >> will
> > > > > > > > > >> > > > >> >> > survive
> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster
> > > failure.
> > > > > if a
> > > > > > > > > >> majority
> > > > > > > > > >> > > > agrees
> > > > > > > > > >> > > > >> the
> > > > > > > > > >> > > > >> >> > added
> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller
> > > forwarding
> > > > > > keys
> > > > > > > to
> > > > > > > > > all
> > > > > > > > > >> > > > broker is
> > > > > > > > > >> > > > >> >> > justified
> > > > > > > > > >> > > > >> >> > >> >> > as
> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security , I am
> > > fine
> > > > > > with
> > > > > > > > that
> > > > > > > > > >> option
> > > > > > > > > >> > > > too.
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not support SSL
> > > we
> > > > > can
> > > > > > > not
> > > > > > > > > >> store
> > > > > > > > > >> > > > master
> > > > > > > > > >> > > > >> keys
> > > > > > > > > >> > > > >> >> > in
> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will be
> > > exposed
> > > > > on
> > > > > > > > wire.
> > > > > > > > > To
> > > > > > > > > >> > > > support
> > > > > > > > > >> > > > >> >> > rotation
> > > > > > > > > >> > > > >> >> > >> >> > without affecting current clients is
> > > > > > > something I
> > > > > > > > > >> need to
> > > > > > > > > >> > > > put
> > > > > > > > > >> > > > >> more
> > > > > > > > > >> > > > >> >> > thought
> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal assumes the
> > > > > rotation
> > > > > > > > will
> > > > > > > > > >> > > > invalidate
> > > > > > > > > >> > > > >> all
> > > > > > > > > >> > > > >> >> > current
> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also review
> > > and
> > > > > post
> > > > > > > > their
> > > > > > > > > >> > > comments
> > > > > > > > > >> > > > >> so we
> > > > > > > > > >> > > > >> >> > can
> > > > > > > > > >> > > > >> >> > >> >> > make
> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> > > > > > > > > >> > > > >> >> > >> >> > Parth
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 AM,
> > > Ashish
> > > > > > Singh
> > > > > > > <
> > > > > > > > > >> > > > >> asingh@cloudera.com
> > > > > > > > > >> > > > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at 11:26 AM,
> > > > > Harsha <
> > > > > > > > > >> > > > kafka@harsha.io>
> > > > > > > > > >> > > > >> >> > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion
> > > threads
> > > > on
> > > > > > > this
> > > > > > > > > KIP.
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from
> > Jitendra
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large number of
> > > > > clients
> > > > > > > that
> > > > > > > > > are
> > > > > > > > > >> > > > running
> > > > > > > > > >> > > > >> all
> > > > > > > > > >> > > > >> >> > over the
> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate with
> > > Kafka
> > > > > > > > brokers,
> > > > > > > > > >> is very
> > > > > > > > > >> > > > >> similar
> > > > > > > > > >> > > > >> >> > to the
> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large number
> > of
> > > > > tasks
> > > > > > > > > running
> > > > > > > > > >> > > across
> > > > > > > > > >> > > > >> the
> > > > > > > > > >> > > > >> >> > cluster
> > > > > > > > > >> > > > >> >> > >> >> that
> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs
> > > > Namenode.
> > > > > > > > > >> Therefore, the
> > > > > > > > > >> > > > >> >> > delegation token
> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a good
> > fit
> > > > for
> > > > > > > this
> > > > > > > > > use
> > > > > > > > > >> case
> > > > > > > > > >> > > > as we
> > > > > > > > > >> > > > >> >> > have seen
> > > > > > > > > >> > > > >> >> > >> >> it
> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in HDFS
> > and
> > > > > YARN.
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is very
> > much
> > > > > > inline
> > > > > > > > with
> > > > > > > > > >> Hadoop
> > > > > > > > > >> > > > >> >> > approach. A few
> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to allow
> > > > > > infinite
> > > > > > > > > >> renewable
> > > > > > > > > >> > > > >> lifetime
> > > > > > > > > >> > > > >> >> > for a
> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a token
> > to a
> > > > max
> > > > > > > life
> > > > > > > > > time
> > > > > > > > > >> > > > (default
> > > > > > > > > >> > > > >> 7
> > > > > > > > > >> > > > >> >> > days). A
> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
> > believed
> > > to
> > > > > > > > increase
> > > > > > > > > >> with
> > > > > > > > > >> > > > time.
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite
> > > lifetime
> > > > > > might
> > > > > > > > not
> > > > > > > > > >> be the
> > > > > > > > > >> > > > best
> > > > > > > > > >> > > > >> idea.
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the tokens
> > are
> > > > > stored
> > > > > > > in
> > > > > > > > > >> zookeeper
> > > > > > > > > >> > > > as
> > > > > > > > > >> > > > >> well,
> > > > > > > > > >> > > > >> >> > and
> > > > > > > > > >> > > > >> >> > >> >> can
> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is clever
> > > as
> > > > it
> > > > > > can
> > > > > > > > > allow
> > > > > > > > > >> > > > >> replacing the
> > > > > > > > > >> > > > >> >> > tokens
> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max life
> > > time,
> > > > > and
> > > > > > > > > clients
> > > > > > > > > >> can
> > > > > > > > > >> > > > >> download
> > > > > > > > > >> > > > >> >> > new
> > > > > > > > > >> > > > >> >> > >> >> tokens
> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It shouldn't be
> > a
> > > > big
> > > > > > load
> > > > > > > > on
> > > > > > > > > >> > > zookeeper
> > > > > > > > > >> > > > >> as a
> > > > > > > > > >> > > > >> >> > client
> > > > > > > > > >> > > > >> >> > >> >> will
> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token once in
> > > > > several
> > > > > > > > days.
> > > > > > > > > >> In this
> > > > > > > > > >> > > > >> approach
> > > > > > > > > >> > > > >> >> > you
> > > > > > > > > >> > > > >> >> > >> >> don't
> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on the
> > > token
> > > > > even
> > > > > > > for
> > > > > > > > > >> long
> > > > > > > > > >> > > > running
> > > > > > > > > >> > > > >> >> > clients.
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are
> > > generated
> > > > > > using
> > > > > > > a
> > > > > > > > > >> master
> > > > > > > > > >> > > key.
> > > > > > > > > >> > > > >> The
> > > > > > > > > >> > > > >> >> > master
> > > > > > > > > >> > > > >> >> > >> >> key
> > > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically
> > > changed.
> > > > > In
> > > > > > > > > Hadoop,
> > > > > > > > > >> the
> > > > > > > > > >> > > > >> default
> > > > > > > > > >> > > > >> >> > renewal
> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require brokers
> > > > > > maintaining
> > > > > > > a
> > > > > > > > > >> list of X
> > > > > > > > > >> > > > most
> > > > > > > > > >> > > > >> >> > recent
> > > > > > > > > >> > > > >> >> > >> >> master
> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to be
> > > > > persisted
> > > > > > > > > >> somewhere, as
> > > > > > > > > >> > > > if a
> > > > > > > > > >> > > > >> >> > broker
> > > > > > > > > >> > > > >> >> > >> >> goes
> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get that list
> > > > again
> > > > > > and
> > > > > > > > > >> storing
> > > > > > > > > >> > > > master
> > > > > > > > > >> > > > >> keys
> > > > > > > > > >> > > > >> >> > on ZK
> > > > > > > > > >> > > > >> >> > >> >> is
> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However, if a
> > > > broker
> > > > > > goes
> > > > > > > > > down
> > > > > > > > > >> then
> > > > > > > > > >> > > we
> > > > > > > > > >> > > > >> have
> > > > > > > > > >> > > > >> >> > much
> > > > > > > > > >> > > > >> >> > >> >> bigger
> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and client can
> > > > always
> > > > > > > > > >> > > re-authenticate
> > > > > > > > > >> > > > is
> > > > > > > > > >> > > > >> such
> > > > > > > > > >> > > > >> >> > >> >> events.
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a look at
> > > other
> > > > > > > > > >> alternatives
> > > > > > > > > >> > > this
> > > > > > > > > >> > > > >> list has
> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough proposal,
> > > > great
> > > > > > > work!"
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at 10:28
> > PM,
> > > > Gwen
> > > > > > > > Shapira
> > > > > > > > > >> wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me. Thanks!
> > > > > > > > > >> > > > >> >> > >> >> > > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at 9:25
> > PM,
> > > > > > Harsha <
> > > > > > > > > >> > > > kafka@harsha.io
> > > > > > > > > >> > > > >> >
> > > > > > > > > >> > > > >> >> > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any release
> > > > > vehicle
> > > > > > > but
> > > > > > > > > >> still the
> > > > > > > > > >> > > > >> work can
> > > > > > > > > >> > > > >> >> > move
> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is interested in
> > the
> > > > KIP
> > > > > > > > please
> > > > > > > > > >> do the
> > > > > > > > > >> > > > >> review and
> > > > > > > > > >> > > > >> >> > >> >> provide
> > > > > > > > > >> > > > >> >> > >> >> > > the
> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at
> > 04:59
> > > > PM,
> > > > > > > Ismael
> > > > > > > > > >> Juma
> > > > > > > > > >> > > > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would be
> > good
> > > > to
> > > > > > have
> > > > > > > > > more
> > > > > > > > > >> time
> > > > > > > > > >> > > to
> > > > > > > > > >> > > > >> review
> > > > > > > > > >> > > > >> >> > and
> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at
> > 12:55
> > > > AM,
> > > > > > Gwen
> > > > > > > > > >> Shapira <
> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 depends on
> > > > KIP-43,
> > > > > > > which
> > > > > > > > > is
> > > > > > > > > >> > > > already a
> > > > > > > > > >> > > > >> bit
> > > > > > > > > >> > > > >> >> > of a
> > > > > > > > > >> > > > >> >> > >> >> risk
> > > > > > > > > >> > > > >> >> > >> >> > > for
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release - any
> > > chance
> > > > > we
> > > > > > > can
> > > > > > > > > >> delay
> > > > > > > > > >> > > > >> delegation
> > > > > > > > > >> > > > >> >> > tokens
> > > > > > > > > >> > > > >> >> > >> >> to
> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community
> > working
> > > > on a
> > > > > > > > release
> > > > > > > > > >> every
> > > > > > > > > >> > > 3
> > > > > > > > > >> > > > >> month,
> > > > > > > > > >> > > > >> >> > this
> > > > > > > > > >> > > > >> >> > >> >> is not
> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, 2016 at
> > > 5:11
> > > > > PM,
> > > > > > > > Ashish
> > > > > > > > > >> Singh
> > > > > > > > > >> > > <
> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for the
> > > > awesome
> > > > > > > write
> > > > > > > > > up.
> > > > > > > > > >> > > > Following
> > > > > > > > > >> > > > >> our
> > > > > > > > > >> > > > >> >> > >> >> discussion
> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it will
> > be
> > > > > easier
> > > > > > > to
> > > > > > > > > >> compare
> > > > > > > > > >> > > > >> various
> > > > > > > > > >> > > > >> >> > >> >> alternatives
> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together. I am
> > > > stating
> > > > > > > > below a
> > > > > > > > > >> few
> > > > > > > > > >> > > > >> >> > alternatives along
> > > > > > > > > >> > > > >> >> > >> >> > > with
> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal)
> > Store
> > > > > > > Delegation
> > > > > > > > > >> Token,
> > > > > > > > > >> > > DT,
> > > > > > > > > >> > > > >> on ZK.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > > authenticates
> > > > > > with a
> > > > > > > > > >> broker.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
> > > > > > > > authenticated,
> > > > > > > > > >> it
> > > > > > > > > >> > > will
> > > > > > > > > >> > > > >> make a
> > > > > > > > > >> > > > >> >> > broker
> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
> > > > token.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
> > > generates
> > > > a
> > > > > > > shared
> > > > > > > > > >> secret
> > > > > > > > > >> > > > based
> > > > > > > > > >> > > > >> on
> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret
> > shared
> > > > > > between
> > > > > > > > all
> > > > > > > > > >> > > brokers,
> > > > > > > > > >> > > > >> >> > randomly
> > > > > > > > > >> > > > >> >> > >> >> > > generated
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker stores
> > this
> > > > > token
> > > > > > in
> > > > > > > > its
> > > > > > > > > >> in
> > > > > > > > > >> > > > memory
> > > > > > > > > >> > > > >> cache.
> > > > > > > > > >> > > > >> >> > >> >> Broker
> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the DelegationToken
> > > > > without
> > > > > > > the
> > > > > > > > > >> hmac in
> > > > > > > > > >> > > the
> > > > > > > > > >> > > > >> >> > zookeeper.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers will
> > > > have a
> > > > > > > cache
> > > > > > > > > >> backed
> > > > > > > > > >> > > by
> > > > > > > > > >> > > > >> >> > zookeeper so
> > > > > > > > > >> > > > >> >> > >> >> they
> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified
> > whenever
> > > a
> > > > > new
> > > > > > > > token
> > > > > > > > > is
> > > > > > > > > >> > > > >> generated and
> > > > > > > > > >> > > > >> >> > they
> > > > > > > > > >> > > > >> >> > >> >> will
> > > > > > > > > >> > > > >> >> > >> >> > > > update
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache whenever
> > > > token
> > > > > > > state
> > > > > > > > > >> changes.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker returns
> > the
> > > > > token
> > > > > > to
> > > > > > > > > >> Client.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
> > fixes
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable race
> > > > > condition,
> > > > > > > > client
> > > > > > > > > >> tries
> > > > > > > > > >> > > to
> > > > > > > > > >> > > > >> >> > authenticate
> > > > > > > > > >> > > > >> >> > >> >> with
> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to be
> > > > updated
> > > > > > with
> > > > > > > > the
> > > > > > > > > >> newly
> > > > > > > > > >> > > > >> generated
> > > > > > > > > >> > > > >> >> > DT.
> > > > > > > > > >> > > > >> >> > >> >> This
> > > > > > > > > >> > > > >> >> > >> >> > > can
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with making
> > > > > dtRequest
> > > > > > > > block
> > > > > > > > > >> until
> > > > > > > > > >> > > all
> > > > > > > > > >> > > > >> >> > brokers have
> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk barrier or
> > > > > similar
> > > > > > > > > >> mechanism
> > > > > > > > > >> > > can
> > > > > > > > > >> > > > be
> > > > > > > > > >> > > > >> used.
> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will
> > > increase
> > > > > > > > > complexity.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a static
> > > secret
> > > > > key
> > > > > > > > from
> > > > > > > > > >> config
> > > > > > > > > >> > > > >> file. Will
> > > > > > > > > >> > > > >> >> > >> >> require
> > > > > > > > > >> > > > >> >> > >> >> > > yet
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and uses a
> > > static
> > > > > > > secret
> > > > > > > > > >> key. It
> > > > > > > > > >> > > is
> > > > > > > > > >> > > > >> advised
> > > > > > > > > >> > > > >> >> > to
> > > > > > > > > >> > > > >> >> > >> >> rotate
> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. This
> > can
> > > > be
> > > > > > > > avoided
> > > > > > > > > >> with
> > > > > > > > > >> > > > >> controller
> > > > > > > > > >> > > > >> >> > >> >> generating
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to brokers
> > > > > > > periodically.
> > > > > > > > > >> However,
> > > > > > > > > >> > > > >> this will
> > > > > > > > > >> > > > >> >> > >> >> require
> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain certain
> > > counts
> > > > of
> > > > > > > > > >> secretKeys.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1) Have
> > > > > controller
> > > > > > > > > >> generate
> > > > > > > > > >> > > > >> delegation
> > > > > > > > > >> > > > >> >> > token.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > > authenticates
> > > > > > with a
> > > > > > > > > >> broker.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
> > > > > > > > authenticated,
> > > > > > > > > >> it
> > > > > > > > > >> > > will
> > > > > > > > > >> > > > >> make a
> > > > > > > > > >> > > > >> >> > broker
> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
> > > > token.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker forwards
> > the
> > > > > > request
> > > > > > > > to
> > > > > > > > > >> > > > controller.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller
> > > generates
> > > > a
> > > > > DT
> > > > > > > and
> > > > > > > > > >> > > broadcasts
> > > > > > > > > >> > > > >> to all
> > > > > > > > > >> > > > >> >> > >> >> brokers.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker stores
> > this
> > > > > token
> > > > > > in
> > > > > > > > its
> > > > > > > > > >> memory
> > > > > > > > > >> > > > >> cache.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller
> > responds
> > > > to
> > > > > > > > broker’s
> > > > > > > > > >> DT
> > > > > > > > > >> > > req.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker returns
> > the
> > > > > token
> > > > > > to
> > > > > > > > > >> Client.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
> > fixes
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will have to
> > add
> > > > new
> > > > > > > APIs
> > > > > > > > to
> > > > > > > > > >> > > support
> > > > > > > > > >> > > > >> >> > controller
> > > > > > > > > >> > > > >> >> > >> >> pushing
> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top of
> > the
> > > > > > minimal
> > > > > > > > APIs
> > > > > > > > > >> that
> > > > > > > > > >> > > are
> > > > > > > > > >> > > > >> >> > currently
> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will also have
> > > to
> > > > > add
> > > > > > > APIs
> > > > > > > > > to
> > > > > > > > > >> > > support
> > > > > > > > > >> > > > >> the
> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new broker
> > > comes
> > > > up
> > > > > > it
> > > > > > > > will
> > > > > > > > > >> have
> > > > > > > > > >> > > to
> > > > > > > > > >> > > > >> get all
> > > > > > > > > >> > > > >> >> > >> >> delegation
> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In catastrophic
> > > > > failures
> > > > > > > > where
> > > > > > > > > >> all
> > > > > > > > > >> > > > brokers
> > > > > > > > > >> > > > >> go
> > > > > > > > > >> > > > >> >> > down,
> > > > > > > > > >> > > > >> >> > >> >> the
> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
> > > servers
> > > > > are
> > > > > > > > > >> restarted as
> > > > > > > > > >> > > > >> tokens
> > > > > > > > > >> > > > >> >> > are not
> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens,
> > then
> > > > > there
> > > > > > > are
> > > > > > > > > more
> > > > > > > > > >> > > > important
> > > > > > > > > >> > > > >> >> > things to
> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better
> > to
> > > > > > > > > >> re-authenticate.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2) Do not
> > > > > > distribute
> > > > > > > > DT
> > > > > > > > > to
> > > > > > > > > >> > > other
> > > > > > > > > >> > > > >> brokers
> > > > > > > > > >> > > > >> >> > at
> > > > > > > > > >> > > > >> >> > >> >> all.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > > authenticates
> > > > > > with a
> > > > > > > > > >> broker.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
> > > > > > > > authenticated,
> > > > > > > > > >> it
> > > > > > > > > >> > > will
> > > > > > > > > >> > > > >> make a
> > > > > > > > > >> > > > >> >> > broker
> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
> > > > token.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
> > > generates
> > > > DT
> > > > > > of
> > > > > > > > > form,
> > > > > > > > > >> > > [hmac +
> > > > > > > > > >> > > > >> (owner,
> > > > > > > > > >> > > > >> >> > >> >> renewer,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime, id,
> > hmac,
> > > > > > > > > >> expirationTime)]
> > > > > > > > > >> > > and
> > > > > > > > > >> > > > >> passes
> > > > > > > > > >> > > > >> >> > back
> > > > > > > > > >> > > > >> >> > >> >> this
> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is generated
> > via
> > > > > > > > > >> {HMAC-SHA256(owner,
> > > > > > > > > >> > > > >> renewer,
> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime)
> > using
> > > > > > > > SecretKey}.
> > > > > > > > > >> Note
> > > > > > > > > >> > > that
> > > > > > > > > >> > > > >> all
> > > > > > > > > >> > > > >> >> > brokers
> > > > > > > > > >> > > > >> >> > >> >> have
> > > > > > > > > >> > > > >> >> > >> >> > > > this
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then goes
> > to
> > > > any
> > > > > > > > broker
> > > > > > > > > >> and to
> > > > > > > > > >> > > > >> >> > authenticate
> > > > > > > > > >> > > > >> >> > >> >> sends
> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker recalculates
> > > hmac
> > > > > > using
> > > > > > > > > >> (owner,
> > > > > > > > > >> > > > >> renewer,
> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) info
> > > > from
> > > > > DT
> > > > > > > and
> > > > > > > > > its
> > > > > > > > > >> > > > >> SecretKey. If
> > > > > > > > > >> > > > >> >> > it
> > > > > > > > > >> > > > >> >> > >> >> matches
> > > > > > > > > >> > > > >> >> > >> >> > > > with
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is
> > > > > authenticated.
> > > > > > > > Yes,
> > > > > > > > > >> it will
> > > > > > > > > >> > > > do
> > > > > > > > > >> > > > >> other
> > > > > > > > > >> > > > >> >> > >> >> obvious
> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp expiry and
> > > > such.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret key
> > will
> > > > be
> > > > > > > > > generated
> > > > > > > > > >> by
> > > > > > > > > >> > > > >> controller
> > > > > > > > > >> > > > >> >> > and
> > > > > > > > > >> > > > >> >> > >> >> passed
> > > > > > > > > >> > > > >> >> > >> >> > > to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
> > fixes
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to delete a
> > DT?
> > > > > Yes,
> > > > > > > that
> > > > > > > > > is
> > > > > > > > > >> a
> > > > > > > > > >> > > > downside
> > > > > > > > > >> > > > >> >> > here.
> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with
> > > brokers
> > > > > > > > > maintaining
> > > > > > > > > >> a
> > > > > > > > > >> > > > >> blacklist of
> > > > > > > > > >> > > > >> >> > DTs,
> > > > > > > > > >> > > > >> >> > >> >> DTs
> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed after
> > > > > expiry.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In catastrophic
> > > > > failures
> > > > > > > > where
> > > > > > > > > >> all
> > > > > > > > > >> > > > brokers
> > > > > > > > > >> > > > >> go
> > > > > > > > > >> > > > >> >> > down,
> > > > > > > > > >> > > > >> >> > >> >> the
> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
> > > servers
> > > > > are
> > > > > > > > > >> restarted as
> > > > > > > > > >> > > > >> tokens
> > > > > > > > > >> > > > >> >> > are not
> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens,
> > then
> > > > > there
> > > > > > > are
> > > > > > > > > more
> > > > > > > > > >> > > > important
> > > > > > > > > >> > > > >> >> > things to
> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better
> > to
> > > > > > > > > >> re-authenticate.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, 2016 at
> > > > 1:58
> > > > > > PM,
> > > > > > > > > Parth
> > > > > > > > > >> > > > >> Brahmbhatt <
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > pbrahmbhatt@hortonworks.com>
> > > > > > > > wrote:
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed KIP-48 so
> > > we
> > > > > can
> > > > > > > > offer
> > > > > > > > > >> hadoop
> > > > > > > > > >> > > > like
> > > > > > > > > >> > > > >> >> > delegation
> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can review
> > > the
> > > > > > design
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends on
> > > KIP-43
> > > > > and
> > > > > > > we
> > > > > > > > > >> have also
> > > > > > > > > >> > > > >> >> > discussed an
> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design here<
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > --
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > > > > > >> > > > >> >> > >> >>
> > > > > > > > > >> > > > >> >> >
> > > > > > > > > >> > > > >>
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Rajini
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > >
> >
>
>
>
> --
> Liquan Pei
> Software Engineer, Confluent Inc
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Becket Qin <be...@gmail.com>.
According to the meeting minutes of KIP hangout on 8/30, it seems the KIP
wiki needs some update?
KIP48 (delegation tokens): Harsha will update the wiki with more details on
how to use delegation tokens and how to configure it.
Not sure if that has been done or not.
On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Hi Guys,
>
> This discussion was dead for a while. Are there still contentious
> points? If not, why are there no votes?
>
> On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> > Ashish,
> >
> > Yes, I will send out a KIP invite for next week to discuss KIP-48 and
> other
> > remaining KIPs.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com>
> wrote:
> >
> >> Thanks Harsha!
> >>
> >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
> >> actually make a call on when we should have next KIP call. As there are
> a
> >> few outstanding KIPs that could not be discussed this week, can we have
> a
> >> KIP hangout call next week?
> >>
> >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io>
> >> wrote:
> >>
> >>> Ashish,
> >>> Yes we are working on it. Lets discuss in the next KIP meeting.
> >>> I'll join.
> >>> -Harsha
> >>>
> >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
> >>> wrote:
> >>>
> >>> > Hello Harsha,
> >>> >
> >>> > Are you still working on this? Wondering if we can discuss this in
> next
> >>> KIP
> >>> > meeting, if you can join.
> >>> >
> >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> kafka@harsha.io>
> >>> > wrote:
> >>> >
> >>> > > Hi Grant,
> >>> > > We are working on it. Will add the details to KIP about
> the
> >>> > > request protocol.
> >>> > >
> >>> > > Thanks,
> >>> > > Harsha
> >>> > >
> >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com>
> >>> wrote:
> >>> > >
> >>> > > > Hi Parth,
> >>> > > >
> >>> > > > Are you still working on this? If you need any help please don't
> >>> > hesitate
> >>> > > > to ask.
> >>> > > >
> >>> > > > Thanks,
> >>> > > > Grant
> >>> > > >
> >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io>
> wrote:
> >>> > > >
> >>> > > > > Parth,
> >>> > > > >
> >>> > > > > Thanks for the reply.
> >>> > > > >
> >>> > > > > It makes sense to only allow the renewal by users that
> >>> authenticated
> >>> > > > using
> >>> > > > > *non* delegation token mechanism. Then, should we make the
> >>> renewal a
> >>> > > > list?
> >>> > > > > For example, in the case of rest proxy, it will be useful for
> >>> every
> >>> > > > > instance of rest proxy to be able to renew the tokens.
> >>> > > > >
> >>> > > > > It would be clearer if we can document the request protocol
> like
> >>> > > > >
> >>> > > > >
> >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> >>> > > Commandlineandcentralizedadministrativeoperations-
> >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> >>> > > > > .
> >>> > > > >
> >>> > > > > It would also be useful to document the client APIs.
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > >
> >>> > > > > Jun
> >>> > > > >
> >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > >
> >>> > > > > > Hi,
> >>> > > > > >
> >>> > > > > > I am suggesting that we will only allow the renewal by users
> >>> that
> >>> > > > > > authenticated using *non* delegation token mechanism. For
> >>> example,
> >>> > If
> >>> > > > > user
> >>> > > > > > Alice authenticated using kerberos and requested delegation
> >>> tokens,
> >>> > > > only
> >>> > > > > > user Alice authenticated via non delegation token mechanism
> can
> >>> > > renew.
> >>> > > > > > Clients that have access to delegation tokens can not issue
> >>> > renewal
> >>> > > > > > request for renewing their own token and this is primarily
> >>> > important
> >>> > > to
> >>> > > > > > reduce the time window for which a compromised token will be
> >>> valid.
> >>> > > > > >
> >>> > > > > > To clarify, Yes any authenticated user can request delegation
> >>> > tokens
> >>> > > > but
> >>> > > > > > even here I would recommend to avoid creating a chain where a
> >>> > client
> >>> > > > > > authenticated via delegation token request for more
> delegation
> >>> > > tokens.
> >>> > > > > > Basically anyone can request delegation token, as long as
> they
> >>> > > > > authenticate
> >>> > > > > > via a non delegation token mechanism.
> >>> > > > > >
> >>> > > > > > Aren't classes listed here
> >>> > > > > > <
> >>> > > > > >
> >>> > > > >
> >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
> >>> upportforKaf
> >>> > > ka-PublicInterfaces
> >>> > > > > > >
> >>> > > > > > sufficient?
> >>> > > > > >
> >>> > > > > > Thanks
> >>> > > > > > Parth
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io>
> >>> wrote:
> >>> > > > > >
> >>> > > > > > > Parth,
> >>> > > > > > >
> >>> > > > > > > Thanks for the reply. A couple of comments inline below.
> >>> > > > > > >
> >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > >
> >>> > > > > > > > 1. Who / how are tokens renewed? By original requester
> >>> only? or
> >>> > > > using
> >>> > > > > > > > Kerberos
> >>> > > > > > > > auth only?
> >>> > > > > > > > My recommendation is to do this only using Kerberos auth
> and
> >>> > only
> >>> > > > > threw
> >>> > > > > > > the
> >>> > > > > > > > renewer specified during the acquisition request.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > Hmm, not sure that I follow this. Are you saying that any
> >>> client
> >>> > > > > > > authenticated with the delegation token can renew, i.e.
> there
> >>> is
> >>> > no
> >>> > > > > > renewer
> >>> > > > > > > needed?
> >>> > > > > > >
> >>> > > > > > > Also, just to be clear, any authenticated client (either
> >>> through
> >>> > > SASL
> >>> > > > > or
> >>> > > > > > > SSL) can request a delegation token for the authenticated
> >>> user,
> >>> > > > right?
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> >>> > > > > > > > My recommendation is still to store in ZK or not store
> them
> >>> at
> >>> > > all.
> >>> > > > > The
> >>> > > > > > > > whole controller based distribution is too much overhead
> >>> with
> >>> > not
> >>> > > > > much
> >>> > > > > > to
> >>> > > > > > > > achieve.
> >>> > > > > > > >
> >>> > > > > > > > 3. How are tokens invalidated / expired?
> >>> > > > > > > > Either by expiration time out or through an explicit
> >>> request to
> >>> > > > > > > invalidate.
> >>> > > > > > > >
> >>> > > > > > > > 4. Which encryption algorithm is used?
> >>> > > > > > > > SCRAM
> >>> > > > > > > >
> >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the
> KIP
> >>> but
> >>> > > was
> >>> > > > > > > > discussed
> >>> > > > > > > > in this thread)?
> >>> > > > > > > > There is no imperonation proposal. I tried and explained
> how
> >>> > its
> >>> > > a
> >>> > > > > > > > different problem and why its not really necessary to
> >>> discuss
> >>> > > that
> >>> > > > as
> >>> > > > > > > part
> >>> > > > > > > > of this KIP. This KIP will not support any
> impersonation,
> >>> it
> >>> > > will
> >>> > > > > just
> >>> > > > > > > be
> >>> > > > > > > > another way to authenticate.
> >>> > > > > > > >
> >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> >>> > > > > > > > We do not need new ACLs.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > Could we document the format of the new request/response
> and
> >>> > their
> >>> > > > > > > associated Resource and Operation for ACL?
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > > 7. How would the delegation token be configured in the
> >>> client?
> >>> > > > > > > > Should be through config. I wasn't planning on supporting
> >>> JAAS
> >>> > > for
> >>> > > > > > > tokens.
> >>> > > > > > > > I don't believe hadoop does this either.
> >>> > > > > > > >
> >>> > > > > > > > Thanks
> >>> > > > > > > > Parth
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> jun@confluent.io>
> >>> > > wrote:
> >>> > > > > > > >
> >>> > > > > > > > > Harsha,
> >>> > > > > > > > >
> >>> > > > > > > > > Another question.
> >>> > > > > > > > >
> >>> > > > > > > > > 9. How would the delegation token be configured in the
> >>> > client?
> >>> > > > The
> >>> > > > > > > > standard
> >>> > > > > > > > > way is to do this through JAAS. However, we will need
> to
> >>> > think
> >>> > > > > > through
> >>> > > > > > > if
> >>> > > > > > > > > this is convenient in a shared environment. For
> example,
> >>> > when a
> >>> > > > new
> >>> > > > > > > task
> >>> > > > > > > > is
> >>> > > > > > > > > added to a Storm worker node, do we need to dynamically
> >>> add a
> >>> > > new
> >>> > > > > > > section
> >>> > > > > > > > > in the JAAS file? It may be more convenient if we can
> >>> pass in
> >>> > > the
> >>> > > > > > token
> >>> > > > > > > > > through the config directly w/o going through JAAS.
> >>> > > > > > > > >
> >>> > > > > > > > > Are you or Parth still actively working on this KIP?
> >>> > > > > > > > >
> >>> > > > > > > > > Thanks,
> >>> > > > > > > > >
> >>> > > > > > > > > Jun
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> >>> jun@confluent.io>
> >>> > > > wrote:
> >>> > > > > > > > >
> >>> > > > > > > > > > Just to add on that list.
> >>> > > > > > > > > >
> >>> > > > > > > > > > 2. It would be good to document the format of the
> data
> >>> > stored
> >>> > > > in
> >>> > > > > > ZK.
> >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the
> tokens
> >>> > > should
> >>> > > > > be
> >>> > > > > > > > > > propagated through ZK like config/acl/quota, or
> through
> >>> the
> >>> > > > > > > controller.
> >>> > > > > > > > > > Currently, the controller is only designed for
> >>> propagating
> >>> > > > topic
> >>> > > > > > > > > metadata,
> >>> > > > > > > > > > but not other data.
> >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
> >>> > > DIGEST-MD5
> >>> > > > > > since
> >>> > > > > > > > it's
> >>> > > > > > > > > > deprecated?
> >>> > > > > > > > > >
> >>> > > > > > > > > > Also, the images in the wiki seem broken.
> >>> > > > > > > > > >
> >>> > > > > > > > > > Thanks,
> >>> > > > > > > > > >
> >>> > > > > > > > > > Jun
> >>> > > > > > > > > >
> >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> >>> > > > > gwen@confluent.io>
> >>> > > > > > > > > wrote:
> >>> > > > > > > > > >
> >>> > > > > > > > > >> From what I can see, remaining questions are:
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
> requester
> >>> > only?
> >>> > > > or
> >>> > > > > > > using
> >>> > > > > > > > > >> Kerberos auth only?
> >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in
> the
> >>> > KIP
> >>> > > > but
> >>> > > > > > was
> >>> > > > > > > > > >> discussed in this thread)?
> >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> Gwen
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> >>> kafka@harsha.io>
> >>> > > > wrote:
> >>> > > > > > > > > >> > Jun & Ismael,
> >>> > > > > > > > > >> > Unfortunately I couldn't
> >>> attend
> >>> > > the
> >>> > > > > KIP
> >>> > > > > > > > > meeting
> >>> > > > > > > > > >> > when delegation tokens
> >>> > discussed.
> >>> > > > > > > > Appreciate
> >>> > > > > > > > > if
> >>> > > > > > > > > >> > you can update the
> thread if
> >>> > you
> >>> > > > have
> >>> > > > > > any
> >>> > > > > > > > > >> > further questions.
> >>> > > > > > > > > >> > Thanks,
> >>> > > > > > > > > >> > Harsha
> >>> > > > > > > > > >> >
> >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei
> wrote:
> >>> > > > > > > > > >> >> It seems that the links to images in the KIP are
> >>> > broken.
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> Liquan
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> brahmbhatt <
> >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> >>> > > > > > > > > >> >> > In the current proposal we only allow a user to
> >>> get
> >>> > > > > > delegation
> >>> > > > > > > > > token
> >>> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > the identity that it authenticated as using
> >>> another
> >>> > > > > > mechanism,
> >>> > > > > > > > i.e.
> >>> > > > > > > > > >> A user
> >>> > > > > > > > > >> >> > that authenticate using a keytab for principal
> >>> > > > > > > user1@EXAMPLE.COM
> >>> > > > > > > > > >> will get
> >>> > > > > > > > > >> >> > delegation tokens for that user only. In
> future I
> >>> > think
> >>> > > > we
> >>> > > > > > will
> >>> > > > > > > > > have
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > extend support such that we allow some set of
> >>> users (
> >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> >>> > storm-nimbus@EXAMPLE.COM)
> >>> > > > to
> >>> > > > > > > > acquire
> >>> > > > > > > > > >> >> > delegation tokens on behalf of other users
> whose
> >>> > > identity
> >>> > > > > > they
> >>> > > > > > > > have
> >>> > > > > > > > > >> >> > verified independently. Kafka brokers will
> have
> >>> ACLs
> >>> > > to
> >>> > > > > > > control
> >>> > > > > > > > > >> which
> >>> > > > > > > > > >> >> > users are allowed to impersonate other users
> and
> >>> get
> >>> > > > tokens
> >>> > > > > > on
> >>> > > > > > > > > >> behalf of
> >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
> different
> >>> > > problem
> >>> > > > in
> >>> > > > > > my
> >>> > > > > > > > > >> opinion and
> >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
> >>> renewing
> >>> > > > > > delegation
> >>> > > > > > > > > >> tokens?
> >>> > > > > > > > > >> >> > Typically this should be very very low, 1
> request
> >>> per
> >>> > > > > minute
> >>> > > > > > > is a
> >>> > > > > > > > > >> >> > relatively high estimate. However it depends on
> >>> the
> >>> > > token
> >>> > > > > > > > > >> expiration. I am
> >>> > > > > > > > > >> >> > less worried about the extra load it puts on
> >>> > controller
> >>> > > > vs
> >>> > > > > > the
> >>> > > > > > > > > added
> >>> > > > > > > > > >> >> > complexity and the value it offers.
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > Thanks
> >>> > > > > > > > > >> >> > Parth
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> >>> > > > > > > ismael@juma.me.uk>
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
> >>> separate
> >>> > > KIP
> >>> > > > > as
> >>> > > > > > it
> >>> > > > > > > > > will
> >>> > > > > > > > > >> >> > > introduce user visible changes. We could also
> >>> > update
> >>> > > > > KIP-48
> >>> > > > > > > to
> >>> > > > > > > > > >> have this
> >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
> >>> > > separately.
> >>> > > > We
> >>> > > > > > can
> >>> > > > > > > > > >> discuss
> >>> > > > > > > > > >> >> > that
> >>> > > > > > > > > >> >> > > in the KIP call today.
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > Ismael
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini
> Sivaram
> >>> <
> >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > > Ismael,
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > I have created a JIRA (
> >>> > > > > > > > > >> >> > https://issues.apache.org/
> jira/browse/KAFKA-3751)
> >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would
> >>> that
> >>> > > need
> >>> > > > > > > another
> >>> > > > > > > > > >> KIP? If
> >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this
> just
> >>> be
> >>> > a
> >>> > > > JIRA
> >>> > > > > > > that
> >>> > > > > > > > > gets
> >>> > > > > > > > > >> >> > > reviewed
> >>> > > > > > > > > >> >> > > > when the PR is ready?
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > Thank you,
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > Rajini
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael
> Juma <
> >>> > > > > > > > > ismael@juma.me.uk>
> >>> > > > > > > > > >> >> > wrote:
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
> >>> > candidate.
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as
> a
> >>> SASL
> >>> > > > > > mechanism
> >>> > > > > > > > > that
> >>> > > > > > > > > >> might
> >>> > > > > > > > > >> >> > be
> >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning
> to
> >>> > check
> >>> > > > it
> >>> > > > > in
> >>> > > > > > > > more
> >>> > > > > > > > > >> detail.
> >>> > > > > > > > > >> >> > > Good
> >>> > > > > > > > > >> >> > > > > to know that you are willing to
> contribute
> >>> an
> >>> > > > > > > > implementation.
> >>> > > > > > > > > >> Maybe
> >>> > > > > > > > > >> >> > we
> >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > Ismael
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
> >>> > Sivaram <
> >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> >>> > Authentication
> >>> > > > > > > > Mechanism)
> >>> > > > > > > > > >> is a
> >>> > > > > > > > > >> >> > > better
> >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't
> >>> come
> >>> > > > with a
> >>> > > > > > > > > built-in
> >>> > > > > > > > > >> SCRAM
> >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be
> >>> happy
> >>> > > to
> >>> > > > > add
> >>> > > > > > > > > support
> >>> > > > > > > > > >> in
> >>> > > > > > > > > >> >> > Kafka
> >>> > > > > > > > > >> >> > > > > since
> >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to
> support
> >>> > > anyway.
> >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
> >>> > describes
> >>> > > > the
> >>> > > > > > > > protocol
> >>> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun
> Rao <
> >>> > > > > > > > jun@confluent.io
> >>> > > > > > > > > >
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > > > > > > Parth,
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple
> of
> >>> > more
> >>> > > > > > > questions.
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs
> >>> mean?
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of
> getting
> >>> and
> >>> > > > > > renewing
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > tokens?
> >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether
> they
> >>> > > should
> >>> > > > be
> >>> > > > > > > > > directed
> >>> > > > > > > > > >> to the
> >>> > > > > > > > > >> >> > > > > > > controller.
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM,
> parth
> >>> > > > > brahmbhatt <
> >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to
> add
> >>> > acls
> >>> > > > on
> >>> > > > > > who
> >>> > > > > > > > can
> >>> > > > > > > > > >> request
> >>> > > > > > > > > >> >> > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case
> for
> >>> that
> >>> > > yet
> >>> > > > > but
> >>> > > > > > > > down
> >>> > > > > > > > > >> the line
> >>> > > > > > > > > >> >> > > > when
> >>> > > > > > > > > >> >> > > > > we
> >>> > > > > > > > > >> >> > > > > > > > start supporting
> getDelegationTokenAs
> >>> it
> >>> > > will
> >>> > > > > be
> >>> > > > > > > > > >> necessary.
> >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be
> only
> >>> > > > > > > used/distributed
> >>> > > > > > > > > >> over
> >>> > > > > > > > > >> >> > secure
> >>> > > > > > > > > >> >> > > > > > > channels.
> >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end
> up
> >>> > > choosing
> >>> > > > > > > > > >> Invalidation will
> >>> > > > > > > > > >> >> > > be
> >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
> >>> > > controller.
> >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
> >>> somewhere
> >>> > > > that
> >>> > > > > > > > > >> invalidation
> >>> > > > > > > > > >> >> > will
> >>> > > > > > > > > >> >> > > > > > directly
> >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is
> not
> >>> the
> >>> > > > > intent.
> >>> > > > > > > > > >> Invalidation
> >>> > > > > > > > > >> >> > > will
> >>> > > > > > > > > >> >> > > > > > either
> >>> > > > > > > > > >> >> > > > > > > > be request based or due to
> >>> expiration. No
> >>> > > > > direct
> >>> > > > > > > > > >> zookeeper
> >>> > > > > > > > > >> >> > > > > interaction
> >>> > > > > > > > > >> >> > > > > > > from
> >>> > > > > > > > > >> >> > > > > > > > any client.
> >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> >>> DelegationToken
> >>> > > > > without
> >>> > > > > > > the
> >>> > > > > > > > > >> hmac in
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
> >>> confusion.
> >>> > > The
> >>> > > > > sole
> >>> > > > > > > > > >> purpose of
> >>> > > > > > > > > >> >> > > > > zookeeper
> >>> > > > > > > > > >> >> > > > > > in
> >>> > > > > > > > > >> >> > > > > > > > this design is as distribution
> channel
> >>> > for
> >>> > > > > tokens
> >>> > > > > > > > > >> between all
> >>> > > > > > > > > >> >> > > > brokers
> >>> > > > > > > > > >> >> > > > > > > and a
> >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that
> >>> were
> >>> > > > > > generated
> >>> > > > > > > by
> >>> > > > > > > > > >> making a
> >>> > > > > > > > > >> >> > > > > request
> >>> > > > > > > > > >> >> > > > > > > to a
> >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on
> this
> >>> in
> >>> > > > second
> >>> > > > > > > > > >> paragraph). The
> >>> > > > > > > > > >> >> > > > token
> >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
> >>> renewer,
> >>> > > > uuid
> >>> > > > > ,
> >>> > > > > > > > > >> expiration,
> >>> > > > > > > > > >> >> > > hmac)
> >>> > > > > > > > > >> >> > > > ,
> >>> > > > > > > > > >> >> > > > > > one
> >>> > > > > > > > > >> >> > > > > > > of
> >>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac
> >>> but
> >>> > > hmac
> >>> > > > it
> >>> > > > > > > self
> >>> > > > > > > > is
> >>> > > > > > > > > >> >> > derivable
> >>> > > > > > > > > >> >> > > > if
> >>> > > > > > > > > >> >> > > > > > you
> >>> > > > > > > > > >> >> > > > > > > > have all the other elements of the
> >>> token
> >>> > +
> >>> > > > > secret
> >>> > > > > > > key
> >>> > > > > > > > > to
> >>> > > > > > > > > >> >> > generate
> >>> > > > > > > > > >> >> > > > > hmac.
> >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide
> SSL
> >>> > > support
> >>> > > > we
> >>> > > > > > do
> >>> > > > > > > > not
> >>> > > > > > > > > >> want the
> >>> > > > > > > > > >> >> > > > > entire
> >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
> >>> zookeeper
> >>> > > as
> >>> > > > > that
> >>> > > > > > > > will
> >>> > > > > > > > > >> be an
> >>> > > > > > > > > >> >> > > > insecure
> >>> > > > > > > > > >> >> > > > > > > wire
> >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all
> >>> the
> >>> > > other
> >>> > > > > > > > elements
> >>> > > > > > > > > >> of a
> >>> > > > > > > > > >> >> > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
> >>> elements
> >>> > and
> >>> > > > > > because
> >>> > > > > > > > > they
> >>> > > > > > > > > >> also
> >>> > > > > > > > > >> >> > > have
> >>> > > > > > > > > >> >> > > > > > access
> >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to
> >>> > generate
> >>> > > > > hmac
> >>> > > > > > on
> >>> > > > > > > > > >> their end.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is
> to
> >>> > avoid
> >>> > > > > > > zookeeper
> >>> > > > > > > > > >> >> > > altogether. A
> >>> > > > > > > > > >> >> > > > > > > Client
> >>> > > > > > > > > >> >> > > > > > > > will call broker with required
> >>> > information
> >>> > > > > > (owner,
> >>> > > > > > > > > >> renwer,
> >>> > > > > > > > > >> >> > > > > expiration)
> >>> > > > > > > > > >> >> > > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid).
> Broker
> >>> > won't
> >>> > > > > store
> >>> > > > > > > this
> >>> > > > > > > > > in
> >>> > > > > > > > > >> >> > > zookeeper.
> >>> > > > > > > > > >> >> > > > > > From
> >>> > > > > > > > > >> >> > > > > > > > this point a client can contact any
> >>> > broker
> >>> > > > with
> >>> > > > > > all
> >>> > > > > > > > the
> >>> > > > > > > > > >> >> > > delegation
> >>> > > > > > > > > >> >> > > > > > token
> >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration,
> hmac,
> >>> > > uuid)
> >>> > > > > the
> >>> > > > > > > > borker
> >>> > > > > > > > > >> will
> >>> > > > > > > > > >> >> > > > > regenerate
> >>> > > > > > > > > >> >> > > > > > > the
> >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with
> >>> hmac
> >>> > > > > > presented
> >>> > > > > > > by
> >>> > > > > > > > > >> client ,
> >>> > > > > > > > > >> >> > > > broker
> >>> > > > > > > > > >> >> > > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate.
> >>> Only
> >>> > > > > problem
> >>> > > > > > > with
> >>> > > > > > > > > >> this
> >>> > > > > > > > > >> >> > > approach
> >>> > > > > > > > > >> >> > > > > is
> >>> > > > > > > > > >> >> > > > > > if
> >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
> >>> client
> >>> > > can
> >>> > > > > now
> >>> > > > > > > > > generate
> >>> > > > > > > > > >> >> > random
> >>> > > > > > > > > >> >> > > > > tokens
> >>> > > > > > > > > >> >> > > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > they will still be able to
> >>> authenticate
> >>> > as
> >>> > > > any
> >>> > > > > > user
> >>> > > > > > > > > they
> >>> > > > > > > > > >> like.
> >>> > > > > > > > > >> >> > > with
> >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
> >>> tokens
> >>> > > > > acquired
> >>> > > > > > > via
> >>> > > > > > > > a
> >>> > > > > > > > > >> broker
> >>> > > > > > > > > >> >> > > > (using
> >>> > > > > > > > > >> >> > > > > > some
> >>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
> >>> token)
> >>> > > will
> >>> > > > > be
> >>> > > > > > > > > >> accepted. We
> >>> > > > > > > > > >> >> > > need
> >>> > > > > > > > > >> >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more
> >>> sense
> >>> > and
> >>> > > > we
> >>> > > > > > can
> >>> > > > > > > go
> >>> > > > > > > > > >> over it
> >>> > > > > > > > > >> >> > in
> >>> > > > > > > > > >> >> > > > > > > tomorrow's
> >>> > > > > > > > > >> >> > > > > > > > meeting.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to
> >>> me?
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Thanks
> >>> > > > > > > > > >> >> > > > > > > > Parth
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM,
> Jun
> >>> > Rao <
> >>> > > > > > > > > >> jun@confluent.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
> comments.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be
> useful
> >>> for
> >>> > > > Kafka
> >>> > > > > > > > Connect
> >>> > > > > > > > > >> and
> >>> > > > > > > > > >> >> > Kafka
> >>> > > > > > > > > >> >> > > > > rest
> >>> > > > > > > > > >> >> > > > > > > > proxy
> >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to
> >>> run a
> >>> > > > task
> >>> > > > > on
> >>> > > > > > > > > behalf
> >>> > > > > > > > > >> of a
> >>> > > > > > > > > >> >> > > > client.
> >>> > > > > > > > > >> >> > > > > > We
> >>> > > > > > > > > >> >> > > > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > > likely need to change how those
> >>> > services
> >>> > > > use
> >>> > > > > > > Kafka
> >>> > > > > > > > > >> clients
> >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
> >>> > shared
> >>> > > > > client
> >>> > > > > > > per
> >>> > > > > > > > > >> worker,
> >>> > > > > > > > > >> >> > we
> >>> > > > > > > > > >> >> > > > will
> >>> > > > > > > > > >> >> > > > > > > need
> >>> > > > > > > > > >> >> > > > > > > > a
> >>> > > > > > > > > >> >> > > > > > > > > client per user task since the
> >>> > > > authentication
> >>> > > > > > > > happens
> >>> > > > > > > > > >> at the
> >>> > > > > > > > > >> >> > > > > > connection
> >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
> >>> renewer
> >>> > > will
> >>> > > > be
> >>> > > > > > the
> >>> > > > > > > > > >> workers.
> >>> > > > > > > > > >> >> > So,
> >>> > > > > > > > > >> >> > > we
> >>> > > > > > > > > >> >> > > > > > > > probably
> >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers.
> For
> >>> > > Kafka
> >>> > > > > rest
> >>> > > > > > > > > proxy,
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > renewer
> >>> > > > > > > > > >> >> > > > > > can
> >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of
> the
> >>> > > token.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who
> can
> >>> > > request
> >>> > > > > > > > delegation
> >>> > > > > > > > > >> tokens?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to
> send
> >>> > > > > delegation
> >>> > > > > > > > tokens
> >>> > > > > > > > > >> in an
> >>> > > > > > > > > >> >> > > > > encrypted
> >>> > > > > > > > > >> >> > > > > > > > > channel?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for
> expiring
> >>> > > > tokens,
> >>> > > > > > > every
> >>> > > > > > > > > >> broker?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens,
> would
> >>> it
> >>> > be
> >>> > > > > > better
> >>> > > > > > > to
> >>> > > > > > > > > do
> >>> > > > > > > > > >> it in
> >>> > > > > > > > > >> >> > a
> >>> > > > > > > > > >> >> > > > > > request
> >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in
> >>> the
> >>> > > wiki
> >>> > > > > > > > sometimes
> >>> > > > > > > > > >> refers
> >>> > > > > > > > > >> >> > to
> >>> > > > > > > > > >> >> > > > the
> >>> > > > > > > > > >> >> > > > > > end
> >>> > > > > > > > > >> >> > > > > > > > > client and some other times
> refers
> >>> to
> >>> > the
> >>> > > > > > client
> >>> > > > > > > > > using
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
> >>> > distinguish
> >>> > > > > > between
> >>> > > > > > > > the
> >>> > > > > > > > > >> two.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
> sentence
> >>> > > "Broker
> >>> > > > > > also
> >>> > > > > > > > > >> stores the
> >>> > > > > > > > > >> >> > > > > > > > DelegationToken
> >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
> zookeeper."
> >>> a
> >>> > bit
> >>> > > > > > more? I
> >>> > > > > > > > > >> thought the
> >>> > > > > > > > > >> >> > > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM,
> Jun
> >>> > Rao
> >>> > > <
> >>> > > > > > > > > >> jun@confluent.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
> >>> invite.
> >>> > We
> >>> > > > can
> >>> > > > > > > > discuss
> >>> > > > > > > > > >> this in
> >>> > > > > > > > > >> >> > > the
> >>> > > > > > > > > >> >> > > > > > > meeting
> >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47
> AM,
> >>> > > Harsha <
> >>> > > > > > > > > >> kafka@harsha.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
> >>> meeting
> >>> > > > > around
> >>> > > > > > > > this.
> >>> > > > > > > > > >> The KIP
> >>> > > > > > > > > >> >> > is
> >>> > > > > > > > > >> >> > > > up
> >>> > > > > > > > > >> >> > > > > > for
> >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if
> there
> >>> are
> >>> > > any
> >>> > > > > > > > questions
> >>> > > > > > > > > >> lets
> >>> > > > > > > > > >> >> > > > quickly
> >>> > > > > > > > > >> >> > > > > > hash
> >>> > > > > > > > > >> >> > > > > > > > out
> >>> > > > > > > > > >> >> > > > > > > > > >> details.
> >>> > > > > > > > > >> >> > > > > > > > > >>
> >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
> >>> > > > > > > > > >> >> > > > > > > > > >>
> >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40
> >>> AM,
> >>> > > parth
> >>> > > > > > > > > brahmbhatt
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
> >>> > system
> >>> > > > uses
> >>> > > > > > so
> >>> > > > > > > no
> >>> > > > > > > > > >> good
> >>> > > > > > > > > >> >> > reason
> >>> > > > > > > > > >> >> > > > > > really.
> >>> > > > > > > > > >> >> > > > > > > > We
> >>> > > > > > > > > >> >> > > > > > > > > >> > could
> >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
> >>> > newest
> >>> > > > > > > > recommended
> >>> > > > > > > > > >> standard
> >>> > > > > > > > > >> >> > > is.
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33
> >>> AM,
> >>> > > > Ismael
> >>> > > > > > > Juma <
> >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
> >>> > started
> >>> > > > > > > reviewing
> >>> > > > > > > > > >> this and
> >>> > > > > > > > > >> >> > > may
> >>> > > > > > > > > >> >> > > > > have
> >>> > > > > > > > > >> >> > > > > > > > > >> additional
> >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
> >>> immediate
> >>> > > > > question
> >>> > > > > > > that
> >>> > > > > > > > > >> came to
> >>> > > > > > > > > >> >> > > mind
> >>> > > > > > > > > >> >> > > > is
> >>> > > > > > > > > >> >> > > > > > our
> >>> > > > > > > > > >> >> > > > > > > > > >> choice of
> >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though
> it's
> >>> > > marked
> >>> > > > > as
> >>> > > > > > > > > >> OBSOLETE in
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > IANA
> >>> > > > > > > > > >> >> > > > > > > > > Registry
> >>> > > > > > > > > >> >> > > > > > > > > >> of
> >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
> >>> original
> >>> > > RFC
> >>> > > > > > > (2831)
> >>> > > > > > > > > has
> >>> > > > > > > > > >> been
> >>> > > > > > > > > >> >> > > moved
> >>> > > > > > > > > >> >> > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > Historic
> >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> https://tools.ietf.org/html/
> >>> > > rfc6331
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >>
> >>> > > > > > >
> >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
> >>> mechanisms.xhtml
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning
> behind
> >>> > that
> >>> > > > > > choice?
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at
> 11:29
> >>> > PM,
> >>> > > > Gwen
> >>> > > > > > > > > Shapira <
> >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize
> >>> that
> >>> > > even
> >>> > > > > > though
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > tokens
> >>> > > > > > > > > >> >> > > > > > > are a
> >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel
> very
> >>> > > strongly
> >>> > > > > > about
> >>> > > > > > > > not
> >>> > > > > > > > > >> adding
> >>> > > > > > > > > >> >> > > > > > dependency
> >>> > > > > > > > > >> >> > > > > > > > on
> >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
> >>> delegation
> >>> > > > > tokens
> >>> > > > > > > for
> >>> > > > > > > > > >> Kafka. The
> >>> > > > > > > > > >> >> > > KIP
> >>> > > > > > > > > >> >> > > > > > > doesn't
> >>> > > > > > > > > >> >> > > > > > > > > >> imply
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but
> if
> >>> you
> >>> > > can
> >>> > > > > > > > clarify...
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
> dependency.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to
> the
> >>> KIP
> >>> > so
> >>> > > > no
> >>> > > > > > one
> >>> > > > > > > > will
> >>> > > > > > > > > >> read
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > KIP
> >>> > > > > > > > > >> >> > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > > panic
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the
> next
> >>> > > > > release...
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
> delegation
> >>> > token
> >>> > > at
> >>> > > > > any
> >>> > > > > > > > time
> >>> > > > > > > > > >> after
> >>> > > > > > > > > >> >> > > > > > > > authenticating?
> >>> > > > > > > > > >> >> > > > > > > > > >> only
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> >>> > > > authenticated
> >>> > > > > > you
> >>> > > > > > > > can
> >>> > > > > > > > > >> get
> >>> > > > > > > > > >> >> > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens.
> >>> > > > > > > > > >> >> > > > > > > > > >> We
> >>> > > > > > > > > >> >> > > > > > > > > >> > > need
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> >>> > > > authenticated
> >>> > > > > > > using
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > > token,
> >>> > > > > > > > > >> >> > > > > > > can
> >>> > > > > > > > > >> >> > > > > > > > > also
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token
> again or
> >>> > not.
> >>> > > > > Also
> >>> > > > > > > > there
> >>> > > > > > > > > >> is the
> >>> > > > > > > > > >> >> > > > > question
> >>> > > > > > > > > >> >> > > > > > of
> >>> > > > > > > > > >> >> > > > > > > > do
> >>> > > > > > > > > >> >> > > > > > > > > we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
> >>> delegation
> >>> > > > token
> >>> > > > > > or
> >>> > > > > > > we
> >>> > > > > > > > > >> want
> >>> > > > > > > > > >> >> > > specific
> >>> > > > > > > > > >> >> > > > > > ACLs
> >>> > > > > > > > > >> >> > > > > > > (I
> >>> > > > > > > > > >> >> > > > > > > > > >> think
> >>> > > > > > > > > >> >> > > > > > > > > >> > > its
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > an
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
> >>> > > overkill.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating
> two
> >>> > > > options:
> >>> > > > > > > > Either
> >>> > > > > > > > > >> require
> >>> > > > > > > > > >> >> > > > > Kerberos
> >>> > > > > > > > > >> >> > > > > > > > auth
> >>> > > > > > > > > >> >> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require
> >>> non-owners
> >>> > to
> >>> > > > > > renew.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
> >>> > simpler
> >>> > > > (it
> >>> > > > > > > > > basically
> >>> > > > > > > > > >> >> > require
> >>> > > > > > > > > >> >> > > a
> >>> > > > > > > > > >> >> > > > > "job
> >>> > > > > > > > > >> >> > > > > > > > > master"
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility
> for
> >>> the
> >>> > > > > > renewal,
> >>> > > > > > > it
> >>> > > > > > > > > >> will have
> >>> > > > > > > > > >> >> > > its
> >>> > > > > > > > > >> >> > > > > own
> >>> > > > > > > > > >> >> > > > > > > > > >> identity
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this
> is
> >>> the
> >>> > > > > correct
> >>> > > > > > > > > design
> >>> > > > > > > > > >> >> > pattern
> >>> > > > > > > > > >> >> > > > > > anyway.
> >>> > > > > > > > > >> >> > > > > > > > For
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect
> Nimbus to
> >>> > > > > coordinate
> >>> > > > > > > > > >> renewals?),
> >>> > > > > > > > > >> >> > but
> >>> > > > > > > > > >> >> > > > it
> >>> > > > > > > > > >> >> > > > > is
> >>> > > > > > > > > >> >> > > > > > > > hard
> >>> > > > > > > > > >> >> > > > > > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity
> without
> >>> > > looking
> >>> > > > at
> >>> > > > > > the
> >>> > > > > > > > > code
> >>> > > > > > > > > >> >> > changes
> >>> > > > > > > > > >> >> > > > > > > required.
> >>> > > > > > > > > >> >> > > > > > > > If
> >>> > > > > > > > > >> >> > > > > > > > > >> you
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
> >>> > "require
> >>> > > > > > > Kerberos"
> >>> > > > > > > > > >> will look
> >>> > > > > > > > > >> >> > > in
> >>> > > > > > > > > >> >> > > > > > Kafka
> >>> > > > > > > > > >> >> > > > > > > > > code,
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a
> >>> look.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is
> >>> that
> >>> > > > tokens
> >>> > > > > > will
> >>> > > > > > > > > >> propagate
> >>> > > > > > > > > >> >> > via
> >>> > > > > > > > > >> >> > > > ZK
> >>> > > > > > > > > >> >> > > > > > but
> >>> > > > > > > > > >> >> > > > > > > > > >> without
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> >>> > > > > UpdateMetadata
> >>> > > > > > > > > >> protocol,
> >>> > > > > > > > > >> >> > > > correct?
> >>> > > > > > > > > >> >> > > > > > > > Clients
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry
> on
> >>> > SASL
> >>> > > > auth
> >>> > > > > > > > failure
> >>> > > > > > > > > >> (IIRC),
> >>> > > > > > > > > >> >> > > but
> >>> > > > > > > > > >> >> > > > > > since
> >>> > > > > > > > > >> >> > > > > > > > the
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate
> between
> >>> > > brokers
> >>> > > > > > > asynch,
> >>> > > > > > > > > we
> >>> > > > > > > > > >> will
> >>> > > > > > > > > >> >> > > need
> >>> > > > > > > > > >> >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > retry a
> >>> > > > > > > > > >> >> > > > > > > > > >> bit
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients
> failing
> >>> > auth
> >>> > > > due
> >>> > > > > to
> >>> > > > > > > > > timing
> >>> > > > > > > > > >> >> > issues.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
> >>> > > alternatives
> >>> > > > > > right
> >>> > > > > > > > > now.
> >>> > > > > > > > > >> The
> >>> > > > > > > > > >> >> > > > current
> >>> > > > > > > > > >> >> > > > > > > > > documented
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based
> and it
> >>> > does
> >>> > > > not
> >>> > > > > > > > require
> >>> > > > > > > > > >> any
> >>> > > > > > > > > >> >> > > changes
> >>> > > > > > > > > >> >> > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An
> alternative
> >>> > > > approach
> >>> > > > > > can
> >>> > > > > > > > > remove
> >>> > > > > > > > > >> >> > > zookeeper
> >>> > > > > > > > > >> >> > > > > > > > > dependency
> >>> > > > > > > > > >> >> > > > > > > > > >> as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > well
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss
> that
> >>> in
> >>> > KIP
> >>> > > > > > > > discussion
> >>> > > > > > > > > >> call.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds
> interesting.
> >>> Do
> >>> > you
> >>> > > > > want
> >>> > > > > > to
> >>> > > > > > > > > ping
> >>> > > > > > > > > >> Jun to
> >>> > > > > > > > > >> >> > > > > > arrange a
> >>> > > > > > > > > >> >> > > > > > > > > call?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
> >>> > suggestion
> >>> > > of
> >>> > > > > > > having
> >>> > > > > > > > > >> just the
> >>> > > > > > > > > >> >> > > > > > controller
> >>> > > > > > > > > >> >> > > > > > > > > issue
> >>> > > > > > > > > >> >> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to
> >>> avoid
> >>> > > > > syncing
> >>> > > > > > a
> >>> > > > > > > > > shared
> >>> > > > > > > > > >> >> > secret.
> >>> > > > > > > > > >> >> > > > Not
> >>> > > > > > > > > >> >> > > > > > > sure
> >>> > > > > > > > > >> >> > > > > > > > if
> >>> > > > > > > > > >> >> > > > > > > > > >> we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
> >>> > > discussion
> >>> > > > > here
> >>> > > > > > > or
> >>> > > > > > > > on
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > wiki. I
> >>> > > > > > > > > >> >> > > > > > think
> >>> > > > > > > > > >> >> > > > > > > > > that
> >>> > > > > > > > > >> >> > > > > > > > > >> we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the
> problem
> >>> of
> >>> > > > "token
> >>> > > > > > > > > >> distribution"
> >>> > > > > > > > > >> >> > > from
> >>> > > > > > > > > >> >> > > > > > > "shared
> >>> > > > > > > > > >> >> > > > > > > > > >> secret
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use
> the
> >>> > > > > controller
> >>> > > > > > > as
> >>> > > > > > > > > the
> >>> > > > > > > > > >> only
> >>> > > > > > > > > >> >> > > token
> >>> > > > > > > > > >> >> > > > > > > > generator
> >>> > > > > > > > > >> >> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second
> issue,
> >>> > while
> >>> > > > > still
> >>> > > > > > > > using
> >>> > > > > > > > > >> ZK async
> >>> > > > > > > > > >> >> > > to
> >>> > > > > > > > > >> >> > > > > > > > distribute
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
> >>> > previous
> >>> > > > > Email
> >>> > > > > > I
> >>> > > > > > > am
> >>> > > > > > > > > >> fine with
> >>> > > > > > > > > >> >> > > > that
> >>> > > > > > > > > >> >> > > > > > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > long
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the
> extra
> >>> > > > complexity
> >>> > > > > > of
> >>> > > > > > > > > >> >> > > adding/updating
> >>> > > > > > > > > >> >> > > > > APIS
> >>> > > > > > > > > >> >> > > > > > > > adds
> >>> > > > > > > > > >> >> > > > > > > > > >> enough
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage
> with
> >>> > the
> >>> > > > > > > controller
> >>> > > > > > > > > >> approach
> >>> > > > > > > > > >> >> > is
> >>> > > > > > > > > >> >> > > > > > secret
> >>> > > > > > > > > >> >> > > > > > > > > >> rotation
> >>> > > > > > > > > >> >> > > > > > > > > >> > > can
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > be
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and
> >>> would
> >>> > > not
> >>> > > > > > > require
> >>> > > > > > > > > >> >> > > deployment. *
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> >>> > > > complexity
> >>> > > > > > (or
> >>> > > > > > > > > point
> >>> > > > > > > > > >> me to
> >>> > > > > > > > > >> >> > > the
> >>> > > > > > > > > >> >> > > > > > email
> >>> > > > > > > > > >> >> > > > > > > I
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis
> are
> >>> > > > required?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell,
> >>> clients
> >>> > > can
> >>> > > > > > > already
> >>> > > > > > > > > >> find the
> >>> > > > > > > > > >> >> > > > > > controller
> >>> > > > > > > > > >> >> > > > > > > > from
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
> >>> > > concerned
> >>> > > > > > about
> >>> > > > > > > > > >> controller
> >>> > > > > > > > > >> >> > > > load.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the
> idea
> >>> of
> >>> > > > forcing
> >>> > > > > > > > > kerberos
> >>> > > > > > > > > >> auth
> >>> > > > > > > > > >> >> > for
> >>> > > > > > > > > >> >> > > > > > > renwal, I
> >>> > > > > > > > > >> >> > > > > > > > > >> think
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport
> >>> layer
> >>> > > the
> >>> > > > > the
> >>> > > > > > > > > request
> >>> > > > > > > > > >> >> > content
> >>> > > > > > > > > >> >> > > > in
> >>> > > > > > > > > >> >> > > > > a
> >>> > > > > > > > > >> >> > > > > > > > pretty
> >>> > > > > > > > > >> >> > > > > > > > > >> ugly
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
> >>> > renewer
> >>> > > to
> >>> > > > > > > > non-owner
> >>> > > > > > > > > >> is
> >>> > > > > > > > > >> >> > > better.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a
> >>> necessary
> >>> > > > evil.
> >>> > > > > > > While
> >>> > > > > > > > > >> this will
> >>> > > > > > > > > >> >> > > make
> >>> > > > > > > > > >> >> > > > > the
> >>> > > > > > > > > >> >> > > > > > > > kafka
> >>> > > > > > > > > >> >> > > > > > > > > >> code
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight
> forward ,
> >>> > > > forcing
> >>> > > > > > > > renewer
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > non-owner
> >>> > > > > > > > > >> >> > > > > > > pushes
> >>> > > > > > > > > >> >> > > > > > > > > >> the code
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and
> >>> makes
> >>> > > it
> >>> > > > > even
> >>> > > > > > > > > harder
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > > > integrate. *
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I
> >>> don't
> >>> > > think
> >>> > > > > the
> >>> > > > > > > > > >> "renewal by
> >>> > > > > > > > > >> >> > > > other"
> >>> > > > > > > > > >> >> > > > > > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> is
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the
> clients
> >>> we
> >>> > > > expect
> >>> > > > > to
> >>> > > > > > > use
> >>> > > > > > > > > >> >> > delegation
> >>> > > > > > > > > >> >> > > > > tokens
> >>> > > > > > > > > >> >> > > > > > > > since
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an
> >>> app-master
> >>> > of
> >>> > > > some
> >>> > > > > > > sort
> >>> > > > > > > > > who
> >>> > > > > > > > > >> >> > > requested
> >>> > > > > > > > > >> >> > > > > the
> >>> > > > > > > > > >> >> > > > > > > > token
> >>> > > > > > > > > >> >> > > > > > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my
> >>> question
> >>> > > on
> >>> > > > > how
> >>> > > > > > > > > multiple
> >>> > > > > > > > > >> >> > > > identities
> >>> > > > > > > > > >> >> > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > be
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super
> >>> clear
> >>> > to
> >>> > > > me -
> >>> > > > > > > > AFAIK,
> >>> > > > > > > > > >> we don't
> >>> > > > > > > > > >> >> > > > > > > > authenticate
> >>> > > > > > > > > >> >> > > > > > > > > >> each
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we
> authenticate
> >>> > > > > > connections.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
> >>> > connections,
> >>> > > > and
> >>> > > > > > only
> >>> > > > > > > > > when
> >>> > > > > > > > > >> they
> >>> > > > > > > > > >> >> > are
> >>> > > > > > > > > >> >> > > > > being
> >>> > > > > > > > > >> >> > > > > > > > > >> established.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Let
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this
> as
> >>> a
> >>> > > > > question,
> >>> > > > > > > in
> >>> > > > > > > > > >> absence of
> >>> > > > > > > > > >> >> > > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > >> tokens if
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use
> >>> case
> >>> > > > using
> >>> > > > > > user
> >>> > > > > > > > > >> TGT's how
> >>> > > > > > > > > >> >> > > would
> >>> > > > > > > > > >> >> > > > > we
> >>> > > > > > > > > >> >> > > > > > do
> >>> > > > > > > > > >> >> > > > > > > > it?
> >>> > > > > > > > > >> >> > > > > > > > > >> My
> >>> > > > > > > > > >> >> > > > > > > > > >> > > point
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no
> >>> different
> >>> > > > with
> >>> > > > > > > > > delegation
> >>> > > > > > > > > >> >> > tokens.
> >>> > > > > > > > > >> >> > > > The
> >>> > > > > > > > > >> >> > > > > > use
> >>> > > > > > > > > >> >> > > > > > > > > case
> >>> > > > > > > > > >> >> > > > > > > > > >> you
> >>> > > > > > > > > >> >> > > > > > > > > >> > > are
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more
> >>> like
> >>> > > > > > > > impersonation.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that
> one of
> >>> > the
> >>> > > > > things
> >>> > > > > > > > that
> >>> > > > > > > > > >> >> > del
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Regards,
> >> Ashish
> >>
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Thanks for the KIP.
Request-level impersonation is going to make the client implementation more
complicated. On the producer side, we batch messages per partition. Now, do
we have to batch per partition, per user? The consumer client is designed
to be single-threaded. So, I am not sure if request-level impersonation
makes sense there. The main benefit of request-level impersonation seems to
be saving the number of socket connections to the broker. However, the
broker is capable of handling lots of connections.
Jun
On Tue, Dec 20, 2016 at 8:42 AM, Manikumar <ma...@gmail.com>
wrote:
> Hi Devs,
>
> If there are no more comments, I will start vote on this KIP later this
> week.
>
>
> Thanks
>
> On Fri, Dec 16, 2016 at 12:28 PM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> >
> >> Can you add a sample Jaas configuration using delegation tokens to the
> >> KIP?
> >>
> >
> > Will add sample Jaas configuration.
> >
> >
> >> To make sure I have understood correctly, KAFKA-3712 is aimed at
> enabling
> >> a
> >> superuser to impersonate another (single) user, say alice. A producer
> >> using
> >> impersonation will authenticate with superuser credentials. All requests
> >> from the producer will be run with the principal alice. But alice is not
> >> involved in the authentication and alice's credentials are not actually
> >> provided to the broker?
> >>
> >>
> > Yes, this matches with my understanding of impersonation work . Even in
> > this approach
> > we have to handle all producer related issues mentioned by you. Yes,
> this
> > is big change
> > and can be implemented if there is a pressing need. I hope we are all in
> > agreement, that
> > this can be done in a separate KIP.
> >
> >
> > I request others give any suggestions/concerns on this KIP.
> >
> >
> > Thanks,
> >
> >
> >
> >>
> >> On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <ma...@gmail.com>
> >> wrote:
> >>
> >> > @Gwen, @Rajini,
> >> >
> >> > As mentioned in the KIP, main motivation for this KIP is to reduce
> load
> >> on
> >> > Kerberos
> >> > server on large kafka deployments with large number of clients.
> >> >
> >> > Also it looks like we are combining two overlapping concepts
> >> > 1. Single client sending requests with multiple users/authentications
> >> > 2. Impersonation
> >> >
> >> > Option 1, is definitely useful in some use cases and can be used to
> >> > implement workaround for
> >> > impersonation
> >> >
> >> > In Impersonation, a super user can send requests on behalf of another
> >> > user(Alice) in a secured way.
> >> > superuser has credentials but user Alice doesn't have any. The
> requests
> >> are
> >> > required
> >> > to run as user Alice and accesses/ACLs on Broker are required to be
> >> done as
> >> > user Alice.
> >> > It is required that user Alice can connect to the Broker on a
> connection
> >> > authenticated with
> >> > superuser's credentials. In other words superuser is impersonating the
> >> user
> >> > Alice.
> >> >
> >> > The approach mentioned by Harsha in previous mail is implemented in
> >> hadoop,
> >> > storm etc..
> >> >
> >> > Some more details here:
> >> > https://hadoop.apache.org/docs/r2.7.2/hadoop-project-
> >> > dist/hadoop-common/Superusers.html
> >> >
> >> >
> >> > @Rajini
> >> >
> >> > Thanks for your comments on SASL/SCRAM usage. I am thinking to send
> >> > tokenHmac (salted-hashed version)
> >> > as password for authentication and tokenID for retrial of tokenHmac at
> >> > server side.
> >> > Does above sound OK?
> >> >
> >> >
> >> > Thanks,
> >> > Manikumar
> >> >
> >> > On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <
> kafka@harsha.io>
> >> > wrote:
> >> >
> >> > > @Gwen @Mani Not sure why we want to authenticate at every request.
> >> Even
> >> > if
> >> > > the token exchange is cheap it still a few calls that need to go
> >> through
> >> > > round trip. Impersonation doesn't require authentication for every
> >> > > request.
> >> > >
> >> > > "So a centralized app can create few producers, do the metadata
> >> request
> >> > and
> >> > > broker discovery with its own user auth, but then use delegation
> >> tokens
> >> > to
> >> > > allow performing produce/fetch requests as different users? Instead
> of
> >> > > having to re-connect for each impersonated user?"
> >> > >
> >> > > Yes. But what we will have is this centralized user as impersonation
> >> user
> >> > > on behalf of other users. When it authenticates initially we will
> >> create
> >> > a
> >> > > "Subject" and from there on wards centralized user can do
> >> > > Subject.doAsPrivileged
> >> > > on behalf, other users.
> >> > > On the server side, we can retrieve two principals out of this one
> is
> >> the
> >> > > authenticated user (centralized user) and another is impersonated
> >> user.
> >> > We
> >> > > will first check if the authenticated user allowed to impersonate
> and
> >> > then
> >> > > move on to check if the user Alice has access to the topic "X" to
> >> > > read/write.
> >> > >
> >> > > @Rajini Intention of this KIP is to support token auth via
> SASL/SCRAM,
> >> > not
> >> > > just with TLS. What you raised is a good point let me take a look
> and
> >> > add
> >> > > details.
> >> > >
> >> > > It will be easier to add impersonation once we reach agreement on
> this
> >> > KIP.
> >> > >
> >> > >
> >> > > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk>
> >> wrote:
> >> > >
> >> > > > Hi Rajini,
> >> > > >
> >> > > > I think it would definitely be valuable to have a KIP for
> >> > impersonation.
> >> > > >
> >> > > > Ismael
> >> > > >
> >> > > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <
> >> rsivaram@pivotal.io>
> >> > > > wrote:
> >> > > >
> >> > > > > It would clearly be very useful to enable clients to send
> >> requests on
> >> > > > > behalf of multiple users. A separate KIP makes sense, but it may
> >> be
> >> > > worth
> >> > > > > thinking through some of the implications now, especially if the
> >> main
> >> > > > > interest in delegation tokens comes from its potential to enable
> >> > > > > impersonation.
> >> > > > >
> >> > > > > I understand that delegation tokens are only expected to be used
> >> with
> >> > > > TLS.
> >> > > > > But the choice of SASL/SCRAM for authentication must be based
> on a
> >> > > > > requirement to protect the tokenHmac - otherwise you could just
> >> use
> >> > > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> >> > > > on-the-wire,
> >> > > > > only a salted-hashed version of it is used in the SASL
> >> authentication
> >> > > > > exchange. If impersonation is based on sending tokenHmac in
> >> requests,
> >> > > any
> >> > > > > benefit of using SCRAM is lost.
> >> > > > >
> >> > > > > An alternative may be to allow clients to authenticate multiple
> >> times
> >> > > > using
> >> > > > > SASL and include one of its authenticated principals in each
> >> request
> >> > > > > (optionally). I haven't thought it through yet, obviously. But
> if
> >> the
> >> > > > > approach is of interest and no one is working on a KIP for
> >> > > impersonation
> >> > > > at
> >> > > > > the moment, I am happy to write one. It may provide something
> for
> >> > > > > comparison at least.
> >> > > > >
> >> > > > > Thoughts?
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <
> >> > manikumar.reddy@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > That's a good idea. Authenticating every request with
> delegation
> >> > > token
> >> > > > > will
> >> > > > > > be useful for
> >> > > > > > impersonation use-cases. But as of now, we are thinking
> >> delegation
> >> > > > token
> >> > > > > as
> >> > > > > > just another way
> >> > > > > > to authenticate the users. We haven't think through all the
> use
> >> > cases
> >> > > > > > related to
> >> > > > > > impersonation or using delegation token for impersonation. We
> >> want
> >> > to
> >> > > > > > handle impersonation
> >> > > > > > (KAFKA-3712) as part of separate KIP.
> >> > > > > >
> >> > > > > > Will that be Ok?
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <
> >> gwen@confluent.io>
> >> > > > wrote:
> >> > > > > >
> >> > > > > > > Thinking out loud here:
> >> > > > > > >
> >> > > > > > > It looks like authentication with a delegation token is
> going
> >> to
> >> > be
> >> > > > > > > super-cheap, right? We just compare the token to a value in
> >> the
> >> > > > broker
> >> > > > > > > cache?
> >> > > > > > >
> >> > > > > > > If I understood the KIP correctly, right now it suggests
> that
> >> > > > > > > authentication happens when establishing the client-broker
> >> > > connection
> >> > > > > (as
> >> > > > > > > normal for Kafka. But perhaps we want to consider
> >> authenticating
> >> > > > every
> >> > > > > > > request with delegation token (if exists)?
> >> > > > > > >
> >> > > > > > > So a centralized app can create few producers, do the
> metadata
> >> > > > request
> >> > > > > > and
> >> > > > > > > broker discovery with its own user auth, but then use
> >> delegation
> >> > > > tokens
> >> > > > > > to
> >> > > > > > > allow performing produce/fetch requests as different users?
> >> > Instead
> >> > > > of
> >> > > > > > > having to re-connect for each impersonated user?
> >> > > > > > >
> >> > > > > > > This may over-complicate things quite a bit (basically
> adding
> >> > extra
> >> > > > > > > information in every request), but maybe it will be useful
> for
> >> > > > > > > impersonation use-cases (which seem to drive much of the
> >> interest
> >> > > in
> >> > > > > this
> >> > > > > > > KIP)?
> >> > > > > > > Kafka Connect, NiFi and friends can probably use this to
> share
> >> > > > clients
> >> > > > > > > between multiple jobs, tasks, etc.
> >> > > > > > >
> >> > > > > > > What do you think?
> >> > > > > > >
> >> > > > > > > Gwen
> >> > > > > > >
> >> > > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> >> > > > manikumar.reddy@gmail.com
> >> > > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Ashish,
> >> > > > > > > >
> >> > > > > > > > Thank you for reviewing the KIP. Please see the replies
> >> > inline.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > > 1. How to disable delegation token authentication?
> >> > > > > > > > >
> >> > > > > > > > > This can be achieved in various ways, however I think
> >> reusing
> >> > > > > > > delegation
> >> > > > > > > > > token secret config for this makes sense here. Avoids
> >> > creating
> >> > > > yet
> >> > > > > > > > another
> >> > > > > > > > > config and forces delegation token users to consciously
> >> set
> >> > the
> >> > > > > > secret.
> >> > > > > > > > If
> >> > > > > > > > > the secret is not set or set to empty string, brokers
> >> should
> >> > > turn
> >> > > > > off
> >> > > > > > > > > delegation token support. This will however require a
> new
> >> > error
> >> > > > > code
> >> > > > > > to
> >> > > > > > > > > indicate delegation token support is turned off on
> broker.
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Thanks for the suggestion. Option to turnoff delegation
> >> token
> >> > > > > > > > authentication will be useful.
> >> > > > > > > > I'll update the KIP.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > 2. ACLs on delegation token?
> >> > > > > > > > >
> >> > > > > > > > > Do we need to have ACLs defined for tokens? I do not
> >> think it
> >> > > > buys
> >> > > > > us
> >> > > > > > > > > anything, as delegation token can be treated as
> >> impersonation
> >> > > of
> >> > > > > the
> >> > > > > > > > owner.
> >> > > > > > > > > Any thing the owner has permission to do, delegation
> >> tokens
> >> > > > should
> >> > > > > be
> >> > > > > > > > > allowed to do as well. If so, we probably won't need to
> >> > return
> >> > > > > > > > > authorization exception error code while creating
> >> delegation
> >> > > > token.
> >> > > > > > It
> >> > > > > > > > > however would make sense to check renew and expire
> >> requests
> >> > are
> >> > > > > > coming
> >> > > > > > > > from
> >> > > > > > > > > owner or renewers of the token, but that does not
> require
> >> > > > explicit
> >> > > > > > > acls.
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Yes, We agreed to not have new acl on who can request
> >> > delegation
> >> > > > > token.
> >> > > > > > > > I'll update the KIP.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > 3. How to restrict max life time of a token?
> >> > > > > > > > >
> >> > > > > > > > > Admins might want to restrict max life time of tokens
> >> created
> >> > > on
> >> > > > a
> >> > > > > > > > cluster,
> >> > > > > > > > > and this can very from cluster to cluster based on
> >> use-cases.
> >> > > > This
> >> > > > > > > might
> >> > > > > > > > > warrant a separate broker config.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > Currently we have "delegation.token.max.lifetime.sec"
> >> server
> >> > > > config
> >> > > > > > > > property
> >> > > > > > > > May be we can take min(User supplied MaxTime, Server
> >> MaxTime)
> >> > as
> >> > > > max
> >> > > > > > life
> >> > > > > > > > time.
> >> > > > > > > > I am open to add new config property.
> >> > > > > > > >
> >> > > > > > > > Few more comments based on recent KIP update.
> >> > > > > > > > >
> >> > > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}?
> >> Can't we
> >> > > use
> >> > > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to
> >> anything
> >> > > > before
> >> > > > > > > > current
> >> > > > > > > > > date?
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > > makes sense. we don't need special request to cancel the
> >> token.
> >> > > We
> >> > > > > can
> >> > > > > > > use
> >> > > > > > > > ExpireTokenRequest.
> >> > > > > > > > I'll update the KIP.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > > 2. Can we change time field names to indicate their unit
> >> is
> >> > > > > > > milliseconds,
> >> > > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > Done.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > > 3. Can we allow users to renew a token for a specified
> >> amount
> >> > > of
> >> > > > > > time?
> >> > > > > > > In
> >> > > > > > > > > current version of KIP, renew request does not take time
> >> as a
> >> > > > > param,
> >> > > > > > > not
> >> > > > > > > > > sure what is expiry time set to after renewal.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > Yes, we need to specify renew period. I'll update the
> KIP.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Mankumar
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> >> > > > > manikumar.reddy@gmail.com
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Hi,
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > I would like to reinitiate the discussion on
> Delegation
> >> > token
> >> > > > > > support
> >> > > > > > > > for
> >> > > > > > > > > >
> >> > > > > > > > > > Kafka.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Brief summary of the past discussion:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All
> >> > brokers
> >> > > > > will
> >> > > > > > > > have a
> >> > > > > > > > > >
> >> > > > > > > > > > cache backed by
> >> > > > > > > > > >
> >> > > > > > > > > > zookeeper so they will all get notified whenever a
> >> new
> >> > > token
> >> > > > > is
> >> > > > > > > > > >
> >> > > > > > > > > > generated and they will
> >> > > > > > > > > >
> >> > > > > > > > > > update their local cache whenever token state
> >> changes.
> >> > > > > > > > > >
> >> > > > > > > > > > 2) The current proposal does not support rotation of
> >> secret
> >> > > > > > > > > >
> >> > > > > > > > > > 3) Only allow the renewal by users that authenticated
> >> using
> >> > > > *non*
> >> > > > > > > > > >
> >> > > > > > > > > > delegation token mechanism
> >> > > > > > > > > >
> >> > > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms.
> >> Kafka
> >> > > > > clients
> >> > > > > > > can
> >> > > > > > > > > >
> >> > > > > > > > > > authenticate using
> >> > > > > > > > > >
> >> > > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC
> as
> >> > > > > password.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Updated the KIP with the following:
> >> > > > > > > > > >
> >> > > > > > > > > > 1. Protocol and Config changes
> >> > > > > > > > > >
> >> > > > > > > > > > 2. format of the data stored in ZK.
> >> > > > > > > > > >
> >> > > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM
> mechanism
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > https://cwiki.apache.org/confl
> uence/display/KAFKA/KIP-
> >> > > > > > > > > >
> >> > > > > > > > > > 48+Delegation+token+support+for+Kafka
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Jun, Ashish, Gwen,
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Pl review the updated KIP.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks
> >> > > > > > > > > >
> >> > > > > > > > > > Manikumar
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> >> > > > > asingh@cloudera.com
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > Harsha/ Gwen,
> >> > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > How do we proceed here? I am willing to help out
> with
> >> > here.
> >> > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> >> > > > > > gwen@confluent.io>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > Is it updated? are all concerns addressed? do you
> >> want
> >> > to
> >> > > > > > start a
> >> > > > > > > > > vote?
> >> > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > Sorry for being pushy, I do appreciate that we are
> >> all
> >> > > > > > volunteers
> >> > > > > > > > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > finding time is difficult. This feature is
> important
> >> > for
> >> > > > > > anything
> >> > > > > > > > > that
> >> > > > > > > > > >
> >> > > > > > > > > > > > integrates with Kafka (stream processors, Flume,
> >> NiFi,
> >> > > etc)
> >> > > > > > and I
> >> > > > > > > > > >
> >> > > > > > > > > > > > don't want to see this getting stuck because we
> lack
> >> > > > > > coordination
> >> > > > > > > > > >
> >> > > > > > > > > > > > within the community.
> >> > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha
> >> Chintalapani <
> >> > > > > > > > > kafka@harsha.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > > The only pending update for the KIP is to write
> up
> >> > the
> >> > > > > > protocol
> >> > > > > > > > > > changes
> >> > > > > > > > > >
> >> > > > > > > > > > > > like
> >> > > > > > > > > >
> >> > > > > > > > > > > > > we've it KIP-4. I'll update the wiki.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> >> > > > > > > > asingh@cloudera.com>
> >> > > > > > > > > >
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> I think we decided to not support secret
> >> rotation, I
> >> > > > guess
> >> > > > > > > this
> >> > > > > > > > > can
> >> > > > > > > > > > be
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> stated clearly on the KIP. Also, more details
> on
> >> how
> >> > > > > clients
> >> > > > > > > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > perform
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> token distribution and how CLI will look like
> >> will
> >> > be
> >> > > > > > helpful.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> >> > > > > > > > gwen@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > Hi Guys,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > This discussion was dead for a while. Are
> there
> >> > > still
> >> > > > > > > > > contentious
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > points? If not, why are there no votes?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> >> > > > > > jun@confluent.io>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > Ashish,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next
> >> week
> >> > to
> >> > > > > > discuss
> >> > > > > > > > > > KIP-48
> >> > > > > > > > > >
> >> > > > > > > > > > > > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > other
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > remaining KIPs.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > Jun
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish
> >> Singh <
> >> > > > > > > > > >
> >> > > > > > > > > > > asingh@cloudera.com>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> Thanks Harsha!
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP
> hangout's
> >> > > > agenda.
> >> > > > > > > Also,
> >> > > > > > > > we
> >> > > > > > > > > > did
> >> > > > > > > > > >
> >> > > > > > > > > > > > not
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> actually make a call on when we should
> have
> >> > next
> >> > > > KIP
> >> > > > > > > call.
> >> > > > > > > > As
> >> > > > > > > > > >
> >> > > > > > > > > > > there
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> are
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> few outstanding KIPs that could not be
> >> > discussed
> >> > > > this
> >> > > > > > > week,
> >> > > > > > > > > can
> >> > > > > > > > > >
> >> > > > > > > > > > > we
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> have
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> KIP hangout call next week?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> >> > > > Chintalapani
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> <ka...@harsha.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> Ashish,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> Yes we are working on it. Lets
> >> discuss
> >> > > in
> >> > > > > the
> >> > > > > > > next
> >> > > > > > > > > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> meeting.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> I'll join.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> -Harsha
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish
> >> Singh
> >> > <
> >> > > > > > > > > >
> >> > > > > > > > > > > > asingh@cloudera.com>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > Hello Harsha,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > Are you still working on this?
> Wondering
> >> if
> >> > we
> >> > > > can
> >> > > > > > > > discuss
> >> > > > > > > > > >
> >> > > > > > > > > > > this
> >> > > > > > > > > >
> >> > > > > > > > > > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > next
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > meeting, if you can join.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> >> > > > > > Chintalapani <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > kafka@harsha.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > Hi Grant,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > We are working on it. Will
> >> add
> >> > the
> >> > > > > > details
> >> > > > > > > > to
> >> > > > > > > > > > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > about
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > request protocol.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > Harsha
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant
> >> > Henke
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > Hi Parth,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > Are you still working on this? If
> you
> >> > need
> >> > > > any
> >> > > > > > > help
> >> > > > > > > > > > please
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > don't
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > hesitate
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > to ask.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > Grant
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM,
> Jun
> >> > Rao <
> >> > > > > > > > > >
> >> > > > > > > > > > > jun@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > Parth,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > Thanks for the reply.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the
> >> > renewal
> >> > > > by
> >> > > > > > > users
> >> > > > > > > > > that
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> authenticated
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > using
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism.
> >> > Then,
> >> > > > > should
> >> > > > > > > we
> >> > > > > > > > > make
> >> > > > > > > > > >
> >> > > > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> renewal a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > list?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > For example, in the case of rest
> >> > proxy,
> >> > > it
> >> > > > > > will
> >> > > > > > > be
> >> > > > > > > > > >
> >> > > > > > > > > > > useful
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > for
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> every
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able
> >> to
> >> > > renew
> >> > > > > the
> >> > > > > > > > > tokens.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > It would be clearer if we can
> >> document
> >> > > the
> >> > > > > > > request
> >> > > > > > > > > >
> >> > > > > > > > > > > > protocol
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > like
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> >> > > > > > > > > uence/display/KAFKA/KIP-
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> >> > > > > centralized+administrative+
> >> > > > > > > > > >
> >> > > > > > > > > > > > operations#KIP-4-
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> >> > > > > > istrativeoperations-
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> >> > > > > > > > 2945):(VotedandPlannedforin0.
> >> > > > > > > > > >
> >> > > > > > > > > > > > 10.1.0)
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > .
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > It would also be useful to
> document
> >> > the
> >> > > > > client
> >> > > > > > > > APIs.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > Jun
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM,
> >> parth
> >> > > > > > > brahmbhatt
> >> > > > > > > > <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com>
> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Hi,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will
> only
> >> > > allow
> >> > > > > the
> >> > > > > > > > > renewal
> >> > > > > > > > > > by
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > users
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> that
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > authenticated using *non*
> >> delegation
> >> > > > token
> >> > > > > > > > > > mechanism.
> >> > > > > > > > > >
> >> > > > > > > > > > > > For
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> example,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > If
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > user
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Alice authenticated using
> >> kerberos
> >> > and
> >> > > > > > > requested
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> tokens,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > only
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via
> non
> >> > > > > delegation
> >> > > > > > > > token
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > mechanism
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > can
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > renew.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Clients that have access to
> >> > > delegation
> >> > > > > > tokens
> >> > > > > > > > can
> >> > > > > > > > > > not
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > issue
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > renewal
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > request for renewing their own
> >> token
> >> > > and
> >> > > > > > this
> >> > > > > > > is
> >> > > > > > > > > >
> >> > > > > > > > > > > > primarily
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > important
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > reduce the time window for
> which
> >> a
> >> > > > > > compromised
> >> > > > > > > > > token
> >> > > > > > > > > >
> >> > > > > > > > > > > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > be
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> valid.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any
> authenticated
> >> > user
> >> > > > can
> >> > > > > > > > request
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > tokens
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > but
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > even here I would recommend to
> >> avoid
> >> > > > > > creating
> >> > > > > > > a
> >> > > > > > > > > > chain
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > where a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > client
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > authenticated via delegation
> >> token
> >> > > > request
> >> > > > > > for
> >> > > > > > > > > more
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > tokens.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Basically anyone can request
> >> > > delegation
> >> > > > > > token,
> >> > > > > > > > as
> >> > > > > > > > > > long
> >> > > > > > > > > >
> >> > > > > > > > > > > > as
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > they
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > authenticate
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > via a non delegation token
> >> > mechanism.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> >> > > > > > > > > uence/display/KAFKA/KIP-
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> >> > > > > > > > > >
> >> > > > > > > > > > > r+Kafka#KIP-48Delegationtokens
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> upportforKaf
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > ka-PublicInterfaces
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > sufficient?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Thanks
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > Parth
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33
> PM,
> >> Jun
> >> > > Rao
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > Parth,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A
> couple
> >> of
> >> > > > > comments
> >> > > > > > > > > inline
> >> > > > > > > > > >
> >> > > > > > > > > > > > below.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36
> >> AM,
> >> > > > parth
> >> > > > > > > > > > brahmbhatt <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com>
> >> > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens
> >> renewed?
> >> > > By
> >> > > > > > > original
> >> > > > > > > > > >
> >> > > > > > > > > > > > requester
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> only? or
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > using
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Kerberos
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > auth only?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do
> >> this
> >> > > only
> >> > > > > > using
> >> > > > > > > > > >
> >> > > > > > > > > > > Kerberos
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > auth
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > only
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > threw
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > renewer specified during
> the
> >> > > > > acquisition
> >> > > > > > > > > > request.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow
> >> this.
> >> > > Are
> >> > > > > you
> >> > > > > > > > saying
> >> > > > > > > > > >
> >> > > > > > > > > > > that
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > any
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> client
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > authenticated with the
> >> delegation
> >> > > > token
> >> > > > > > can
> >> > > > > > > > > renew,
> >> > > > > > > > > >
> >> > > > > > > > > > > > i.e.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > there
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> is
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > no
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > renewer
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > needed?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> >> > > > > authenticated
> >> > > > > > > > client
> >> > > > > > > > > >
> >> > > > > > > > > > > > (either
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> through
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > SASL
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > or
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation
> >> > token
> >> > > > for
> >> > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > authenticated
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> user,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > right?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on
> each
> >> > > broker
> >> > > > or
> >> > > > > > in
> >> > > > > > > > ZK?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still
> to
> >> > > store
> >> > > > in
> >> > > > > > ZK
> >> > > > > > > or
> >> > > > > > > > > not
> >> > > > > > > > > >
> >> > > > > > > > > > > > store
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > them
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> at
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > all.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > The
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > whole controller based
> >> > > distribution
> >> > > > is
> >> > > > > > too
> >> > > > > > > > > much
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > overhead
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> with
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > not
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > much
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > achieve.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens
> >> invalidated /
> >> > > > > expired?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time
> >> out or
> >> > > > > through
> >> > > > > > > an
> >> > > > > > > > > >
> >> > > > > > > > > > > explicit
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> request to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > invalidate.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption
> >> algorithm is
> >> > > > used?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > SCRAM
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 5. What is the
> impersonation
> >> > > > proposal
> >> > > > > > (it
> >> > > > > > > > > wasn't
> >> > > > > > > > > >
> >> > > > > > > > > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> but
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > was
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > discussed
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > in this thread)?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
> >> > > proposal. I
> >> > > > > > tried
> >> > > > > > > > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > explained
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > how
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > its
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > different problem and why
> its
> >> > not
> >> > > > > really
> >> > > > > > > > > > necessary
> >> > > > > > > > > >
> >> > > > > > > > > > > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> discuss
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > that
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > as
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > part
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will
> >> not
> >> > > > > support
> >> > > > > > > any
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > impersonation,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> it
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > just
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > be
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > another way to
> authenticate.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if
> >> so -
> >> > > for
> >> > > > > what
> >> > > > > > > > > > actions?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > Could we document the format
> of
> >> > the
> >> > > > new
> >> > > > > > > > > >
> >> > > > > > > > > > > > request/response
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > their
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > associated Resource and
> >> Operation
> >> > > for
> >> > > > > ACL?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation
> >> > token
> >> > > be
> >> > > > > > > > > configured
> >> > > > > > > > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> client?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I
> >> > wasn't
> >> > > > > > > planning
> >> > > > > > > > on
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > supporting
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> JAAS
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > for
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > tokens.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does
> >> this
> >> > > > > either.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Thanks
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Parth
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at
> 4:03
> >> PM,
> >> > > Jun
> >> > > > > > Rao <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > jun@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > Harsha,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > Another question.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the
> delegation
> >> > > token
> >> > > > be
> >> > > > > > > > > > configured
> >> > > > > > > > > >
> >> > > > > > > > > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > client?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > The
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > standard
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through
> >> > JAAS.
> >> > > > > > However,
> >> > > > > > > > we
> >> > > > > > > > > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > need
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > think
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > through
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > if
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a
> >> shared
> >> > > > > > > > environment.
> >> > > > > > > > > > For
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > example,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > when a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > new
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > task
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > is
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker
> >> node,
> >> > do
> >> > > > we
> >> > > > > > need
> >> > > > > > > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > dynamically
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> add a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > new
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > section
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may
> be
> >> > more
> >> > > > > > > > convenient
> >> > > > > > > > > if
> >> > > > > > > > > >
> >> > > > > > > > > > > we
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > can
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> pass in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > token
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > through the config
> directly
> >> > w/o
> >> > > > > going
> >> > > > > > > > > through
> >> > > > > > > > > >
> >> > > > > > > > > > > > JAAS.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still
> >> > actively
> >> > > > > > working
> >> > > > > > > on
> >> > > > > > > > > > this
> >> > > > > > > > > >
> >> > > > > > > > > > > > KIP?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > Jun
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at
> >> 2:18
> >> > PM,
> >> > > > Jun
> >> > > > > > > Rao <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> jun@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that
> list.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
> >> > > document
> >> > > > > the
> >> > > > > > > > format
> >> > > > > > > > > > of
> >> > > > > > > > > >
> >> > > > > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > data
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > stored
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > ZK.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> >> > > > discussion
> >> > > > > > on
> >> > > > > > > > > > whether
> >> > > > > > > > > >
> >> > > > > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > tokens
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > should
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > be
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK
> >> like
> >> > > > > > > > > config/acl/quota,
> >> > > > > > > > > >
> >> > > > > > > > > > > or
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > through
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > controller.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the
> >> controller is
> >> > > > only
> >> > > > > > > > designed
> >> > > > > > > > > > for
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> propagating
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > topic
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > metadata,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM
> to
> >> > send
> >> > > > the
> >> > > > > > > token
> >> > > > > > > > > >
> >> > > > > > > > > > > instead
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > of
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > DIGEST-MD5
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > since
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > it's
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the
> >> wiki
> >> > > > seem
> >> > > > > > > > broken.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > Jun
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at
> >> > 10:02
> >> > > > AM,
> >> > > > > > Gwen
> >> > > > > > > > > >
> >> > > > > > > > > > > Shapira <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
> >> > > remaining
> >> > > > > > > > questions
> >> > > > > > > > > > are:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are
> tokens
> >> > > > renewed?
> >> > > > > By
> >> > > > > > > > > > original
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > requester
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > only?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > or
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > using
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored
> on
> >> > each
> >> > > > > broker
> >> > > > > > > or
> >> > > > > > > > in
> >> > > > > > > > > > ZK?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
> >> > > invalidated /
> >> > > > > > > > expired?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption
> >> > algorithm
> >> > > > is
> >> > > > > > > used?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the
> >> > impersonation
> >> > > > > > proposal
> >> > > > > > > > (it
> >> > > > > > > > > >
> >> > > > > > > > > > > > wasn't
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > but
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > was
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this
> >> thread)?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new
> ACLs,
> >> if
> >> > > so -
> >> > > > > for
> >> > > > > > > > what
> >> > > > > > > > > >
> >> > > > > > > > > > > > actions?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at
> >> 7:48
> >> > > PM,
> >> > > > > > > Harsha
> >> > > > > > > > <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> kafka@harsha.io>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> >> > > > > > > > Unfortunately
> >> > > > > > > > > I
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> attend
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > meeting
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> >> > > when
> >> > > > > > > > > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > tokens
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > discussed.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > Appreciate
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > if
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> >> > > you
> >> > > > > can
> >> > > > > > > > update
> >> > > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > thread if
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > you
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > have
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > any
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> >> > > > > further
> >> > > > > > > > > > questions.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24,
> 2016,
> >> at
> >> > > > 11:32
> >> > > > > > AM,
> >> > > > > > > > > Liquan
> >> > > > > > > > > >
> >> > > > > > > > > > > Pei
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the
> >> links
> >> > to
> >> > > > > > images
> >> > > > > > > in
> >> > > > > > > > > the
> >> > > > > > > > > >
> >> > > > > > > > > > > KIP
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> are
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > broken.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24,
> 2016
> >> at
> >> > > 9:33
> >> > > > > AM,
> >> > > > > > > > parth
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > brahmbhatt <
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> >> > > brahmbhatt.parth@gmail.com>
> >> > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> >> > > > > > > > getDelegationTokenAs
> >> > > > > > > > > >
> >> > > > > > > > > > > mean?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current
> >> > proposal
> >> > > we
> >> > > > > > only
> >> > > > > > > > > allow
> >> > > > > > > > > > a
> >> > > > > > > > > >
> >> > > > > > > > > > > > user
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> get
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > token
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> for
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that
> >> it
> >> > > > > > > authenticated
> >> > > > > > > > > as
> >> > > > > > > > > >
> >> > > > > > > > > > > > using
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> another
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > mechanism,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > i.e.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> A user
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate
> >> > using
> >> > > a
> >> > > > > > keytab
> >> > > > > > > > for
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> will get
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
> >> for
> >> > > that
> >> > > > > > user
> >> > > > > > > > > only.
> >> > > > > > > > > > In
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > future I
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > think
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > we
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > have
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support
> such
> >> > that
> >> > > > we
> >> > > > > > > allow
> >> > > > > > > > > some
> >> > > > > > > > > >
> >> > > > > > > > > > > set
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> users (
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> >> > > > kafka-rest-user@EXAMPLE.COM
> >> > > > > ,
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > acquire
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
> >> on
> >> > > > behalf
> >> > > > > of
> >> > > > > > > > other
> >> > > > > > > > > >
> >> > > > > > > > > > > users
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > whose
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > identity
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > they
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > have
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified
> >> > independently.
> >> > > > > Kafka
> >> > > > > > > > > brokers
> >> > > > > > > > > >
> >> > > > > > > > > > > > will
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > have
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> ACLs
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > to
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > control
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> which
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed
> >> to
> >> > > > > > impersonate
> >> > > > > > > > > other
> >> > > > > > > > > >
> >> > > > > > > > > > > > users
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> get
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > tokens
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > on
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> >> > > > Impersonation
> >> > > > > > is a
> >> > > > > > > > > whole
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > different
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > problem
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > in
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > my
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can
> >> tackle
> >> > it
> >> > > > in
> >> > > > > > > > separate
> >> > > > > > > > > >
> >> > > > > > > > > > > KIP.
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the
> >> > typical
> >> > > > rate
> >> > > > > > of
> >> > > > > > > > > > getting
> >> > > > > > > > > >
> >> > > > > > > > > > > > and
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> renewing
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > delegation
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this
> >> should
> >> > be
> >> > > > > very
> >> > > > > > > very
> >> > > > > > > > > > low,
> >> > > > > > > > > >
> >> > > > > > > > > > > 1
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > request
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> per
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > minute
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > is a
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
> >> > > estimate.
> >> > > > > > > However
> >> > > > > > > > it
> >> > > > > > > > > >
> >> > > > > > > > > > > > depends
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> the
> >> > > > > > > > > >
> >> > > > > > > > > > > > >> > >>> > > token
> >> > > > > > > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi Devs,
If there are no more comments, I will start vote on this KIP later this
week.
Thanks
On Fri, Dec 16, 2016 at 12:28 PM, Manikumar <ma...@gmail.com>
wrote:
> Hi,
>
>
>> Can you add a sample Jaas configuration using delegation tokens to the
>> KIP?
>>
>
> Will add sample Jaas configuration.
>
>
>> To make sure I have understood correctly, KAFKA-3712 is aimed at enabling
>> a
>> superuser to impersonate another (single) user, say alice. A producer
>> using
>> impersonation will authenticate with superuser credentials. All requests
>> from the producer will be run with the principal alice. But alice is not
>> involved in the authentication and alice's credentials are not actually
>> provided to the broker?
>>
>>
> Yes, this matches with my understanding of impersonation work . Even in
> this approach
> we have to handle all producer related issues mentioned by you. Yes, this
> is big change
> and can be implemented if there is a pressing need. I hope we are all in
> agreement, that
> this can be done in a separate KIP.
>
>
> I request others give any suggestions/concerns on this KIP.
>
>
> Thanks,
>
>
>
>>
>> On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <ma...@gmail.com>
>> wrote:
>>
>> > @Gwen, @Rajini,
>> >
>> > As mentioned in the KIP, main motivation for this KIP is to reduce load
>> on
>> > Kerberos
>> > server on large kafka deployments with large number of clients.
>> >
>> > Also it looks like we are combining two overlapping concepts
>> > 1. Single client sending requests with multiple users/authentications
>> > 2. Impersonation
>> >
>> > Option 1, is definitely useful in some use cases and can be used to
>> > implement workaround for
>> > impersonation
>> >
>> > In Impersonation, a super user can send requests on behalf of another
>> > user(Alice) in a secured way.
>> > superuser has credentials but user Alice doesn't have any. The requests
>> are
>> > required
>> > to run as user Alice and accesses/ACLs on Broker are required to be
>> done as
>> > user Alice.
>> > It is required that user Alice can connect to the Broker on a connection
>> > authenticated with
>> > superuser's credentials. In other words superuser is impersonating the
>> user
>> > Alice.
>> >
>> > The approach mentioned by Harsha in previous mail is implemented in
>> hadoop,
>> > storm etc..
>> >
>> > Some more details here:
>> > https://hadoop.apache.org/docs/r2.7.2/hadoop-project-
>> > dist/hadoop-common/Superusers.html
>> >
>> >
>> > @Rajini
>> >
>> > Thanks for your comments on SASL/SCRAM usage. I am thinking to send
>> > tokenHmac (salted-hashed version)
>> > as password for authentication and tokenID for retrial of tokenHmac at
>> > server side.
>> > Does above sound OK?
>> >
>> >
>> > Thanks,
>> > Manikumar
>> >
>> > On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io>
>> > wrote:
>> >
>> > > @Gwen @Mani Not sure why we want to authenticate at every request.
>> Even
>> > if
>> > > the token exchange is cheap it still a few calls that need to go
>> through
>> > > round trip. Impersonation doesn't require authentication for every
>> > > request.
>> > >
>> > > "So a centralized app can create few producers, do the metadata
>> request
>> > and
>> > > broker discovery with its own user auth, but then use delegation
>> tokens
>> > to
>> > > allow performing produce/fetch requests as different users? Instead of
>> > > having to re-connect for each impersonated user?"
>> > >
>> > > Yes. But what we will have is this centralized user as impersonation
>> user
>> > > on behalf of other users. When it authenticates initially we will
>> create
>> > a
>> > > "Subject" and from there on wards centralized user can do
>> > > Subject.doAsPrivileged
>> > > on behalf, other users.
>> > > On the server side, we can retrieve two principals out of this one is
>> the
>> > > authenticated user (centralized user) and another is impersonated
>> user.
>> > We
>> > > will first check if the authenticated user allowed to impersonate and
>> > then
>> > > move on to check if the user Alice has access to the topic "X" to
>> > > read/write.
>> > >
>> > > @Rajini Intention of this KIP is to support token auth via SASL/SCRAM,
>> > not
>> > > just with TLS. What you raised is a good point let me take a look and
>> > add
>> > > details.
>> > >
>> > > It will be easier to add impersonation once we reach agreement on this
>> > KIP.
>> > >
>> > >
>> > > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk>
>> wrote:
>> > >
>> > > > Hi Rajini,
>> > > >
>> > > > I think it would definitely be valuable to have a KIP for
>> > impersonation.
>> > > >
>> > > > Ismael
>> > > >
>> > > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <
>> rsivaram@pivotal.io>
>> > > > wrote:
>> > > >
>> > > > > It would clearly be very useful to enable clients to send
>> requests on
>> > > > > behalf of multiple users. A separate KIP makes sense, but it may
>> be
>> > > worth
>> > > > > thinking through some of the implications now, especially if the
>> main
>> > > > > interest in delegation tokens comes from its potential to enable
>> > > > > impersonation.
>> > > > >
>> > > > > I understand that delegation tokens are only expected to be used
>> with
>> > > > TLS.
>> > > > > But the choice of SASL/SCRAM for authentication must be based on a
>> > > > > requirement to protect the tokenHmac - otherwise you could just
>> use
>> > > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
>> > > > on-the-wire,
>> > > > > only a salted-hashed version of it is used in the SASL
>> authentication
>> > > > > exchange. If impersonation is based on sending tokenHmac in
>> requests,
>> > > any
>> > > > > benefit of using SCRAM is lost.
>> > > > >
>> > > > > An alternative may be to allow clients to authenticate multiple
>> times
>> > > > using
>> > > > > SASL and include one of its authenticated principals in each
>> request
>> > > > > (optionally). I haven't thought it through yet, obviously. But if
>> the
>> > > > > approach is of interest and no one is working on a KIP for
>> > > impersonation
>> > > > at
>> > > > > the moment, I am happy to write one. It may provide something for
>> > > > > comparison at least.
>> > > > >
>> > > > > Thoughts?
>> > > > >
>> > > > >
>> > > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <
>> > manikumar.reddy@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > That's a good idea. Authenticating every request with delegation
>> > > token
>> > > > > will
>> > > > > > be useful for
>> > > > > > impersonation use-cases. But as of now, we are thinking
>> delegation
>> > > > token
>> > > > > as
>> > > > > > just another way
>> > > > > > to authenticate the users. We haven't think through all the use
>> > cases
>> > > > > > related to
>> > > > > > impersonation or using delegation token for impersonation. We
>> want
>> > to
>> > > > > > handle impersonation
>> > > > > > (KAFKA-3712) as part of separate KIP.
>> > > > > >
>> > > > > > Will that be Ok?
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <
>> gwen@confluent.io>
>> > > > wrote:
>> > > > > >
>> > > > > > > Thinking out loud here:
>> > > > > > >
>> > > > > > > It looks like authentication with a delegation token is going
>> to
>> > be
>> > > > > > > super-cheap, right? We just compare the token to a value in
>> the
>> > > > broker
>> > > > > > > cache?
>> > > > > > >
>> > > > > > > If I understood the KIP correctly, right now it suggests that
>> > > > > > > authentication happens when establishing the client-broker
>> > > connection
>> > > > > (as
>> > > > > > > normal for Kafka. But perhaps we want to consider
>> authenticating
>> > > > every
>> > > > > > > request with delegation token (if exists)?
>> > > > > > >
>> > > > > > > So a centralized app can create few producers, do the metadata
>> > > > request
>> > > > > > and
>> > > > > > > broker discovery with its own user auth, but then use
>> delegation
>> > > > tokens
>> > > > > > to
>> > > > > > > allow performing produce/fetch requests as different users?
>> > Instead
>> > > > of
>> > > > > > > having to re-connect for each impersonated user?
>> > > > > > >
>> > > > > > > This may over-complicate things quite a bit (basically adding
>> > extra
>> > > > > > > information in every request), but maybe it will be useful for
>> > > > > > > impersonation use-cases (which seem to drive much of the
>> interest
>> > > in
>> > > > > this
>> > > > > > > KIP)?
>> > > > > > > Kafka Connect, NiFi and friends can probably use this to share
>> > > > clients
>> > > > > > > between multiple jobs, tasks, etc.
>> > > > > > >
>> > > > > > > What do you think?
>> > > > > > >
>> > > > > > > Gwen
>> > > > > > >
>> > > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
>> > > > manikumar.reddy@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Ashish,
>> > > > > > > >
>> > > > > > > > Thank you for reviewing the KIP. Please see the replies
>> > inline.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > > 1. How to disable delegation token authentication?
>> > > > > > > > >
>> > > > > > > > > This can be achieved in various ways, however I think
>> reusing
>> > > > > > > delegation
>> > > > > > > > > token secret config for this makes sense here. Avoids
>> > creating
>> > > > yet
>> > > > > > > > another
>> > > > > > > > > config and forces delegation token users to consciously
>> set
>> > the
>> > > > > > secret.
>> > > > > > > > If
>> > > > > > > > > the secret is not set or set to empty string, brokers
>> should
>> > > turn
>> > > > > off
>> > > > > > > > > delegation token support. This will however require a new
>> > error
>> > > > > code
>> > > > > > to
>> > > > > > > > > indicate delegation token support is turned off on broker.
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks for the suggestion. Option to turnoff delegation
>> token
>> > > > > > > > authentication will be useful.
>> > > > > > > > I'll update the KIP.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > >
>> > > > > > > > > 2. ACLs on delegation token?
>> > > > > > > > >
>> > > > > > > > > Do we need to have ACLs defined for tokens? I do not
>> think it
>> > > > buys
>> > > > > us
>> > > > > > > > > anything, as delegation token can be treated as
>> impersonation
>> > > of
>> > > > > the
>> > > > > > > > owner.
>> > > > > > > > > Any thing the owner has permission to do, delegation
>> tokens
>> > > > should
>> > > > > be
>> > > > > > > > > allowed to do as well. If so, we probably won't need to
>> > return
>> > > > > > > > > authorization exception error code while creating
>> delegation
>> > > > token.
>> > > > > > It
>> > > > > > > > > however would make sense to check renew and expire
>> requests
>> > are
>> > > > > > coming
>> > > > > > > > from
>> > > > > > > > > owner or renewers of the token, but that does not require
>> > > > explicit
>> > > > > > > acls.
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Yes, We agreed to not have new acl on who can request
>> > delegation
>> > > > > token.
>> > > > > > > > I'll update the KIP.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > >
>> > > > > > > > > 3. How to restrict max life time of a token?
>> > > > > > > > >
>> > > > > > > > > Admins might want to restrict max life time of tokens
>> created
>> > > on
>> > > > a
>> > > > > > > > cluster,
>> > > > > > > > > and this can very from cluster to cluster based on
>> use-cases.
>> > > > This
>> > > > > > > might
>> > > > > > > > > warrant a separate broker config.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > Currently we have "delegation.token.max.lifetime.sec"
>> server
>> > > > config
>> > > > > > > > property
>> > > > > > > > May be we can take min(User supplied MaxTime, Server
>> MaxTime)
>> > as
>> > > > max
>> > > > > > life
>> > > > > > > > time.
>> > > > > > > > I am open to add new config property.
>> > > > > > > >
>> > > > > > > > Few more comments based on recent KIP update.
>> > > > > > > > >
>> > > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}?
>> Can't we
>> > > use
>> > > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to
>> anything
>> > > > before
>> > > > > > > > current
>> > > > > > > > > date?
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > > makes sense. we don't need special request to cancel the
>> token.
>> > > We
>> > > > > can
>> > > > > > > use
>> > > > > > > > ExpireTokenRequest.
>> > > > > > > > I'll update the KIP.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > > 2. Can we change time field names to indicate their unit
>> is
>> > > > > > > milliseconds,
>> > > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > Done.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > > 3. Can we allow users to renew a token for a specified
>> amount
>> > > of
>> > > > > > time?
>> > > > > > > In
>> > > > > > > > > current version of KIP, renew request does not take time
>> as a
>> > > > > param,
>> > > > > > > not
>> > > > > > > > > sure what is expiry time set to after renewal.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > Yes, we need to specify renew period. I'll update the KIP.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Mankumar
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
>> > > > > manikumar.reddy@gmail.com
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi,
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > I would like to reinitiate the discussion on Delegation
>> > token
>> > > > > > support
>> > > > > > > > for
>> > > > > > > > > >
>> > > > > > > > > > Kafka.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Brief summary of the past discussion:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All
>> > brokers
>> > > > > will
>> > > > > > > > have a
>> > > > > > > > > >
>> > > > > > > > > > cache backed by
>> > > > > > > > > >
>> > > > > > > > > > zookeeper so they will all get notified whenever a
>> new
>> > > token
>> > > > > is
>> > > > > > > > > >
>> > > > > > > > > > generated and they will
>> > > > > > > > > >
>> > > > > > > > > > update their local cache whenever token state
>> changes.
>> > > > > > > > > >
>> > > > > > > > > > 2) The current proposal does not support rotation of
>> secret
>> > > > > > > > > >
>> > > > > > > > > > 3) Only allow the renewal by users that authenticated
>> using
>> > > > *non*
>> > > > > > > > > >
>> > > > > > > > > > delegation token mechanism
>> > > > > > > > > >
>> > > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms.
>> Kafka
>> > > > > clients
>> > > > > > > can
>> > > > > > > > > >
>> > > > > > > > > > authenticate using
>> > > > > > > > > >
>> > > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
>> > > > > password.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Updated the KIP with the following:
>> > > > > > > > > >
>> > > > > > > > > > 1. Protocol and Config changes
>> > > > > > > > > >
>> > > > > > > > > > 2. format of the data stored in ZK.
>> > > > > > > > > >
>> > > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > > > > > > >
>> > > > > > > > > > 48+Delegation+token+support+for+Kafka
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Jun, Ashish, Gwen,
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Pl review the updated KIP.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Thanks
>> > > > > > > > > >
>> > > > > > > > > > Manikumar
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
>> > > > > asingh@cloudera.com
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > Harsha/ Gwen,
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > How do we proceed here? I am willing to help out with
>> > here.
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
>> > > > > > gwen@confluent.io>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > Is it updated? are all concerns addressed? do you
>> want
>> > to
>> > > > > > start a
>> > > > > > > > > vote?
>> > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > Sorry for being pushy, I do appreciate that we are
>> all
>> > > > > > volunteers
>> > > > > > > > and
>> > > > > > > > > >
>> > > > > > > > > > > > finding time is difficult. This feature is important
>> > for
>> > > > > > anything
>> > > > > > > > > that
>> > > > > > > > > >
>> > > > > > > > > > > > integrates with Kafka (stream processors, Flume,
>> NiFi,
>> > > etc)
>> > > > > > and I
>> > > > > > > > > >
>> > > > > > > > > > > > don't want to see this getting stuck because we lack
>> > > > > > coordination
>> > > > > > > > > >
>> > > > > > > > > > > > within the community.
>> > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha
>> Chintalapani <
>> > > > > > > > > kafka@harsha.io>
>> > > > > > > > > >
>> > > > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > > The only pending update for the KIP is to write up
>> > the
>> > > > > > protocol
>> > > > > > > > > > changes
>> > > > > > > > > >
>> > > > > > > > > > > > like
>> > > > > > > > > >
>> > > > > > > > > > > > > we've it KIP-4. I'll update the wiki.
>> > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
>> > > > > > > > asingh@cloudera.com>
>> > > > > > > > > >
>> > > > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> I think we decided to not support secret
>> rotation, I
>> > > > guess
>> > > > > > > this
>> > > > > > > > > can
>> > > > > > > > > > be
>> > > > > > > > > >
>> > > > > > > > > > > > >> stated clearly on the KIP. Also, more details on
>> how
>> > > > > clients
>> > > > > > > > will
>> > > > > > > > > >
>> > > > > > > > > > > > perform
>> > > > > > > > > >
>> > > > > > > > > > > > >> token distribution and how CLI will look like
>> will
>> > be
>> > > > > > helpful.
>> > > > > > > > > >
>> > > > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
>> > > > > > > > gwen@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > Hi Guys,
>> > > > > > > > > >
>> > > > > > > > > > > > >> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > This discussion was dead for a while. Are there
>> > > still
>> > > > > > > > > contentious
>> > > > > > > > > >
>> > > > > > > > > > > > >> > points? If not, why are there no votes?
>> > > > > > > > > >
>> > > > > > > > > > > > >> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
>> > > > > > jun@confluent.io>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > Ashish,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next
>> week
>> > to
>> > > > > > discuss
>> > > > > > > > > > KIP-48
>> > > > > > > > > >
>> > > > > > > > > > > > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > other
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > remaining KIPs.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > Jun
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish
>> Singh <
>> > > > > > > > > >
>> > > > > > > > > > > asingh@cloudera.com>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> Thanks Harsha!
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
>> > > > agenda.
>> > > > > > > Also,
>> > > > > > > > we
>> > > > > > > > > > did
>> > > > > > > > > >
>> > > > > > > > > > > > not
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> actually make a call on when we should have
>> > next
>> > > > KIP
>> > > > > > > call.
>> > > > > > > > As
>> > > > > > > > > >
>> > > > > > > > > > > there
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> are
>> > > > > > > > > >
>> > > > > > > > > > > > >> > a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> few outstanding KIPs that could not be
>> > discussed
>> > > > this
>> > > > > > > week,
>> > > > > > > > > can
>> > > > > > > > > >
>> > > > > > > > > > > we
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> have
>> > > > > > > > > >
>> > > > > > > > > > > > >> > a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> KIP hangout call next week?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
>> > > > Chintalapani
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> <ka...@harsha.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> Ashish,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> Yes we are working on it. Lets
>> discuss
>> > > in
>> > > > > the
>> > > > > > > next
>> > > > > > > > > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> meeting.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> I'll join.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> -Harsha
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish
>> Singh
>> > <
>> > > > > > > > > >
>> > > > > > > > > > > > asingh@cloudera.com>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > Hello Harsha,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > Are you still working on this? Wondering
>> if
>> > we
>> > > > can
>> > > > > > > > discuss
>> > > > > > > > > >
>> > > > > > > > > > > this
>> > > > > > > > > >
>> > > > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > next
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > meeting, if you can join.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
>> > > > > > Chintalapani <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > kafka@harsha.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > Hi Grant,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > We are working on it. Will
>> add
>> > the
>> > > > > > details
>> > > > > > > > to
>> > > > > > > > > > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > about
>> > > > > > > > > >
>> > > > > > > > > > > > >> > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > request protocol.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > Harsha
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant
>> > Henke
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > Hi Parth,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > Are you still working on this? If you
>> > need
>> > > > any
>> > > > > > > help
>> > > > > > > > > > please
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > don't
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > hesitate
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > to ask.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > Grant
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun
>> > Rao <
>> > > > > > > > > >
>> > > > > > > > > > > jun@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > Parth,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > Thanks for the reply.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the
>> > renewal
>> > > > by
>> > > > > > > users
>> > > > > > > > > that
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> authenticated
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > using
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism.
>> > Then,
>> > > > > should
>> > > > > > > we
>> > > > > > > > > make
>> > > > > > > > > >
>> > > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> renewal a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > list?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > For example, in the case of rest
>> > proxy,
>> > > it
>> > > > > > will
>> > > > > > > be
>> > > > > > > > > >
>> > > > > > > > > > > useful
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > for
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> every
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able
>> to
>> > > renew
>> > > > > the
>> > > > > > > > > tokens.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > It would be clearer if we can
>> document
>> > > the
>> > > > > > > request
>> > > > > > > > > >
>> > > > > > > > > > > > protocol
>> > > > > > > > > >
>> > > > > > > > > > > > >> > like
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
>> > > > > > > > > uence/display/KAFKA/KIP-
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
>> > > > > centralized+administrative+
>> > > > > > > > > >
>> > > > > > > > > > > > operations#KIP-4-
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
>> > > > > > istrativeoperations-
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
>> > > > > > > > 2945):(VotedandPlannedforin0.
>> > > > > > > > > >
>> > > > > > > > > > > > 10.1.0)
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > .
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > It would also be useful to document
>> > the
>> > > > > client
>> > > > > > > > APIs.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM,
>> parth
>> > > > > > > brahmbhatt
>> > > > > > > > <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Hi,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only
>> > > allow
>> > > > > the
>> > > > > > > > > renewal
>> > > > > > > > > > by
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > users
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> that
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > authenticated using *non*
>> delegation
>> > > > token
>> > > > > > > > > > mechanism.
>> > > > > > > > > >
>> > > > > > > > > > > > For
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> example,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > If
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > user
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Alice authenticated using
>> kerberos
>> > and
>> > > > > > > requested
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> tokens,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > only
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
>> > > > > delegation
>> > > > > > > > token
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > mechanism
>> > > > > > > > > >
>> > > > > > > > > > > > >> > can
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > renew.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Clients that have access to
>> > > delegation
>> > > > > > tokens
>> > > > > > > > can
>> > > > > > > > > > not
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > issue
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > renewal
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > request for renewing their own
>> token
>> > > and
>> > > > > > this
>> > > > > > > is
>> > > > > > > > > >
>> > > > > > > > > > > > primarily
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > important
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > reduce the time window for which
>> a
>> > > > > > compromised
>> > > > > > > > > token
>> > > > > > > > > >
>> > > > > > > > > > > > will
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > be
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> valid.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated
>> > user
>> > > > can
>> > > > > > > > request
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > tokens
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > but
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > even here I would recommend to
>> avoid
>> > > > > > creating
>> > > > > > > a
>> > > > > > > > > > chain
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > where a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > client
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > authenticated via delegation
>> token
>> > > > request
>> > > > > > for
>> > > > > > > > > more
>> > > > > > > > > >
>> > > > > > > > > > > > >> > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > tokens.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Basically anyone can request
>> > > delegation
>> > > > > > token,
>> > > > > > > > as
>> > > > > > > > > > long
>> > > > > > > > > >
>> > > > > > > > > > > > as
>> > > > > > > > > >
>> > > > > > > > > > > > >> > they
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > authenticate
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > via a non delegation token
>> > mechanism.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
>> > > > > > > > > uence/display/KAFKA/KIP-
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
>> > > > > > > > > >
>> > > > > > > > > > > r+Kafka#KIP-48Delegationtokens
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> upportforKaf
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > ka-PublicInterfaces
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > sufficient?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Thanks
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > Parth
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM,
>> Jun
>> > > Rao
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > Parth,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple
>> of
>> > > > > comments
>> > > > > > > > > inline
>> > > > > > > > > >
>> > > > > > > > > > > > below.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36
>> AM,
>> > > > parth
>> > > > > > > > > > brahmbhatt <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com>
>> > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens
>> renewed?
>> > > By
>> > > > > > > original
>> > > > > > > > > >
>> > > > > > > > > > > > requester
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> only? or
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > using
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Kerberos
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > auth only?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do
>> this
>> > > only
>> > > > > > using
>> > > > > > > > > >
>> > > > > > > > > > > Kerberos
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > auth
>> > > > > > > > > >
>> > > > > > > > > > > > >> > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > only
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > threw
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > renewer specified during the
>> > > > > acquisition
>> > > > > > > > > > request.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow
>> this.
>> > > Are
>> > > > > you
>> > > > > > > > saying
>> > > > > > > > > >
>> > > > > > > > > > > that
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > any
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> client
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > authenticated with the
>> delegation
>> > > > token
>> > > > > > can
>> > > > > > > > > renew,
>> > > > > > > > > >
>> > > > > > > > > > > > i.e.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > there
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> is
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > no
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > renewer
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > needed?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
>> > > > > authenticated
>> > > > > > > > client
>> > > > > > > > > >
>> > > > > > > > > > > > (either
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> through
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > SASL
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > or
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation
>> > token
>> > > > for
>> > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > authenticated
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> user,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > right?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each
>> > > broker
>> > > > or
>> > > > > > in
>> > > > > > > > ZK?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to
>> > > store
>> > > > in
>> > > > > > ZK
>> > > > > > > or
>> > > > > > > > > not
>> > > > > > > > > >
>> > > > > > > > > > > > store
>> > > > > > > > > >
>> > > > > > > > > > > > >> > them
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> at
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > all.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > The
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > whole controller based
>> > > distribution
>> > > > is
>> > > > > > too
>> > > > > > > > > much
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > overhead
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> with
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > not
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > much
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > achieve.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens
>> invalidated /
>> > > > > expired?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time
>> out or
>> > > > > through
>> > > > > > > an
>> > > > > > > > > >
>> > > > > > > > > > > explicit
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> request to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > invalidate.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption
>> algorithm is
>> > > > used?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > SCRAM
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
>> > > > proposal
>> > > > > > (it
>> > > > > > > > > wasn't
>> > > > > > > > > >
>> > > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> but
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > was
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > discussed
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > in this thread)?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
>> > > proposal. I
>> > > > > > tried
>> > > > > > > > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > explained
>> > > > > > > > > >
>> > > > > > > > > > > > >> > how
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > its
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > different problem and why its
>> > not
>> > > > > really
>> > > > > > > > > > necessary
>> > > > > > > > > >
>> > > > > > > > > > > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> discuss
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > that
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > as
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > part
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will
>> not
>> > > > > support
>> > > > > > > any
>> > > > > > > > > >
>> > > > > > > > > > > > >> > impersonation,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> it
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > will
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > just
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > be
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if
>> so -
>> > > for
>> > > > > what
>> > > > > > > > > > actions?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > Could we document the format of
>> > the
>> > > > new
>> > > > > > > > > >
>> > > > > > > > > > > > request/response
>> > > > > > > > > >
>> > > > > > > > > > > > >> > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > their
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > associated Resource and
>> Operation
>> > > for
>> > > > > ACL?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation
>> > token
>> > > be
>> > > > > > > > > configured
>> > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> client?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I
>> > wasn't
>> > > > > > > planning
>> > > > > > > > on
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > supporting
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> JAAS
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > for
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > tokens.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does
>> this
>> > > > > either.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Thanks
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Parth
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03
>> PM,
>> > > Jun
>> > > > > > Rao <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > jun@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > Harsha,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > Another question.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation
>> > > token
>> > > > be
>> > > > > > > > > > configured
>> > > > > > > > > >
>> > > > > > > > > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > client?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > The
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > standard
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through
>> > JAAS.
>> > > > > > However,
>> > > > > > > > we
>> > > > > > > > > > will
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > need
>> > > > > > > > > >
>> > > > > > > > > > > > >> > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > think
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > through
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > if
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a
>> shared
>> > > > > > > > environment.
>> > > > > > > > > > For
>> > > > > > > > > >
>> > > > > > > > > > > > >> > example,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > when a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > new
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > task
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > is
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker
>> node,
>> > do
>> > > > we
>> > > > > > need
>> > > > > > > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > dynamically
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> add a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > new
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > section
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be
>> > more
>> > > > > > > > convenient
>> > > > > > > > > if
>> > > > > > > > > >
>> > > > > > > > > > > we
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > can
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> pass in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > token
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > through the config directly
>> > w/o
>> > > > > going
>> > > > > > > > > through
>> > > > > > > > > >
>> > > > > > > > > > > > JAAS.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still
>> > actively
>> > > > > > working
>> > > > > > > on
>> > > > > > > > > > this
>> > > > > > > > > >
>> > > > > > > > > > > > KIP?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at
>> 2:18
>> > PM,
>> > > > Jun
>> > > > > > > Rao <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> jun@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
>> > > document
>> > > > > the
>> > > > > > > > format
>> > > > > > > > > > of
>> > > > > > > > > >
>> > > > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > data
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > stored
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > ZK.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
>> > > > discussion
>> > > > > > on
>> > > > > > > > > > whether
>> > > > > > > > > >
>> > > > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > tokens
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > should
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > be
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK
>> like
>> > > > > > > > > config/acl/quota,
>> > > > > > > > > >
>> > > > > > > > > > > or
>> > > > > > > > > >
>> > > > > > > > > > > > >> > through
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > controller.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the
>> controller is
>> > > > only
>> > > > > > > > designed
>> > > > > > > > > > for
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> propagating
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > topic
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > metadata,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to
>> > send
>> > > > the
>> > > > > > > token
>> > > > > > > > > >
>> > > > > > > > > > > instead
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > of
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > DIGEST-MD5
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > since
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > it's
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the
>> wiki
>> > > > seem
>> > > > > > > > broken.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at
>> > 10:02
>> > > > AM,
>> > > > > > Gwen
>> > > > > > > > > >
>> > > > > > > > > > > Shapira <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
>> > > remaining
>> > > > > > > > questions
>> > > > > > > > > > are:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
>> > > > renewed?
>> > > > > By
>> > > > > > > > > > original
>> > > > > > > > > >
>> > > > > > > > > > > > >> > requester
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > only?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > or
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > using
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on
>> > each
>> > > > > broker
>> > > > > > > or
>> > > > > > > > in
>> > > > > > > > > > ZK?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
>> > > invalidated /
>> > > > > > > > expired?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption
>> > algorithm
>> > > > is
>> > > > > > > used?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the
>> > impersonation
>> > > > > > proposal
>> > > > > > > > (it
>> > > > > > > > > >
>> > > > > > > > > > > > wasn't
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > but
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > was
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this
>> thread)?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs,
>> if
>> > > so -
>> > > > > for
>> > > > > > > > what
>> > > > > > > > > >
>> > > > > > > > > > > > actions?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at
>> 7:48
>> > > PM,
>> > > > > > > Harsha
>> > > > > > > > <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> kafka@harsha.io>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
>> > > > > > > > Unfortunately
>> > > > > > > > > I
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> attend
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > meeting
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
>> > > when
>> > > > > > > > > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > tokens
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > discussed.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > Appreciate
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > if
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
>> > > you
>> > > > > can
>> > > > > > > > update
>> > > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > thread if
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > you
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > have
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > any
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
>> > > > > further
>> > > > > > > > > > questions.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016,
>> at
>> > > > 11:32
>> > > > > > AM,
>> > > > > > > > > Liquan
>> > > > > > > > > >
>> > > > > > > > > > > Pei
>> > > > > > > > > >
>> > > > > > > > > > > > >> > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the
>> links
>> > to
>> > > > > > images
>> > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > >
>> > > > > > > > > > > KIP
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> are
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > broken.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016
>> at
>> > > 9:33
>> > > > > AM,
>> > > > > > > > parth
>> > > > > > > > > >
>> > > > > > > > > > > > >> > brahmbhatt <
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
>> > > brahmbhatt.parth@gmail.com>
>> > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
>> > > > > > > > getDelegationTokenAs
>> > > > > > > > > >
>> > > > > > > > > > > mean?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current
>> > proposal
>> > > we
>> > > > > > only
>> > > > > > > > > allow
>> > > > > > > > > > a
>> > > > > > > > > >
>> > > > > > > > > > > > user
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> get
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > token
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> for
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that
>> it
>> > > > > > > authenticated
>> > > > > > > > > as
>> > > > > > > > > >
>> > > > > > > > > > > > using
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> another
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > mechanism,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > i.e.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> A user
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate
>> > using
>> > > a
>> > > > > > keytab
>> > > > > > > > for
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> will get
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
>> for
>> > > that
>> > > > > > user
>> > > > > > > > > only.
>> > > > > > > > > > In
>> > > > > > > > > >
>> > > > > > > > > > > > >> > future I
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > think
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > we
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > will
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > have
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such
>> > that
>> > > > we
>> > > > > > > allow
>> > > > > > > > > some
>> > > > > > > > > >
>> > > > > > > > > > > set
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> users (
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
>> > > > kafka-rest-user@EXAMPLE.COM
>> > > > > ,
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > acquire
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
>> on
>> > > > behalf
>> > > > > of
>> > > > > > > > other
>> > > > > > > > > >
>> > > > > > > > > > > users
>> > > > > > > > > >
>> > > > > > > > > > > > >> > whose
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > identity
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > they
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > have
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified
>> > independently.
>> > > > > Kafka
>> > > > > > > > > brokers
>> > > > > > > > > >
>> > > > > > > > > > > > will
>> > > > > > > > > >
>> > > > > > > > > > > > >> > have
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> ACLs
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > to
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > control
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> which
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed
>> to
>> > > > > > impersonate
>> > > > > > > > > other
>> > > > > > > > > >
>> > > > > > > > > > > > users
>> > > > > > > > > >
>> > > > > > > > > > > > >> > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> get
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > tokens
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > on
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
>> > > > Impersonation
>> > > > > > is a
>> > > > > > > > > whole
>> > > > > > > > > >
>> > > > > > > > > > > > >> > different
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > problem
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > in
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > my
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can
>> tackle
>> > it
>> > > > in
>> > > > > > > > separate
>> > > > > > > > > >
>> > > > > > > > > > > KIP.
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the
>> > typical
>> > > > rate
>> > > > > > of
>> > > > > > > > > > getting
>> > > > > > > > > >
>> > > > > > > > > > > > and
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> renewing
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > delegation
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this
>> should
>> > be
>> > > > > very
>> > > > > > > very
>> > > > > > > > > > low,
>> > > > > > > > > >
>> > > > > > > > > > > 1
>> > > > > > > > > >
>> > > > > > > > > > > > >> > request
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> per
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > minute
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > is a
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
>> > > estimate.
>> > > > > > > However
>> > > > > > > > it
>> > > > > > > > > >
>> > > > > > > > > > > > depends
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> the
>> > > > > > > > > >
>> > > > > > > > > > > > >> > >>> > > token
>> > > > > > > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi,
> Can you add a sample Jaas configuration using delegation tokens to the KIP?
>
Will add sample Jaas configuration.
> To make sure I have understood correctly, KAFKA-3712 is aimed at enabling a
> superuser to impersonate another (single) user, say alice. A producer using
> impersonation will authenticate with superuser credentials. All requests
> from the producer will be run with the principal alice. But alice is not
> involved in the authentication and alice's credentials are not actually
> provided to the broker?
>
>
Yes, this matches with my understanding of impersonation work . Even in
this approach
we have to handle all producer related issues mentioned by you. Yes, this
is big change
and can be implemented if there is a pressing need. I hope we are all in
agreement, that
this can be done in a separate KIP.
I request others give any suggestions/concerns on this KIP.
Thanks,
>
> On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > @Gwen, @Rajini,
> >
> > As mentioned in the KIP, main motivation for this KIP is to reduce load
> on
> > Kerberos
> > server on large kafka deployments with large number of clients.
> >
> > Also it looks like we are combining two overlapping concepts
> > 1. Single client sending requests with multiple users/authentications
> > 2. Impersonation
> >
> > Option 1, is definitely useful in some use cases and can be used to
> > implement workaround for
> > impersonation
> >
> > In Impersonation, a super user can send requests on behalf of another
> > user(Alice) in a secured way.
> > superuser has credentials but user Alice doesn't have any. The requests
> are
> > required
> > to run as user Alice and accesses/ACLs on Broker are required to be done
> as
> > user Alice.
> > It is required that user Alice can connect to the Broker on a connection
> > authenticated with
> > superuser's credentials. In other words superuser is impersonating the
> user
> > Alice.
> >
> > The approach mentioned by Harsha in previous mail is implemented in
> hadoop,
> > storm etc..
> >
> > Some more details here:
> > https://hadoop.apache.org/docs/r2.7.2/hadoop-project-
> > dist/hadoop-common/Superusers.html
> >
> >
> > @Rajini
> >
> > Thanks for your comments on SASL/SCRAM usage. I am thinking to send
> > tokenHmac (salted-hashed version)
> > as password for authentication and tokenID for retrial of tokenHmac at
> > server side.
> > Does above sound OK?
> >
> >
> > Thanks,
> > Manikumar
> >
> > On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> > > @Gwen @Mani Not sure why we want to authenticate at every request.
> Even
> > if
> > > the token exchange is cheap it still a few calls that need to go
> through
> > > round trip. Impersonation doesn't require authentication for every
> > > request.
> > >
> > > "So a centralized app can create few producers, do the metadata request
> > and
> > > broker discovery with its own user auth, but then use delegation tokens
> > to
> > > allow performing produce/fetch requests as different users? Instead of
> > > having to re-connect for each impersonated user?"
> > >
> > > Yes. But what we will have is this centralized user as impersonation
> user
> > > on behalf of other users. When it authenticates initially we will
> create
> > a
> > > "Subject" and from there on wards centralized user can do
> > > Subject.doAsPrivileged
> > > on behalf, other users.
> > > On the server side, we can retrieve two principals out of this one is
> the
> > > authenticated user (centralized user) and another is impersonated user.
> > We
> > > will first check if the authenticated user allowed to impersonate and
> > then
> > > move on to check if the user Alice has access to the topic "X" to
> > > read/write.
> > >
> > > @Rajini Intention of this KIP is to support token auth via SASL/SCRAM,
> > not
> > > just with TLS. What you raised is a good point let me take a look and
> > add
> > > details.
> > >
> > > It will be easier to add impersonation once we reach agreement on this
> > KIP.
> > >
> > >
> > > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > > Hi Rajini,
> > > >
> > > > I think it would definitely be valuable to have a KIP for
> > impersonation.
> > > >
> > > > Ismael
> > > >
> > > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rsivaram@pivotal.io
> >
> > > > wrote:
> > > >
> > > > > It would clearly be very useful to enable clients to send requests
> on
> > > > > behalf of multiple users. A separate KIP makes sense, but it may be
> > > worth
> > > > > thinking through some of the implications now, especially if the
> main
> > > > > interest in delegation tokens comes from its potential to enable
> > > > > impersonation.
> > > > >
> > > > > I understand that delegation tokens are only expected to be used
> with
> > > > TLS.
> > > > > But the choice of SASL/SCRAM for authentication must be based on a
> > > > > requirement to protect the tokenHmac - otherwise you could just use
> > > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> > > > on-the-wire,
> > > > > only a salted-hashed version of it is used in the SASL
> authentication
> > > > > exchange. If impersonation is based on sending tokenHmac in
> requests,
> > > any
> > > > > benefit of using SCRAM is lost.
> > > > >
> > > > > An alternative may be to allow clients to authenticate multiple
> times
> > > > using
> > > > > SASL and include one of its authenticated principals in each
> request
> > > > > (optionally). I haven't thought it through yet, obviously. But if
> the
> > > > > approach is of interest and no one is working on a KIP for
> > > impersonation
> > > > at
> > > > > the moment, I am happy to write one. It may provide something for
> > > > > comparison at least.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > >
> > > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <
> > manikumar.reddy@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > That's a good idea. Authenticating every request with delegation
> > > token
> > > > > will
> > > > > > be useful for
> > > > > > impersonation use-cases. But as of now, we are thinking
> delegation
> > > > token
> > > > > as
> > > > > > just another way
> > > > > > to authenticate the users. We haven't think through all the use
> > cases
> > > > > > related to
> > > > > > impersonation or using delegation token for impersonation. We
> want
> > to
> > > > > > handle impersonation
> > > > > > (KAFKA-3712) as part of separate KIP.
> > > > > >
> > > > > > Will that be Ok?
> > > > > >
> > > > > >
> > > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gwen@confluent.io
> >
> > > > wrote:
> > > > > >
> > > > > > > Thinking out loud here:
> > > > > > >
> > > > > > > It looks like authentication with a delegation token is going
> to
> > be
> > > > > > > super-cheap, right? We just compare the token to a value in the
> > > > broker
> > > > > > > cache?
> > > > > > >
> > > > > > > If I understood the KIP correctly, right now it suggests that
> > > > > > > authentication happens when establishing the client-broker
> > > connection
> > > > > (as
> > > > > > > normal for Kafka. But perhaps we want to consider
> authenticating
> > > > every
> > > > > > > request with delegation token (if exists)?
> > > > > > >
> > > > > > > So a centralized app can create few producers, do the metadata
> > > > request
> > > > > > and
> > > > > > > broker discovery with its own user auth, but then use
> delegation
> > > > tokens
> > > > > > to
> > > > > > > allow performing produce/fetch requests as different users?
> > Instead
> > > > of
> > > > > > > having to re-connect for each impersonated user?
> > > > > > >
> > > > > > > This may over-complicate things quite a bit (basically adding
> > extra
> > > > > > > information in every request), but maybe it will be useful for
> > > > > > > impersonation use-cases (which seem to drive much of the
> interest
> > > in
> > > > > this
> > > > > > > KIP)?
> > > > > > > Kafka Connect, NiFi and friends can probably use this to share
> > > > clients
> > > > > > > between multiple jobs, tasks, etc.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Gwen
> > > > > > >
> > > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> > > > manikumar.reddy@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Ashish,
> > > > > > > >
> > > > > > > > Thank you for reviewing the KIP. Please see the replies
> > inline.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 1. How to disable delegation token authentication?
> > > > > > > > >
> > > > > > > > > This can be achieved in various ways, however I think
> reusing
> > > > > > > delegation
> > > > > > > > > token secret config for this makes sense here. Avoids
> > creating
> > > > yet
> > > > > > > > another
> > > > > > > > > config and forces delegation token users to consciously set
> > the
> > > > > > secret.
> > > > > > > > If
> > > > > > > > > the secret is not set or set to empty string, brokers
> should
> > > turn
> > > > > off
> > > > > > > > > delegation token support. This will however require a new
> > error
> > > > > code
> > > > > > to
> > > > > > > > > indicate delegation token support is turned off on broker.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the suggestion. Option to turnoff delegation
> token
> > > > > > > > authentication will be useful.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 2. ACLs on delegation token?
> > > > > > > > >
> > > > > > > > > Do we need to have ACLs defined for tokens? I do not think
> it
> > > > buys
> > > > > us
> > > > > > > > > anything, as delegation token can be treated as
> impersonation
> > > of
> > > > > the
> > > > > > > > owner.
> > > > > > > > > Any thing the owner has permission to do, delegation tokens
> > > > should
> > > > > be
> > > > > > > > > allowed to do as well. If so, we probably won't need to
> > return
> > > > > > > > > authorization exception error code while creating
> delegation
> > > > token.
> > > > > > It
> > > > > > > > > however would make sense to check renew and expire requests
> > are
> > > > > > coming
> > > > > > > > from
> > > > > > > > > owner or renewers of the token, but that does not require
> > > > explicit
> > > > > > > acls.
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes, We agreed to not have new acl on who can request
> > delegation
> > > > > token.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 3. How to restrict max life time of a token?
> > > > > > > > >
> > > > > > > > > Admins might want to restrict max life time of tokens
> created
> > > on
> > > > a
> > > > > > > > cluster,
> > > > > > > > > and this can very from cluster to cluster based on
> use-cases.
> > > > This
> > > > > > > might
> > > > > > > > > warrant a separate broker config.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Currently we have "delegation.token.max.lifetime.sec"
> server
> > > > config
> > > > > > > > property
> > > > > > > > May be we can take min(User supplied MaxTime, Server MaxTime)
> > as
> > > > max
> > > > > > life
> > > > > > > > time.
> > > > > > > > I am open to add new config property.
> > > > > > > >
> > > > > > > > Few more comments based on recent KIP update.
> > > > > > > > >
> > > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't
> we
> > > use
> > > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything
> > > > before
> > > > > > > > current
> > > > > > > > > date?
> > > > > > > > >
> > > > > > > >
> > > > > > > > makes sense. we don't need special request to cancel the
> token.
> > > We
> > > > > can
> > > > > > > use
> > > > > > > > ExpireTokenRequest.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 2. Can we change time field names to indicate their unit is
> > > > > > > milliseconds,
> > > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Done.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 3. Can we allow users to renew a token for a specified
> amount
> > > of
> > > > > > time?
> > > > > > > In
> > > > > > > > > current version of KIP, renew request does not take time
> as a
> > > > > param,
> > > > > > > not
> > > > > > > > > sure what is expiry time set to after renewal.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Yes, we need to specify renew period. I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mankumar
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> > > > > manikumar.reddy@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I would like to reinitiate the discussion on Delegation
> > token
> > > > > > support
> > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > Kafka.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Brief summary of the past discussion:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All
> > brokers
> > > > > will
> > > > > > > > have a
> > > > > > > > > >
> > > > > > > > > > cache backed by
> > > > > > > > > >
> > > > > > > > > > zookeeper so they will all get notified whenever a new
> > > token
> > > > > is
> > > > > > > > > >
> > > > > > > > > > generated and they will
> > > > > > > > > >
> > > > > > > > > > update their local cache whenever token state changes.
> > > > > > > > > >
> > > > > > > > > > 2) The current proposal does not support rotation of
> secret
> > > > > > > > > >
> > > > > > > > > > 3) Only allow the renewal by users that authenticated
> using
> > > > *non*
> > > > > > > > > >
> > > > > > > > > > delegation token mechanism
> > > > > > > > > >
> > > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms.
> Kafka
> > > > > clients
> > > > > > > can
> > > > > > > > > >
> > > > > > > > > > authenticate using
> > > > > > > > > >
> > > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> > > > > password.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Updated the KIP with the following:
> > > > > > > > > >
> > > > > > > > > > 1. Protocol and Config changes
> > > > > > > > > >
> > > > > > > > > > 2. format of the data stored in ZK.
> > > > > > > > > >
> > > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > 48+Delegation+token+support+for+Kafka
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Jun, Ashish, Gwen,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Pl review the updated KIP.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > Manikumar
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> > > > > asingh@cloudera.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Harsha/ Gwen,
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > How do we proceed here? I am willing to help out with
> > here.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > > > > > gwen@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Is it updated? are all concerns addressed? do you
> want
> > to
> > > > > > start a
> > > > > > > > > vote?
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Sorry for being pushy, I do appreciate that we are
> all
> > > > > > volunteers
> > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > finding time is difficult. This feature is important
> > for
> > > > > > anything
> > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > integrates with Kafka (stream processors, Flume,
> NiFi,
> > > etc)
> > > > > > and I
> > > > > > > > > >
> > > > > > > > > > > > don't want to see this getting stuck because we lack
> > > > > > coordination
> > > > > > > > > >
> > > > > > > > > > > > within the community.
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani
> <
> > > > > > > > > kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > > The only pending update for the KIP is to write up
> > the
> > > > > > protocol
> > > > > > > > > > changes
> > > > > > > > > >
> > > > > > > > > > > > like
> > > > > > > > > >
> > > > > > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> I think we decided to not support secret
> rotation, I
> > > > guess
> > > > > > > this
> > > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> stated clearly on the KIP. Also, more details on
> how
> > > > > clients
> > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > perform
> > > > > > > > > >
> > > > > > > > > > > > >> token distribution and how CLI will look like will
> > be
> > > > > > helpful.
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > > > > > gwen@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > Hi Guys,
> > > > > > > > > >
> > > > > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > This discussion was dead for a while. Are there
> > > still
> > > > > > > > > contentious
> > > > > > > > > >
> > > > > > > > > > > > >> > points? If not, why are there no votes?
> > > > > > > > > >
> > > > > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > > > > > jun@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > > Ashish,
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next
> week
> > to
> > > > > > discuss
> > > > > > > > > > KIP-48
> > > > > > > > > >
> > > > > > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > other
> > > > > > > > > >
> > > > > > > > > > > > >> > > remaining KIPs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh
> <
> > > > > > > > > >
> > > > > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >> Thanks Harsha!
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
> > > > agenda.
> > > > > > > Also,
> > > > > > > > we
> > > > > > > > > > did
> > > > > > > > > >
> > > > > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >> actually make a call on when we should have
> > next
> > > > KIP
> > > > > > > call.
> > > > > > > > As
> > > > > > > > > >
> > > > > > > > > > > there
> > > > > > > > > >
> > > > > > > > > > > > >> > >> are
> > > > > > > > > >
> > > > > > > > > > > > >> > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >> few outstanding KIPs that could not be
> > discussed
> > > > this
> > > > > > > week,
> > > > > > > > > can
> > > > > > > > > >
> > > > > > > > > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >> have
> > > > > > > > > >
> > > > > > > > > > > > >> > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >> KIP hangout call next week?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> > > > Chintalapani
> > > > > > > > > >
> > > > > > > > > > > > >> > >> <ka...@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> Ashish,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> Yes we are working on it. Lets
> discuss
> > > in
> > > > > the
> > > > > > > next
> > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> meeting.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> I'll join.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> -Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish
> Singh
> > <
> > > > > > > > > >
> > > > > > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > Hello Harsha,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > Are you still working on this? Wondering
> if
> > we
> > > > can
> > > > > > > > discuss
> > > > > > > > > >
> > > > > > > > > > > this
> > > > > > > > > >
> > > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > next
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > meeting, if you can join.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > > > > > Chintalapani <
> > > > > > > > > >
> > > > > > > > > > > > >> > kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Hi Grant,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > We are working on it. Will add
> > the
> > > > > > details
> > > > > > > > to
> > > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > about
> > > > > > > > > >
> > > > > > > > > > > > >> > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > request protocol.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant
> > Henke
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Hi Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Are you still working on this? If you
> > need
> > > > any
> > > > > > > help
> > > > > > > > > > please
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > don't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > hesitate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > to ask.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Grant
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun
> > Rao <
> > > > > > > > > >
> > > > > > > > > > > jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the
> > renewal
> > > > by
> > > > > > > users
> > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> authenticated
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism.
> > Then,
> > > > > should
> > > > > > > we
> > > > > > > > > make
> > > > > > > > > >
> > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> renewal a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > list?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > For example, in the case of rest
> > proxy,
> > > it
> > > > > > will
> > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > useful
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> every
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to
> > > renew
> > > > > the
> > > > > > > > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It would be clearer if we can
> document
> > > the
> > > > > > > request
> > > > > > > > > >
> > > > > > > > > > > > protocol
> > > > > > > > > >
> > > > > > > > > > > > >> > like
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> > > > > centralized+administrative+
> > > > > > > > > >
> > > > > > > > > > > > operations#KIP-4-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > > > > > istrativeoperations-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > > > > > 2945):(VotedandPlannedforin0.
> > > > > > > > > >
> > > > > > > > > > > > 10.1.0)
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > .
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It would also be useful to document
> > the
> > > > > client
> > > > > > > > APIs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM,
> parth
> > > > > > > brahmbhatt
> > > > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only
> > > allow
> > > > > the
> > > > > > > > > renewal
> > > > > > > > > > by
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > authenticated using *non*
> delegation
> > > > token
> > > > > > > > > > mechanism.
> > > > > > > > > >
> > > > > > > > > > > > For
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> example,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > If
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos
> > and
> > > > > > > requested
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> tokens,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > only
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> > > > > delegation
> > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > mechanism
> > > > > > > > > >
> > > > > > > > > > > > >> > can
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > renew.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Clients that have access to
> > > delegation
> > > > > > tokens
> > > > > > > > can
> > > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > issue
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > renewal
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > request for renewing their own
> token
> > > and
> > > > > > this
> > > > > > > is
> > > > > > > > > >
> > > > > > > > > > > > primarily
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > important
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > > > > > compromised
> > > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> valid.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated
> > user
> > > > can
> > > > > > > > request
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > even here I would recommend to
> avoid
> > > > > > creating
> > > > > > > a
> > > > > > > > > > chain
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > where a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > client
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > authenticated via delegation token
> > > > request
> > > > > > for
> > > > > > > > > more
> > > > > > > > > >
> > > > > > > > > > > > >> > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Basically anyone can request
> > > delegation
> > > > > > token,
> > > > > > > > as
> > > > > > > > > > long
> > > > > > > > > >
> > > > > > > > > > > > as
> > > > > > > > > >
> > > > > > > > > > > > >> > they
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > authenticate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > via a non delegation token
> > mechanism.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > > > > > >
> > > > > > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> upportforKaf
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > sufficient?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Parth
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM,
> Jun
> > > Rao
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple
> of
> > > > > comments
> > > > > > > > > inline
> > > > > > > > > >
> > > > > > > > > > > > below.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36
> AM,
> > > > parth
> > > > > > > > > > brahmbhatt <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com>
> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens
> renewed?
> > > By
> > > > > > > original
> > > > > > > > > >
> > > > > > > > > > > > requester
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> only? or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do
> this
> > > only
> > > > > > using
> > > > > > > > > >
> > > > > > > > > > > Kerberos
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > auth
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > only
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > threw
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > renewer specified during the
> > > > > acquisition
> > > > > > > > > > request.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow
> this.
> > > Are
> > > > > you
> > > > > > > > saying
> > > > > > > > > >
> > > > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> client
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > authenticated with the
> delegation
> > > > token
> > > > > > can
> > > > > > > > > renew,
> > > > > > > > > >
> > > > > > > > > > > > i.e.
> > > > > > > > > >
> > > > > > > > > > > > >> > there
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> is
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > no
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > renewer
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > needed?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> > > > > authenticated
> > > > > > > > client
> > > > > > > > > >
> > > > > > > > > > > > (either
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > SASL
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation
> > token
> > > > for
> > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > authenticated
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> user,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > right?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each
> > > broker
> > > > or
> > > > > > in
> > > > > > > > ZK?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to
> > > store
> > > > in
> > > > > > ZK
> > > > > > > or
> > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > store
> > > > > > > > > >
> > > > > > > > > > > > >> > them
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> at
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > all.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > The
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > whole controller based
> > > distribution
> > > > is
> > > > > > too
> > > > > > > > > much
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > overhead
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> with
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > much
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated
> /
> > > > > expired?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time out
> or
> > > > > through
> > > > > > > an
> > > > > > > > > >
> > > > > > > > > > > explicit
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> request to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm
> is
> > > > used?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
> > > > proposal
> > > > > > (it
> > > > > > > > > wasn't
> > > > > > > > > >
> > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > was
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > discussed
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
> > > proposal. I
> > > > > > tried
> > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > explained
> > > > > > > > > >
> > > > > > > > > > > > >> > how
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > its
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > different problem and why its
> > not
> > > > > really
> > > > > > > > > > necessary
> > > > > > > > > >
> > > > > > > > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> discuss
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > as
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > part
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will
> not
> > > > > support
> > > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > impersonation,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> it
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > just
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so
> -
> > > for
> > > > > what
> > > > > > > > > > actions?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Could we document the format of
> > the
> > > > new
> > > > > > > > > >
> > > > > > > > > > > > request/response
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > their
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > associated Resource and
> Operation
> > > for
> > > > > ACL?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation
> > token
> > > be
> > > > > > > > > configured
> > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> client?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I
> > wasn't
> > > > > > > planning
> > > > > > > > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > supporting
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> JAAS
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does
> this
> > > > > either.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Parth
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03
> PM,
> > > Jun
> > > > > > Rao <
> > > > > > > > > >
> > > > > > > > > > > > >> > jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation
> > > token
> > > > be
> > > > > > > > > > configured
> > > > > > > > > >
> > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > client?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > The
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > standard
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through
> > JAAS.
> > > > > > However,
> > > > > > > > we
> > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > need
> > > > > > > > > >
> > > > > > > > > > > > >> > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > think
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a
> shared
> > > > > > > > environment.
> > > > > > > > > > For
> > > > > > > > > >
> > > > > > > > > > > > >> > example,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > when a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > new
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > task
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > is
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker
> node,
> > do
> > > > we
> > > > > > need
> > > > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> add a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > new
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > section
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be
> > more
> > > > > > > > convenient
> > > > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > can
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> pass in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > through the config directly
> > w/o
> > > > > going
> > > > > > > > > through
> > > > > > > > > >
> > > > > > > > > > > > JAAS.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still
> > actively
> > > > > > working
> > > > > > > on
> > > > > > > > > > this
> > > > > > > > > >
> > > > > > > > > > > > KIP?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18
> > PM,
> > > > Jun
> > > > > > > Rao <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
> > > document
> > > > > the
> > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > data
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > stored
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > ZK.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> > > > discussion
> > > > > > on
> > > > > > > > > > whether
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > should
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > > > > > config/acl/quota,
> > > > > > > > > >
> > > > > > > > > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > controller.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller
> is
> > > > only
> > > > > > > > designed
> > > > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> propagating
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > topic
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to
> > send
> > > > the
> > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > instead
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > since
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > it's
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the
> wiki
> > > > seem
> > > > > > > > broken.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at
> > 10:02
> > > > AM,
> > > > > > Gwen
> > > > > > > > > >
> > > > > > > > > > > Shapira <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
> > > remaining
> > > > > > > > questions
> > > > > > > > > > are:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
> > > > renewed?
> > > > > By
> > > > > > > > > > original
> > > > > > > > > >
> > > > > > > > > > > > >> > requester
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on
> > each
> > > > > broker
> > > > > > > or
> > > > > > > > in
> > > > > > > > > > ZK?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
> > > invalidated /
> > > > > > > > expired?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption
> > algorithm
> > > > is
> > > > > > > used?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the
> > impersonation
> > > > > > proposal
> > > > > > > > (it
> > > > > > > > > >
> > > > > > > > > > > > wasn't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > > > > > >
> > > > > > > > > > > > >> > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > was
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this
> thread)?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs,
> if
> > > so -
> > > > > for
> > > > > > > > what
> > > > > > > > > >
> > > > > > > > > > > > actions?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at
> 7:48
> > > PM,
> > > > > > > Harsha
> > > > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > > Unfortunately
> > > > > > > > > I
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> attend
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > when
> > > > > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > discussed.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > you
> > > > > can
> > > > > > > > update
> > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > thread if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > you
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > further
> > > > > > > > > > questions.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016,
> at
> > > > 11:32
> > > > > > AM,
> > > > > > > > > Liquan
> > > > > > > > > >
> > > > > > > > > > > Pei
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the
> links
> > to
> > > > > > images
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > broken.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016
> at
> > > 9:33
> > > > > AM,
> > > > > > > > parth
> > > > > > > > > >
> > > > > > > > > > > > >> > brahmbhatt <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > brahmbhatt.parth@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > > > > > getDelegationTokenAs
> > > > > > > > > >
> > > > > > > > > > > mean?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current
> > proposal
> > > we
> > > > > > only
> > > > > > > > > allow
> > > > > > > > > > a
> > > > > > > > > >
> > > > > > > > > > > > user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > > > > > authenticated
> > > > > > > > > as
> > > > > > > > > >
> > > > > > > > > > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> another
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > mechanism,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate
> > using
> > > a
> > > > > > keytab
> > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
> for
> > > that
> > > > > > user
> > > > > > > > > only.
> > > > > > > > > > In
> > > > > > > > > >
> > > > > > > > > > > > >> > future I
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > think
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such
> > that
> > > > we
> > > > > > > allow
> > > > > > > > > some
> > > > > > > > > >
> > > > > > > > > > > set
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> users (
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > kafka-rest-user@EXAMPLE.COM
> > > > > ,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > acquire
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on
> > > > behalf
> > > > > of
> > > > > > > > other
> > > > > > > > > >
> > > > > > > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > whose
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > identity
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > they
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified
> > independently.
> > > > > Kafka
> > > > > > > > > brokers
> > > > > > > > > >
> > > > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> ACLs
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > control
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > > > > > impersonate
> > > > > > > > > other
> > > > > > > > > >
> > > > > > > > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> > > > Impersonation
> > > > > > is a
> > > > > > > > > whole
> > > > > > > > > >
> > > > > > > > > > > > >> > different
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > problem
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > my
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can
> tackle
> > it
> > > > in
> > > > > > > > separate
> > > > > > > > > >
> > > > > > > > > > > KIP.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the
> > typical
> > > > rate
> > > > > > of
> > > > > > > > > > getting
> > > > > > > > > >
> > > > > > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> renewing
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this
> should
> > be
> > > > > very
> > > > > > > very
> > > > > > > > > > low,
> > > > > > > > > >
> > > > > > > > > > > 1
> > > > > > > > > >
> > > > > > > > > > > > >> > request
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> per
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > minute
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > is a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
> > > estimate.
> > > > > > > However
> > > > > > > > it
> > > > > > > > > >
> > > > > > > > > > > > depends
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > token
> > > > > > > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Rajini,
The use case you outline is indeed the one I was thinking of. The concern
is indeed what you pointed out, it could be a large change.
Ismael
On Thu, Dec 15, 2016 at 3:34 AM, Rajini Sivaram <rs...@pivotal.io> wrote:
> @Mani
>
> Can you add a sample Jaas configuration using delegation tokens to the KIP?
> Since delegation tokens will be handled differently from other SCRAM
> credentials, it should work anyway, but it will be good to see an example
> of the configuration the user provides. It sounds like users provide both
> tokenID and delegation token, but I wasn't sure.
>
> @Gwen, @Ismael, @Harsha, @Mani
>
> To make sure I have understood correctly, KAFKA-3712 is aimed at enabling a
> superuser to impersonate another (single) user, say alice. A producer using
> impersonation will authenticate with superuser credentials. All requests
> from the producer will be run with the principal alice. But alice is not
> involved in the authentication and alice's credentials are not actually
> provided to the broker?
>
> The use case I was thinking of was the other one on Mani's list above. I
> want to make sure that my understanding matches Gwen's and Ismael's for
> this one. Using REST proxy as an example, users sending requests to the
> REST proxy provide a delegation token as an auth token. REST proxy itself
> authenticates as a different user, perhaps with access to read metadata of
> all topics. But Kafka requests corresponding to each REST request should be
> authenticated based on the authentication token provided in the request.
> At the moment, the REST proxy has to create producers for each user (or
> perform authentication and authorization for the user). Instead we want to
> create a single producer which has the ability to send requests on behalf
> of multiple users where the user is authenticated by Kafka and each request
> from the producer is authorized based on the user who initiated the
> request.
>
> 1a) I think Gwen's suggestion was something along the lines of:
> producer.send(record1, delegationToken1);
> producer.send(record2, delegationToken2);
> 1b) I was thinking more along the lines of:
> subject1 = producer.authenticate(user1Credentials);
> subject2 = producer.authenticate(user2Credentials);
> ....
> Subject.doAs(subject1, (PrivilegedExceptionAction<Future<RecordMetadata>>)
> () -> producer.send(record1));
> Subject.doAs(subject2, (PrivilegedExceptionAction<Future<RecordMetadata>>)
> () -> producer.send(record2));
>
> To make either approach work, each request needs to be associated with a
> user. This would be delegation token in 1a) and user principal in 1b). And
> both need to ensure that producers dont send records from multiple users in
> one request. And if producers hold one metadata rather than one per-user,
> calls like partitionsFor() shouldn't return metadata for unauthorized
> topics. It is a very big change, so it will be good to know whether there
> is a real requirement to support this.
>
>
> On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > @Gwen, @Rajini,
> >
> > As mentioned in the KIP, main motivation for this KIP is to reduce load
> on
> > Kerberos
> > server on large kafka deployments with large number of clients.
> >
> > Also it looks like we are combining two overlapping concepts
> > 1. Single client sending requests with multiple users/authentications
> > 2. Impersonation
> >
> > Option 1, is definitely useful in some use cases and can be used to
> > implement workaround for
> > impersonation
> >
> > In Impersonation, a super user can send requests on behalf of another
> > user(Alice) in a secured way.
> > superuser has credentials but user Alice doesn't have any. The requests
> are
> > required
> > to run as user Alice and accesses/ACLs on Broker are required to be done
> as
> > user Alice.
> > It is required that user Alice can connect to the Broker on a connection
> > authenticated with
> > superuser's credentials. In other words superuser is impersonating the
> user
> > Alice.
> >
> > The approach mentioned by Harsha in previous mail is implemented in
> hadoop,
> > storm etc..
> >
> > Some more details here:
> > https://hadoop.apache.org/docs/r2.7.2/hadoop-project-
> > dist/hadoop-common/Superusers.html
> >
> >
> > @Rajini
> >
> > Thanks for your comments on SASL/SCRAM usage. I am thinking to send
> > tokenHmac (salted-hashed version)
> > as password for authentication and tokenID for retrial of tokenHmac at
> > server side.
> > Does above sound OK?
> >
> >
> > Thanks,
> > Manikumar
> >
> > On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> > > @Gwen @Mani Not sure why we want to authenticate at every request.
> Even
> > if
> > > the token exchange is cheap it still a few calls that need to go
> through
> > > round trip. Impersonation doesn't require authentication for every
> > > request.
> > >
> > > "So a centralized app can create few producers, do the metadata request
> > and
> > > broker discovery with its own user auth, but then use delegation tokens
> > to
> > > allow performing produce/fetch requests as different users? Instead of
> > > having to re-connect for each impersonated user?"
> > >
> > > Yes. But what we will have is this centralized user as impersonation
> user
> > > on behalf of other users. When it authenticates initially we will
> create
> > a
> > > "Subject" and from there on wards centralized user can do
> > > Subject.doAsPrivileged
> > > on behalf, other users.
> > > On the server side, we can retrieve two principals out of this one is
> the
> > > authenticated user (centralized user) and another is impersonated user.
> > We
> > > will first check if the authenticated user allowed to impersonate and
> > then
> > > move on to check if the user Alice has access to the topic "X" to
> > > read/write.
> > >
> > > @Rajini Intention of this KIP is to support token auth via SASL/SCRAM,
> > not
> > > just with TLS. What you raised is a good point let me take a look and
> > add
> > > details.
> > >
> > > It will be easier to add impersonation once we reach agreement on this
> > KIP.
> > >
> > >
> > > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > > Hi Rajini,
> > > >
> > > > I think it would definitely be valuable to have a KIP for
> > impersonation.
> > > >
> > > > Ismael
> > > >
> > > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rsivaram@pivotal.io
> >
> > > > wrote:
> > > >
> > > > > It would clearly be very useful to enable clients to send requests
> on
> > > > > behalf of multiple users. A separate KIP makes sense, but it may be
> > > worth
> > > > > thinking through some of the implications now, especially if the
> main
> > > > > interest in delegation tokens comes from its potential to enable
> > > > > impersonation.
> > > > >
> > > > > I understand that delegation tokens are only expected to be used
> with
> > > > TLS.
> > > > > But the choice of SASL/SCRAM for authentication must be based on a
> > > > > requirement to protect the tokenHmac - otherwise you could just use
> > > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> > > > on-the-wire,
> > > > > only a salted-hashed version of it is used in the SASL
> authentication
> > > > > exchange. If impersonation is based on sending tokenHmac in
> requests,
> > > any
> > > > > benefit of using SCRAM is lost.
> > > > >
> > > > > An alternative may be to allow clients to authenticate multiple
> times
> > > > using
> > > > > SASL and include one of its authenticated principals in each
> request
> > > > > (optionally). I haven't thought it through yet, obviously. But if
> the
> > > > > approach is of interest and no one is working on a KIP for
> > > impersonation
> > > > at
> > > > > the moment, I am happy to write one. It may provide something for
> > > > > comparison at least.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > >
> > > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <
> > manikumar.reddy@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > That's a good idea. Authenticating every request with delegation
> > > token
> > > > > will
> > > > > > be useful for
> > > > > > impersonation use-cases. But as of now, we are thinking
> delegation
> > > > token
> > > > > as
> > > > > > just another way
> > > > > > to authenticate the users. We haven't think through all the use
> > cases
> > > > > > related to
> > > > > > impersonation or using delegation token for impersonation. We
> want
> > to
> > > > > > handle impersonation
> > > > > > (KAFKA-3712) as part of separate KIP.
> > > > > >
> > > > > > Will that be Ok?
> > > > > >
> > > > > >
> > > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gwen@confluent.io
> >
> > > > wrote:
> > > > > >
> > > > > > > Thinking out loud here:
> > > > > > >
> > > > > > > It looks like authentication with a delegation token is going
> to
> > be
> > > > > > > super-cheap, right? We just compare the token to a value in the
> > > > broker
> > > > > > > cache?
> > > > > > >
> > > > > > > If I understood the KIP correctly, right now it suggests that
> > > > > > > authentication happens when establishing the client-broker
> > > connection
> > > > > (as
> > > > > > > normal for Kafka. But perhaps we want to consider
> authenticating
> > > > every
> > > > > > > request with delegation token (if exists)?
> > > > > > >
> > > > > > > So a centralized app can create few producers, do the metadata
> > > > request
> > > > > > and
> > > > > > > broker discovery with its own user auth, but then use
> delegation
> > > > tokens
> > > > > > to
> > > > > > > allow performing produce/fetch requests as different users?
> > Instead
> > > > of
> > > > > > > having to re-connect for each impersonated user?
> > > > > > >
> > > > > > > This may over-complicate things quite a bit (basically adding
> > extra
> > > > > > > information in every request), but maybe it will be useful for
> > > > > > > impersonation use-cases (which seem to drive much of the
> interest
> > > in
> > > > > this
> > > > > > > KIP)?
> > > > > > > Kafka Connect, NiFi and friends can probably use this to share
> > > > clients
> > > > > > > between multiple jobs, tasks, etc.
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Gwen
> > > > > > >
> > > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> > > > manikumar.reddy@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Ashish,
> > > > > > > >
> > > > > > > > Thank you for reviewing the KIP. Please see the replies
> > inline.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 1. How to disable delegation token authentication?
> > > > > > > > >
> > > > > > > > > This can be achieved in various ways, however I think
> reusing
> > > > > > > delegation
> > > > > > > > > token secret config for this makes sense here. Avoids
> > creating
> > > > yet
> > > > > > > > another
> > > > > > > > > config and forces delegation token users to consciously set
> > the
> > > > > > secret.
> > > > > > > > If
> > > > > > > > > the secret is not set or set to empty string, brokers
> should
> > > turn
> > > > > off
> > > > > > > > > delegation token support. This will however require a new
> > error
> > > > > code
> > > > > > to
> > > > > > > > > indicate delegation token support is turned off on broker.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the suggestion. Option to turnoff delegation
> token
> > > > > > > > authentication will be useful.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 2. ACLs on delegation token?
> > > > > > > > >
> > > > > > > > > Do we need to have ACLs defined for tokens? I do not think
> it
> > > > buys
> > > > > us
> > > > > > > > > anything, as delegation token can be treated as
> impersonation
> > > of
> > > > > the
> > > > > > > > owner.
> > > > > > > > > Any thing the owner has permission to do, delegation tokens
> > > > should
> > > > > be
> > > > > > > > > allowed to do as well. If so, we probably won't need to
> > return
> > > > > > > > > authorization exception error code while creating
> delegation
> > > > token.
> > > > > > It
> > > > > > > > > however would make sense to check renew and expire requests
> > are
> > > > > > coming
> > > > > > > > from
> > > > > > > > > owner or renewers of the token, but that does not require
> > > > explicit
> > > > > > > acls.
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes, We agreed to not have new acl on who can request
> > delegation
> > > > > token.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > 3. How to restrict max life time of a token?
> > > > > > > > >
> > > > > > > > > Admins might want to restrict max life time of tokens
> created
> > > on
> > > > a
> > > > > > > > cluster,
> > > > > > > > > and this can very from cluster to cluster based on
> use-cases.
> > > > This
> > > > > > > might
> > > > > > > > > warrant a separate broker config.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Currently we have "delegation.token.max.lifetime.sec"
> server
> > > > config
> > > > > > > > property
> > > > > > > > May be we can take min(User supplied MaxTime, Server MaxTime)
> > as
> > > > max
> > > > > > life
> > > > > > > > time.
> > > > > > > > I am open to add new config property.
> > > > > > > >
> > > > > > > > Few more comments based on recent KIP update.
> > > > > > > > >
> > > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't
> we
> > > use
> > > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything
> > > > before
> > > > > > > > current
> > > > > > > > > date?
> > > > > > > > >
> > > > > > > >
> > > > > > > > makes sense. we don't need special request to cancel the
> token.
> > > We
> > > > > can
> > > > > > > use
> > > > > > > > ExpireTokenRequest.
> > > > > > > > I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 2. Can we change time field names to indicate their unit is
> > > > > > > milliseconds,
> > > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Done.
> > > > > > > >
> > > > > > > >
> > > > > > > > > 3. Can we allow users to renew a token for a specified
> amount
> > > of
> > > > > > time?
> > > > > > > In
> > > > > > > > > current version of KIP, renew request does not take time
> as a
> > > > > param,
> > > > > > > not
> > > > > > > > > sure what is expiry time set to after renewal.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Yes, we need to specify renew period. I'll update the KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mankumar
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> > > > > manikumar.reddy@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I would like to reinitiate the discussion on Delegation
> > token
> > > > > > support
> > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > Kafka.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Brief summary of the past discussion:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All
> > brokers
> > > > > will
> > > > > > > > have a
> > > > > > > > > >
> > > > > > > > > > cache backed by
> > > > > > > > > >
> > > > > > > > > > zookeeper so they will all get notified whenever a new
> > > token
> > > > > is
> > > > > > > > > >
> > > > > > > > > > generated and they will
> > > > > > > > > >
> > > > > > > > > > update their local cache whenever token state changes.
> > > > > > > > > >
> > > > > > > > > > 2) The current proposal does not support rotation of
> secret
> > > > > > > > > >
> > > > > > > > > > 3) Only allow the renewal by users that authenticated
> using
> > > > *non*
> > > > > > > > > >
> > > > > > > > > > delegation token mechanism
> > > > > > > > > >
> > > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms.
> Kafka
> > > > > clients
> > > > > > > can
> > > > > > > > > >
> > > > > > > > > > authenticate using
> > > > > > > > > >
> > > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> > > > > password.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Updated the KIP with the following:
> > > > > > > > > >
> > > > > > > > > > 1. Protocol and Config changes
> > > > > > > > > >
> > > > > > > > > > 2. format of the data stored in ZK.
> > > > > > > > > >
> > > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > 48+Delegation+token+support+for+Kafka
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Jun, Ashish, Gwen,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Pl review the updated KIP.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > Manikumar
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> > > > > asingh@cloudera.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Harsha/ Gwen,
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > How do we proceed here? I am willing to help out with
> > here.
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > > > > > gwen@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Is it updated? are all concerns addressed? do you
> want
> > to
> > > > > > start a
> > > > > > > > > vote?
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > Sorry for being pushy, I do appreciate that we are
> all
> > > > > > volunteers
> > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > finding time is difficult. This feature is important
> > for
> > > > > > anything
> > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > integrates with Kafka (stream processors, Flume,
> NiFi,
> > > etc)
> > > > > > and I
> > > > > > > > > >
> > > > > > > > > > > > don't want to see this getting stuck because we lack
> > > > > > coordination
> > > > > > > > > >
> > > > > > > > > > > > within the community.
> > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani
> <
> > > > > > > > > kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > > The only pending update for the KIP is to write up
> > the
> > > > > > protocol
> > > > > > > > > > changes
> > > > > > > > > >
> > > > > > > > > > > > like
> > > > > > > > > >
> > > > > > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> I think we decided to not support secret
> rotation, I
> > > > guess
> > > > > > > this
> > > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> stated clearly on the KIP. Also, more details on
> how
> > > > > clients
> > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > perform
> > > > > > > > > >
> > > > > > > > > > > > >> token distribution and how CLI will look like will
> > be
> > > > > > helpful.
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > > > > > gwen@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > Hi Guys,
> > > > > > > > > >
> > > > > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > This discussion was dead for a while. Are there
> > > still
> > > > > > > > > contentious
> > > > > > > > > >
> > > > > > > > > > > > >> > points? If not, why are there no votes?
> > > > > > > > > >
> > > > > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > > > > > jun@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > > Ashish,
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next
> week
> > to
> > > > > > discuss
> > > > > > > > > > KIP-48
> > > > > > > > > >
> > > > > > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > other
> > > > > > > > > >
> > > > > > > > > > > > >> > > remaining KIPs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh
> <
> > > > > > > > > >
> > > > > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >> Thanks Harsha!
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
> > > > agenda.
> > > > > > > Also,
> > > > > > > > we
> > > > > > > > > > did
> > > > > > > > > >
> > > > > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >> actually make a call on when we should have
> > next
> > > > KIP
> > > > > > > call.
> > > > > > > > As
> > > > > > > > > >
> > > > > > > > > > > there
> > > > > > > > > >
> > > > > > > > > > > > >> > >> are
> > > > > > > > > >
> > > > > > > > > > > > >> > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >> few outstanding KIPs that could not be
> > discussed
> > > > this
> > > > > > > week,
> > > > > > > > > can
> > > > > > > > > >
> > > > > > > > > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >> have
> > > > > > > > > >
> > > > > > > > > > > > >> > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >> KIP hangout call next week?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> > > > Chintalapani
> > > > > > > > > >
> > > > > > > > > > > > >> > >> <ka...@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> Ashish,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> Yes we are working on it. Lets
> discuss
> > > in
> > > > > the
> > > > > > > next
> > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> meeting.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> I'll join.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> -Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish
> Singh
> > <
> > > > > > > > > >
> > > > > > > > > > > > asingh@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > Hello Harsha,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > Are you still working on this? Wondering
> if
> > we
> > > > can
> > > > > > > > discuss
> > > > > > > > > >
> > > > > > > > > > > this
> > > > > > > > > >
> > > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > next
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > meeting, if you can join.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > > > > > Chintalapani <
> > > > > > > > > >
> > > > > > > > > > > > >> > kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Hi Grant,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > We are working on it. Will add
> > the
> > > > > > details
> > > > > > > > to
> > > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > about
> > > > > > > > > >
> > > > > > > > > > > > >> > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > request protocol.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant
> > Henke
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Hi Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Are you still working on this? If you
> > need
> > > > any
> > > > > > > help
> > > > > > > > > > please
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > don't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > hesitate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > to ask.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > Grant
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun
> > Rao <
> > > > > > > > > >
> > > > > > > > > > > jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the
> > renewal
> > > > by
> > > > > > > users
> > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> authenticated
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism.
> > Then,
> > > > > should
> > > > > > > we
> > > > > > > > > make
> > > > > > > > > >
> > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> renewal a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > list?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > For example, in the case of rest
> > proxy,
> > > it
> > > > > > will
> > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > useful
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> every
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to
> > > renew
> > > > > the
> > > > > > > > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It would be clearer if we can
> document
> > > the
> > > > > > > request
> > > > > > > > > >
> > > > > > > > > > > > protocol
> > > > > > > > > >
> > > > > > > > > > > > >> > like
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> > > > > centralized+administrative+
> > > > > > > > > >
> > > > > > > > > > > > operations#KIP-4-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > > > > > istrativeoperations-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > > > > > 2945):(VotedandPlannedforin0.
> > > > > > > > > >
> > > > > > > > > > > > 10.1.0)
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > .
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > It would also be useful to document
> > the
> > > > > client
> > > > > > > > APIs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM,
> parth
> > > > > > > brahmbhatt
> > > > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only
> > > allow
> > > > > the
> > > > > > > > > renewal
> > > > > > > > > > by
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > authenticated using *non*
> delegation
> > > > token
> > > > > > > > > > mechanism.
> > > > > > > > > >
> > > > > > > > > > > > For
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> example,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > If
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos
> > and
> > > > > > > requested
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> tokens,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > only
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> > > > > delegation
> > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > mechanism
> > > > > > > > > >
> > > > > > > > > > > > >> > can
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > renew.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Clients that have access to
> > > delegation
> > > > > > tokens
> > > > > > > > can
> > > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > issue
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > renewal
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > request for renewing their own
> token
> > > and
> > > > > > this
> > > > > > > is
> > > > > > > > > >
> > > > > > > > > > > > primarily
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > important
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > > > > > compromised
> > > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> valid.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated
> > user
> > > > can
> > > > > > > > request
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > even here I would recommend to
> avoid
> > > > > > creating
> > > > > > > a
> > > > > > > > > > chain
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > where a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > client
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > authenticated via delegation token
> > > > request
> > > > > > for
> > > > > > > > > more
> > > > > > > > > >
> > > > > > > > > > > > >> > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Basically anyone can request
> > > delegation
> > > > > > token,
> > > > > > > > as
> > > > > > > > > > long
> > > > > > > > > >
> > > > > > > > > > > > as
> > > > > > > > > >
> > > > > > > > > > > > >> > they
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > authenticate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > via a non delegation token
> > mechanism.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > > > > > >
> > > > > > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> upportforKaf
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > sufficient?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > Parth
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM,
> Jun
> > > Rao
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Parth,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple
> of
> > > > > comments
> > > > > > > > > inline
> > > > > > > > > >
> > > > > > > > > > > > below.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36
> AM,
> > > > parth
> > > > > > > > > > brahmbhatt <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com>
> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens
> renewed?
> > > By
> > > > > > > original
> > > > > > > > > >
> > > > > > > > > > > > requester
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> only? or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do
> this
> > > only
> > > > > > using
> > > > > > > > > >
> > > > > > > > > > > Kerberos
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > auth
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > only
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > threw
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > renewer specified during the
> > > > > acquisition
> > > > > > > > > > request.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow
> this.
> > > Are
> > > > > you
> > > > > > > > saying
> > > > > > > > > >
> > > > > > > > > > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> client
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > authenticated with the
> delegation
> > > > token
> > > > > > can
> > > > > > > > > renew,
> > > > > > > > > >
> > > > > > > > > > > > i.e.
> > > > > > > > > >
> > > > > > > > > > > > >> > there
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> is
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > no
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > renewer
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > needed?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> > > > > authenticated
> > > > > > > > client
> > > > > > > > > >
> > > > > > > > > > > > (either
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > SASL
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation
> > token
> > > > for
> > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > authenticated
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> user,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > right?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each
> > > broker
> > > > or
> > > > > > in
> > > > > > > > ZK?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to
> > > store
> > > > in
> > > > > > ZK
> > > > > > > or
> > > > > > > > > not
> > > > > > > > > >
> > > > > > > > > > > > store
> > > > > > > > > >
> > > > > > > > > > > > >> > them
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> at
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > all.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > The
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > whole controller based
> > > distribution
> > > > is
> > > > > > too
> > > > > > > > > much
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > overhead
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> with
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > not
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > much
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated
> /
> > > > > expired?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time out
> or
> > > > > through
> > > > > > > an
> > > > > > > > > >
> > > > > > > > > > > explicit
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> request to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm
> is
> > > > used?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
> > > > proposal
> > > > > > (it
> > > > > > > > > wasn't
> > > > > > > > > >
> > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > was
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > discussed
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
> > > proposal. I
> > > > > > tried
> > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > explained
> > > > > > > > > >
> > > > > > > > > > > > >> > how
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > its
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > different problem and why its
> > not
> > > > > really
> > > > > > > > > > necessary
> > > > > > > > > >
> > > > > > > > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> discuss
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > that
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > as
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > part
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will
> not
> > > > > support
> > > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > impersonation,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> it
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > just
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so
> -
> > > for
> > > > > what
> > > > > > > > > > actions?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > Could we document the format of
> > the
> > > > new
> > > > > > > > > >
> > > > > > > > > > > > request/response
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > their
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > associated Resource and
> Operation
> > > for
> > > > > ACL?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation
> > token
> > > be
> > > > > > > > > configured
> > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> client?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I
> > wasn't
> > > > > > > planning
> > > > > > > > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > supporting
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> JAAS
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > tokens.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does
> this
> > > > > either.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Parth
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03
> PM,
> > > Jun
> > > > > > Rao <
> > > > > > > > > >
> > > > > > > > > > > > >> > jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation
> > > token
> > > > be
> > > > > > > > > > configured
> > > > > > > > > >
> > > > > > > > > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > client?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > The
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > standard
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through
> > JAAS.
> > > > > > However,
> > > > > > > > we
> > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > need
> > > > > > > > > >
> > > > > > > > > > > > >> > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > think
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a
> shared
> > > > > > > > environment.
> > > > > > > > > > For
> > > > > > > > > >
> > > > > > > > > > > > >> > example,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > when a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > new
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > task
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > is
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker
> node,
> > do
> > > > we
> > > > > > need
> > > > > > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> add a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > new
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > section
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be
> > more
> > > > > > > > convenient
> > > > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > can
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> pass in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > through the config directly
> > w/o
> > > > > going
> > > > > > > > > through
> > > > > > > > > >
> > > > > > > > > > > > JAAS.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still
> > actively
> > > > > > working
> > > > > > > on
> > > > > > > > > > this
> > > > > > > > > >
> > > > > > > > > > > > KIP?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18
> > PM,
> > > > Jun
> > > > > > > Rao <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> jun@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
> > > document
> > > > > the
> > > > > > > > format
> > > > > > > > > > of
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > data
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > stored
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > ZK.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> > > > discussion
> > > > > > on
> > > > > > > > > > whether
> > > > > > > > > >
> > > > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > should
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > be
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > > > > > config/acl/quota,
> > > > > > > > > >
> > > > > > > > > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > through
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > controller.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller
> is
> > > > only
> > > > > > > > designed
> > > > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> propagating
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > topic
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to
> > send
> > > > the
> > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > instead
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > since
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > it's
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the
> wiki
> > > > seem
> > > > > > > > broken.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at
> > 10:02
> > > > AM,
> > > > > > Gwen
> > > > > > > > > >
> > > > > > > > > > > Shapira <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
> > > remaining
> > > > > > > > questions
> > > > > > > > > > are:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
> > > > renewed?
> > > > > By
> > > > > > > > > > original
> > > > > > > > > >
> > > > > > > > > > > > >> > requester
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > or
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on
> > each
> > > > > broker
> > > > > > > or
> > > > > > > > in
> > > > > > > > > > ZK?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
> > > invalidated /
> > > > > > > > expired?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption
> > algorithm
> > > > is
> > > > > > > used?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the
> > impersonation
> > > > > > proposal
> > > > > > > > (it
> > > > > > > > > >
> > > > > > > > > > > > wasn't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > > > > > >
> > > > > > > > > > > > >> > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > but
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > was
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this
> thread)?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs,
> if
> > > so -
> > > > > for
> > > > > > > > what
> > > > > > > > > >
> > > > > > > > > > > > actions?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at
> 7:48
> > > PM,
> > > > > > > Harsha
> > > > > > > > <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> kafka@harsha.io>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > > Unfortunately
> > > > > > > > > I
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> attend
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > when
> > > > > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > discussed.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > you
> > > > > can
> > > > > > > > update
> > > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > > >> > thread if
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > you
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > any
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > further
> > > > > > > > > > questions.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016,
> at
> > > > 11:32
> > > > > > AM,
> > > > > > > > > Liquan
> > > > > > > > > >
> > > > > > > > > > > Pei
> > > > > > > > > >
> > > > > > > > > > > > >> > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the
> links
> > to
> > > > > > images
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > >
> > > > > > > > > > > KIP
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > broken.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016
> at
> > > 9:33
> > > > > AM,
> > > > > > > > parth
> > > > > > > > > >
> > > > > > > > > > > > >> > brahmbhatt <
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > brahmbhatt.parth@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > > > > > getDelegationTokenAs
> > > > > > > > > >
> > > > > > > > > > > mean?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current
> > proposal
> > > we
> > > > > > only
> > > > > > > > > allow
> > > > > > > > > > a
> > > > > > > > > >
> > > > > > > > > > > > user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > token
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > > > > > authenticated
> > > > > > > > > as
> > > > > > > > > >
> > > > > > > > > > > > using
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> another
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > mechanism,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate
> > using
> > > a
> > > > > > keytab
> > > > > > > > for
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens
> for
> > > that
> > > > > > user
> > > > > > > > > only.
> > > > > > > > > > In
> > > > > > > > > >
> > > > > > > > > > > > >> > future I
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > think
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > we
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such
> > that
> > > > we
> > > > > > > allow
> > > > > > > > > some
> > > > > > > > > >
> > > > > > > > > > > set
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> users (
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > kafka-rest-user@EXAMPLE.COM
> > > > > ,
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > acquire
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on
> > > > behalf
> > > > > of
> > > > > > > > other
> > > > > > > > > >
> > > > > > > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > whose
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > identity
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > they
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified
> > independently.
> > > > > Kafka
> > > > > > > > > brokers
> > > > > > > > > >
> > > > > > > > > > > > will
> > > > > > > > > >
> > > > > > > > > > > > >> > have
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> ACLs
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > to
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > control
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > > > > > impersonate
> > > > > > > > > other
> > > > > > > > > >
> > > > > > > > > > > > users
> > > > > > > > > >
> > > > > > > > > > > > >> > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> get
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > tokens
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> > > > Impersonation
> > > > > > is a
> > > > > > > > > whole
> > > > > > > > > >
> > > > > > > > > > > > >> > different
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > problem
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > in
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > my
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can
> tackle
> > it
> > > > in
> > > > > > > > separate
> > > > > > > > > >
> > > > > > > > > > > KIP.
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the
> > typical
> > > > rate
> > > > > > of
> > > > > > > > > > getting
> > > > > > > > > >
> > > > > > > > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> renewing
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this
> should
> > be
> > > > > very
> > > > > > > very
> > > > > > > > > > low,
> > > > > > > > > >
> > > > > > > > > > > 1
> > > > > > > > > >
> > > > > > > > > > > > >> > request
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> per
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > minute
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > is a
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
> > > estimate.
> > > > > > > However
> > > > > > > > it
> > > > > > > > > >
> > > > > > > > > > > > depends
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> the
> > > > > > > > > >
> > > > > > > > > > > > >> > >>> > > token
> > > > > > > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Rajini Sivaram <rs...@pivotal.io>.
@Mani
Can you add a sample Jaas configuration using delegation tokens to the KIP?
Since delegation tokens will be handled differently from other SCRAM
credentials, it should work anyway, but it will be good to see an example
of the configuration the user provides. It sounds like users provide both
tokenID and delegation token, but I wasn't sure.
@Gwen, @Ismael, @Harsha, @Mani
To make sure I have understood correctly, KAFKA-3712 is aimed at enabling a
superuser to impersonate another (single) user, say alice. A producer using
impersonation will authenticate with superuser credentials. All requests
from the producer will be run with the principal alice. But alice is not
involved in the authentication and alice's credentials are not actually
provided to the broker?
The use case I was thinking of was the other one on Mani's list above. I
want to make sure that my understanding matches Gwen's and Ismael's for
this one. Using REST proxy as an example, users sending requests to the
REST proxy provide a delegation token as an auth token. REST proxy itself
authenticates as a different user, perhaps with access to read metadata of
all topics. But Kafka requests corresponding to each REST request should be
authenticated based on the authentication token provided in the request.
At the moment, the REST proxy has to create producers for each user (or
perform authentication and authorization for the user). Instead we want to
create a single producer which has the ability to send requests on behalf
of multiple users where the user is authenticated by Kafka and each request
from the producer is authorized based on the user who initiated the request.
1a) I think Gwen's suggestion was something along the lines of:
producer.send(record1, delegationToken1);
producer.send(record2, delegationToken2);
1b) I was thinking more along the lines of:
subject1 = producer.authenticate(user1Credentials);
subject2 = producer.authenticate(user2Credentials);
....
Subject.doAs(subject1, (PrivilegedExceptionAction<Future<RecordMetadata>>)
() -> producer.send(record1));
Subject.doAs(subject2, (PrivilegedExceptionAction<Future<RecordMetadata>>)
() -> producer.send(record2));
To make either approach work, each request needs to be associated with a
user. This would be delegation token in 1a) and user principal in 1b). And
both need to ensure that producers dont send records from multiple users in
one request. And if producers hold one metadata rather than one per-user,
calls like partitionsFor() shouldn't return metadata for unauthorized
topics. It is a very big change, so it will be good to know whether there
is a real requirement to support this.
On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <ma...@gmail.com>
wrote:
> @Gwen, @Rajini,
>
> As mentioned in the KIP, main motivation for this KIP is to reduce load on
> Kerberos
> server on large kafka deployments with large number of clients.
>
> Also it looks like we are combining two overlapping concepts
> 1. Single client sending requests with multiple users/authentications
> 2. Impersonation
>
> Option 1, is definitely useful in some use cases and can be used to
> implement workaround for
> impersonation
>
> In Impersonation, a super user can send requests on behalf of another
> user(Alice) in a secured way.
> superuser has credentials but user Alice doesn't have any. The requests are
> required
> to run as user Alice and accesses/ACLs on Broker are required to be done as
> user Alice.
> It is required that user Alice can connect to the Broker on a connection
> authenticated with
> superuser's credentials. In other words superuser is impersonating the user
> Alice.
>
> The approach mentioned by Harsha in previous mail is implemented in hadoop,
> storm etc..
>
> Some more details here:
> https://hadoop.apache.org/docs/r2.7.2/hadoop-project-
> dist/hadoop-common/Superusers.html
>
>
> @Rajini
>
> Thanks for your comments on SASL/SCRAM usage. I am thinking to send
> tokenHmac (salted-hashed version)
> as password for authentication and tokenID for retrial of tokenHmac at
> server side.
> Does above sound OK?
>
>
> Thanks,
> Manikumar
>
> On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > @Gwen @Mani Not sure why we want to authenticate at every request. Even
> if
> > the token exchange is cheap it still a few calls that need to go through
> > round trip. Impersonation doesn't require authentication for every
> > request.
> >
> > "So a centralized app can create few producers, do the metadata request
> and
> > broker discovery with its own user auth, but then use delegation tokens
> to
> > allow performing produce/fetch requests as different users? Instead of
> > having to re-connect for each impersonated user?"
> >
> > Yes. But what we will have is this centralized user as impersonation user
> > on behalf of other users. When it authenticates initially we will create
> a
> > "Subject" and from there on wards centralized user can do
> > Subject.doAsPrivileged
> > on behalf, other users.
> > On the server side, we can retrieve two principals out of this one is the
> > authenticated user (centralized user) and another is impersonated user.
> We
> > will first check if the authenticated user allowed to impersonate and
> then
> > move on to check if the user Alice has access to the topic "X" to
> > read/write.
> >
> > @Rajini Intention of this KIP is to support token auth via SASL/SCRAM,
> not
> > just with TLS. What you raised is a good point let me take a look and
> add
> > details.
> >
> > It will be easier to add impersonation once we reach agreement on this
> KIP.
> >
> >
> > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Hi Rajini,
> > >
> > > I think it would definitely be valuable to have a KIP for
> impersonation.
> > >
> > > Ismael
> > >
> > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rs...@pivotal.io>
> > > wrote:
> > >
> > > > It would clearly be very useful to enable clients to send requests on
> > > > behalf of multiple users. A separate KIP makes sense, but it may be
> > worth
> > > > thinking through some of the implications now, especially if the main
> > > > interest in delegation tokens comes from its potential to enable
> > > > impersonation.
> > > >
> > > > I understand that delegation tokens are only expected to be used with
> > > TLS.
> > > > But the choice of SASL/SCRAM for authentication must be based on a
> > > > requirement to protect the tokenHmac - otherwise you could just use
> > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> > > on-the-wire,
> > > > only a salted-hashed version of it is used in the SASL authentication
> > > > exchange. If impersonation is based on sending tokenHmac in requests,
> > any
> > > > benefit of using SCRAM is lost.
> > > >
> > > > An alternative may be to allow clients to authenticate multiple times
> > > using
> > > > SASL and include one of its authenticated principals in each request
> > > > (optionally). I haven't thought it through yet, obviously. But if the
> > > > approach is of interest and no one is working on a KIP for
> > impersonation
> > > at
> > > > the moment, I am happy to write one. It may provide something for
> > > > comparison at least.
> > > >
> > > > Thoughts?
> > > >
> > > >
> > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <
> manikumar.reddy@gmail.com>
> > > > wrote:
> > > >
> > > > > That's a good idea. Authenticating every request with delegation
> > token
> > > > will
> > > > > be useful for
> > > > > impersonation use-cases. But as of now, we are thinking delegation
> > > token
> > > > as
> > > > > just another way
> > > > > to authenticate the users. We haven't think through all the use
> cases
> > > > > related to
> > > > > impersonation or using delegation token for impersonation. We want
> to
> > > > > handle impersonation
> > > > > (KAFKA-3712) as part of separate KIP.
> > > > >
> > > > > Will that be Ok?
> > > > >
> > > > >
> > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > > >
> > > > > > Thinking out loud here:
> > > > > >
> > > > > > It looks like authentication with a delegation token is going to
> be
> > > > > > super-cheap, right? We just compare the token to a value in the
> > > broker
> > > > > > cache?
> > > > > >
> > > > > > If I understood the KIP correctly, right now it suggests that
> > > > > > authentication happens when establishing the client-broker
> > connection
> > > > (as
> > > > > > normal for Kafka. But perhaps we want to consider authenticating
> > > every
> > > > > > request with delegation token (if exists)?
> > > > > >
> > > > > > So a centralized app can create few producers, do the metadata
> > > request
> > > > > and
> > > > > > broker discovery with its own user auth, but then use delegation
> > > tokens
> > > > > to
> > > > > > allow performing produce/fetch requests as different users?
> Instead
> > > of
> > > > > > having to re-connect for each impersonated user?
> > > > > >
> > > > > > This may over-complicate things quite a bit (basically adding
> extra
> > > > > > information in every request), but maybe it will be useful for
> > > > > > impersonation use-cases (which seem to drive much of the interest
> > in
> > > > this
> > > > > > KIP)?
> > > > > > Kafka Connect, NiFi and friends can probably use this to share
> > > clients
> > > > > > between multiple jobs, tasks, etc.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Gwen
> > > > > >
> > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> > > manikumar.reddy@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Ashish,
> > > > > > >
> > > > > > > Thank you for reviewing the KIP. Please see the replies
> inline.
> > > > > > >
> > > > > > >
> > > > > > > > 1. How to disable delegation token authentication?
> > > > > > > >
> > > > > > > > This can be achieved in various ways, however I think reusing
> > > > > > delegation
> > > > > > > > token secret config for this makes sense here. Avoids
> creating
> > > yet
> > > > > > > another
> > > > > > > > config and forces delegation token users to consciously set
> the
> > > > > secret.
> > > > > > > If
> > > > > > > > the secret is not set or set to empty string, brokers should
> > turn
> > > > off
> > > > > > > > delegation token support. This will however require a new
> error
> > > > code
> > > > > to
> > > > > > > > indicate delegation token support is turned off on broker.
> > > > > > > >
> > > > > > >
> > > > > > > Thanks for the suggestion. Option to turnoff delegation token
> > > > > > > authentication will be useful.
> > > > > > > I'll update the KIP.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > 2. ACLs on delegation token?
> > > > > > > >
> > > > > > > > Do we need to have ACLs defined for tokens? I do not think it
> > > buys
> > > > us
> > > > > > > > anything, as delegation token can be treated as impersonation
> > of
> > > > the
> > > > > > > owner.
> > > > > > > > Any thing the owner has permission to do, delegation tokens
> > > should
> > > > be
> > > > > > > > allowed to do as well. If so, we probably won't need to
> return
> > > > > > > > authorization exception error code while creating delegation
> > > token.
> > > > > It
> > > > > > > > however would make sense to check renew and expire requests
> are
> > > > > coming
> > > > > > > from
> > > > > > > > owner or renewers of the token, but that does not require
> > > explicit
> > > > > > acls.
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Yes, We agreed to not have new acl on who can request
> delegation
> > > > token.
> > > > > > > I'll update the KIP.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > 3. How to restrict max life time of a token?
> > > > > > > >
> > > > > > > > Admins might want to restrict max life time of tokens created
> > on
> > > a
> > > > > > > cluster,
> > > > > > > > and this can very from cluster to cluster based on use-cases.
> > > This
> > > > > > might
> > > > > > > > warrant a separate broker config.
> > > > > > > >
> > > > > > > >
> > > > > > > Currently we have "delegation.token.max.lifetime.sec" server
> > > config
> > > > > > > property
> > > > > > > May be we can take min(User supplied MaxTime, Server MaxTime)
> as
> > > max
> > > > > life
> > > > > > > time.
> > > > > > > I am open to add new config property.
> > > > > > >
> > > > > > > Few more comments based on recent KIP update.
> > > > > > > >
> > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we
> > use
> > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything
> > > before
> > > > > > > current
> > > > > > > > date?
> > > > > > > >
> > > > > > >
> > > > > > > makes sense. we don't need special request to cancel the token.
> > We
> > > > can
> > > > > > use
> > > > > > > ExpireTokenRequest.
> > > > > > > I'll update the KIP.
> > > > > > >
> > > > > > >
> > > > > > > > 2. Can we change time field names to indicate their unit is
> > > > > > milliseconds,
> > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > > > > >
> > > > > > > >
> > > > > > > Done.
> > > > > > >
> > > > > > >
> > > > > > > > 3. Can we allow users to renew a token for a specified amount
> > of
> > > > > time?
> > > > > > In
> > > > > > > > current version of KIP, renew request does not take time as a
> > > > param,
> > > > > > not
> > > > > > > > sure what is expiry time set to after renewal.
> > > > > > > >
> > > > > > > >
> > > > > > > Yes, we need to specify renew period. I'll update the KIP.
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mankumar
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> > > > manikumar.reddy@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I would like to reinitiate the discussion on Delegation
> token
> > > > > support
> > > > > > > for
> > > > > > > > >
> > > > > > > > > Kafka.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Brief summary of the past discussion:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All
> brokers
> > > > will
> > > > > > > have a
> > > > > > > > >
> > > > > > > > > cache backed by
> > > > > > > > >
> > > > > > > > > zookeeper so they will all get notified whenever a new
> > token
> > > > is
> > > > > > > > >
> > > > > > > > > generated and they will
> > > > > > > > >
> > > > > > > > > update their local cache whenever token state changes.
> > > > > > > > >
> > > > > > > > > 2) The current proposal does not support rotation of secret
> > > > > > > > >
> > > > > > > > > 3) Only allow the renewal by users that authenticated using
> > > *non*
> > > > > > > > >
> > > > > > > > > delegation token mechanism
> > > > > > > > >
> > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka
> > > > clients
> > > > > > can
> > > > > > > > >
> > > > > > > > > authenticate using
> > > > > > > > >
> > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> > > > password.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Updated the KIP with the following:
> > > > > > > > >
> > > > > > > > > 1. Protocol and Config changes
> > > > > > > > >
> > > > > > > > > 2. format of the data stored in ZK.
> > > > > > > > >
> > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > > > >
> > > > > > > > > 48+Delegation+token+support+for+Kafka
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Jun, Ashish, Gwen,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Pl review the updated KIP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > Manikumar
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> > > > asingh@cloudera.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Harsha/ Gwen,
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > How do we proceed here? I am willing to help out with
> here.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > > > > gwen@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > Is it updated? are all concerns addressed? do you want
> to
> > > > > start a
> > > > > > > > vote?
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > Sorry for being pushy, I do appreciate that we are all
> > > > > volunteers
> > > > > > > and
> > > > > > > > >
> > > > > > > > > > > finding time is difficult. This feature is important
> for
> > > > > anything
> > > > > > > > that
> > > > > > > > >
> > > > > > > > > > > integrates with Kafka (stream processors, Flume, NiFi,
> > etc)
> > > > > and I
> > > > > > > > >
> > > > > > > > > > > don't want to see this getting stuck because we lack
> > > > > coordination
> > > > > > > > >
> > > > > > > > > > > within the community.
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > > > > > > kafka@harsha.io>
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > > The only pending update for the KIP is to write up
> the
> > > > > protocol
> > > > > > > > > changes
> > > > > > > > >
> > > > > > > > > > > like
> > > > > > > > >
> > > > > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > > > > asingh@cloudera.com>
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> I think we decided to not support secret rotation, I
> > > guess
> > > > > > this
> > > > > > > > can
> > > > > > > > > be
> > > > > > > > >
> > > > > > > > > > > >> stated clearly on the KIP. Also, more details on how
> > > > clients
> > > > > > > will
> > > > > > > > >
> > > > > > > > > > > perform
> > > > > > > > >
> > > > > > > > > > > >> token distribution and how CLI will look like will
> be
> > > > > helpful.
> > > > > > > > >
> > > > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > > > > gwen@confluent.io>
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> > Hi Guys,
> > > > > > > > >
> > > > > > > > > > > >> >
> > > > > > > > >
> > > > > > > > > > > >> > This discussion was dead for a while. Are there
> > still
> > > > > > > > contentious
> > > > > > > > >
> > > > > > > > > > > >> > points? If not, why are there no votes?
> > > > > > > > >
> > > > > > > > > > > >> >
> > > > > > > > >
> > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > > > > jun@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > > Ashish,
> > > > > > > > >
> > > > > > > > > > > >> > >
> > > > > > > > >
> > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next week
> to
> > > > > discuss
> > > > > > > > > KIP-48
> > > > > > > > >
> > > > > > > > > > > and
> > > > > > > > >
> > > > > > > > > > > >> > other
> > > > > > > > >
> > > > > > > > > > > >> > > remaining KIPs.
> > > > > > > > >
> > > > > > > > > > > >> > >
> > > > > > > > >
> > > > > > > > > > > >> > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >
> > > > > > > > >
> > > > > > > > > > > >> > > Jun
> > > > > > > > >
> > > > > > > > > > > >> > >
> > > > > > > > >
> > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > > > > > > >
> > > > > > > > > > asingh@cloudera.com>
> > > > > > > > >
> > > > > > > > > > > >> > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >
> > > > > > > > >
> > > > > > > > > > > >> > >> Thanks Harsha!
> > > > > > > > >
> > > > > > > > > > > >> > >>
> > > > > > > > >
> > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
> > > agenda.
> > > > > > Also,
> > > > > > > we
> > > > > > > > > did
> > > > > > > > >
> > > > > > > > > > > not
> > > > > > > > >
> > > > > > > > > > > >> > >> actually make a call on when we should have
> next
> > > KIP
> > > > > > call.
> > > > > > > As
> > > > > > > > >
> > > > > > > > > > there
> > > > > > > > >
> > > > > > > > > > > >> > >> are
> > > > > > > > >
> > > > > > > > > > > >> > a
> > > > > > > > >
> > > > > > > > > > > >> > >> few outstanding KIPs that could not be
> discussed
> > > this
> > > > > > week,
> > > > > > > > can
> > > > > > > > >
> > > > > > > > > > we
> > > > > > > > >
> > > > > > > > > > > >> > >> have
> > > > > > > > >
> > > > > > > > > > > >> > a
> > > > > > > > >
> > > > > > > > > > > >> > >> KIP hangout call next week?
> > > > > > > > >
> > > > > > > > > > > >> > >>
> > > > > > > > >
> > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> > > Chintalapani
> > > > > > > > >
> > > > > > > > > > > >> > >> <ka...@harsha.io>
> > > > > > > > >
> > > > > > > > > > > >> > >> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> Ashish,
> > > > > > > > >
> > > > > > > > > > > >> > >>> Yes we are working on it. Lets discuss
> > in
> > > > the
> > > > > > next
> > > > > > > > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> meeting.
> > > > > > > > >
> > > > > > > > > > > >> > >>> I'll join.
> > > > > > > > >
> > > > > > > > > > > >> > >>> -Harsha
> > > > > > > > >
> > > > > > > > > > > >> > >>>
> > > > > > > > >
> > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh
> <
> > > > > > > > >
> > > > > > > > > > > asingh@cloudera.com>
> > > > > > > > >
> > > > > > > > > > > >> > >>> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > Hello Harsha,
> > > > > > > > >
> > > > > > > > > > > >> > >>> >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > Are you still working on this? Wondering if
> we
> > > can
> > > > > > > discuss
> > > > > > > > >
> > > > > > > > > > this
> > > > > > > > >
> > > > > > > > > > > in
> > > > > > > > >
> > > > > > > > > > > >> > next
> > > > > > > > >
> > > > > > > > > > > >> > >>> KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> > meeting, if you can join.
> > > > > > > > >
> > > > > > > > > > > >> > >>> >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > > > > Chintalapani <
> > > > > > > > >
> > > > > > > > > > > >> > kafka@harsha.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > Hi Grant,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > We are working on it. Will add
> the
> > > > > details
> > > > > > > to
> > > > > > > > > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > about
> > > > > > > > >
> > > > > > > > > > > >> > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > request protocol.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > Harsha
> > > > > > > > >
> > > > > > > > > > > >> > >>> > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant
> Henke
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > > > > >
> > > > > > > > > > > >> > >>> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > Hi Parth,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > Are you still working on this? If you
> need
> > > any
> > > > > > help
> > > > > > > > > please
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > don't
> > > > > > > > >
> > > > > > > > > > > >> > >>> > hesitate
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > to ask.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > Grant
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun
> Rao <
> > > > > > > > >
> > > > > > > > > > jun@confluent.io>
> > > > > > > > >
> > > > > > > > > > > >> > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > Parth,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the
> renewal
> > > by
> > > > > > users
> > > > > > > > that
> > > > > > > > >
> > > > > > > > > > > >> > >>> authenticated
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > using
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism.
> Then,
> > > > should
> > > > > > we
> > > > > > > > make
> > > > > > > > >
> > > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> renewal a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > list?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > For example, in the case of rest
> proxy,
> > it
> > > > > will
> > > > > > be
> > > > > > > > >
> > > > > > > > > > useful
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > for
> > > > > > > > >
> > > > > > > > > > > >> > >>> every
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to
> > renew
> > > > the
> > > > > > > > tokens.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > It would be clearer if we can document
> > the
> > > > > > request
> > > > > > > > >
> > > > > > > > > > > protocol
> > > > > > > > >
> > > > > > > > > > > >> > like
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > >
> > > > > > > > > > > >> > >>> > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> > > > centralized+administrative+
> > > > > > > > >
> > > > > > > > > > > operations#KIP-4-
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > > > > istrativeoperations-
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > > > > 2945):(VotedandPlannedforin0.
> > > > > > > > >
> > > > > > > > > > > 10.1.0)
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > .
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > It would also be useful to document
> the
> > > > client
> > > > > > > APIs.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > Jun
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> > > > > > brahmbhatt
> > > > > > > <
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Hi,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only
> > allow
> > > > the
> > > > > > > > renewal
> > > > > > > > > by
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > users
> > > > > > > > >
> > > > > > > > > > > >> > >>> that
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > authenticated using *non* delegation
> > > token
> > > > > > > > > mechanism.
> > > > > > > > >
> > > > > > > > > > > For
> > > > > > > > >
> > > > > > > > > > > >> > >>> example,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > If
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > user
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos
> and
> > > > > > requested
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > >
> > > > > > > > > > > >> > >>> tokens,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > only
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> > > > delegation
> > > > > > > token
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > mechanism
> > > > > > > > >
> > > > > > > > > > > >> > can
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > renew.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Clients that have access to
> > delegation
> > > > > tokens
> > > > > > > can
> > > > > > > > > not
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > issue
> > > > > > > > >
> > > > > > > > > > > >> > >>> > renewal
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > request for renewing their own token
> > and
> > > > > this
> > > > > > is
> > > > > > > > >
> > > > > > > > > > > primarily
> > > > > > > > >
> > > > > > > > > > > >> > >>> > important
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > > > > compromised
> > > > > > > > token
> > > > > > > > >
> > > > > > > > > > > will
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > be
> > > > > > > > >
> > > > > > > > > > > >> > >>> valid.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated
> user
> > > can
> > > > > > > request
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > >
> > > > > > > > > > > >> > >>> > tokens
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > but
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > even here I would recommend to avoid
> > > > > creating
> > > > > > a
> > > > > > > > > chain
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > where a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > client
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > authenticated via delegation token
> > > request
> > > > > for
> > > > > > > > more
> > > > > > > > >
> > > > > > > > > > > >> > delegation
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > tokens.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Basically anyone can request
> > delegation
> > > > > token,
> > > > > > > as
> > > > > > > > > long
> > > > > > > > >
> > > > > > > > > > > as
> > > > > > > > >
> > > > > > > > > > > >> > they
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > authenticate
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > via a non delegation token
> mechanism.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > <
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > > uence/display/KAFKA/KIP-
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > > > > >
> > > > > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > > > > >
> > > > > > > > > > > >> > >>> upportforKaf
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > sufficient?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Thanks
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > Parth
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun
> > Rao
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > Parth,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of
> > > > comments
> > > > > > > > inline
> > > > > > > > >
> > > > > > > > > > > below.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM,
> > > parth
> > > > > > > > > brahmbhatt <
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com>
> wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed?
> > By
> > > > > > original
> > > > > > > > >
> > > > > > > > > > > requester
> > > > > > > > >
> > > > > > > > > > > >> > >>> only? or
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > using
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do this
> > only
> > > > > using
> > > > > > > > >
> > > > > > > > > > Kerberos
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > auth
> > > > > > > > >
> > > > > > > > > > > >> > and
> > > > > > > > >
> > > > > > > > > > > >> > >>> > only
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > threw
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > renewer specified during the
> > > > acquisition
> > > > > > > > > request.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this.
> > Are
> > > > you
> > > > > > > saying
> > > > > > > > >
> > > > > > > > > > that
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > any
> > > > > > > > >
> > > > > > > > > > > >> > >>> client
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > authenticated with the delegation
> > > token
> > > > > can
> > > > > > > > renew,
> > > > > > > > >
> > > > > > > > > > > i.e.
> > > > > > > > >
> > > > > > > > > > > >> > there
> > > > > > > > >
> > > > > > > > > > > >> > >>> is
> > > > > > > > >
> > > > > > > > > > > >> > >>> > no
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > renewer
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > needed?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> > > > authenticated
> > > > > > > client
> > > > > > > > >
> > > > > > > > > > > (either
> > > > > > > > >
> > > > > > > > > > > >> > >>> through
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > SASL
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > or
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation
> token
> > > for
> > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > authenticated
> > > > > > > > >
> > > > > > > > > > > >> > >>> user,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > right?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each
> > broker
> > > or
> > > > > in
> > > > > > > ZK?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to
> > store
> > > in
> > > > > ZK
> > > > > > or
> > > > > > > > not
> > > > > > > > >
> > > > > > > > > > > store
> > > > > > > > >
> > > > > > > > > > > >> > them
> > > > > > > > >
> > > > > > > > > > > >> > >>> at
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > all.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > The
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > whole controller based
> > distribution
> > > is
> > > > > too
> > > > > > > > much
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > overhead
> > > > > > > > >
> > > > > > > > > > > >> > >>> with
> > > > > > > > >
> > > > > > > > > > > >> > >>> > not
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > much
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated /
> > > > expired?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time out or
> > > > through
> > > > > > an
> > > > > > > > >
> > > > > > > > > > explicit
> > > > > > > > >
> > > > > > > > > > > >> > >>> request to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is
> > > used?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
> > > proposal
> > > > > (it
> > > > > > > > wasn't
> > > > > > > > >
> > > > > > > > > > in
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> but
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > was
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > discussed
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
> > proposal. I
> > > > > tried
> > > > > > > and
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > explained
> > > > > > > > >
> > > > > > > > > > > >> > how
> > > > > > > > >
> > > > > > > > > > > >> > >>> > its
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > different problem and why its
> not
> > > > really
> > > > > > > > > necessary
> > > > > > > > >
> > > > > > > > > > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> discuss
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > that
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > as
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > part
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not
> > > > support
> > > > > > any
> > > > > > > > >
> > > > > > > > > > > >> > impersonation,
> > > > > > > > >
> > > > > > > > > > > >> > >>> it
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > will
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > just
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > be
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so -
> > for
> > > > what
> > > > > > > > > actions?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > Could we document the format of
> the
> > > new
> > > > > > > > >
> > > > > > > > > > > request/response
> > > > > > > > >
> > > > > > > > > > > >> > and
> > > > > > > > >
> > > > > > > > > > > >> > >>> > their
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > associated Resource and Operation
> > for
> > > > ACL?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation
> token
> > be
> > > > > > > > configured
> > > > > > > > > in
> > > > > > > > >
> > > > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> client?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I
> wasn't
> > > > > > planning
> > > > > > > on
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > supporting
> > > > > > > > >
> > > > > > > > > > > >> > >>> JAAS
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > for
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > tokens.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does this
> > > > either.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Parth
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM,
> > Jun
> > > > > Rao <
> > > > > > > > >
> > > > > > > > > > > >> > jun@confluent.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation
> > token
> > > be
> > > > > > > > > configured
> > > > > > > > >
> > > > > > > > > > in
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > client?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > The
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > standard
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through
> JAAS.
> > > > > However,
> > > > > > > we
> > > > > > > > > will
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > need
> > > > > > > > >
> > > > > > > > > > > >> > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > think
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > through
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > if
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > > > > > > environment.
> > > > > > > > > For
> > > > > > > > >
> > > > > > > > > > > >> > example,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > when a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > new
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > task
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > is
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker node,
> do
> > > we
> > > > > need
> > > > > > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > > > > >
> > > > > > > > > > > >> > >>> add a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > new
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > section
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be
> more
> > > > > > > convenient
> > > > > > > > if
> > > > > > > > >
> > > > > > > > > > we
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > can
> > > > > > > > >
> > > > > > > > > > > >> > >>> pass in
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > token
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > through the config directly
> w/o
> > > > going
> > > > > > > > through
> > > > > > > > >
> > > > > > > > > > > JAAS.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still
> actively
> > > > > working
> > > > > > on
> > > > > > > > > this
> > > > > > > > >
> > > > > > > > > > > KIP?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18
> PM,
> > > Jun
> > > > > > Rao <
> > > > > > > > >
> > > > > > > > > > > >> > >>> jun@confluent.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
> > document
> > > > the
> > > > > > > format
> > > > > > > > > of
> > > > > > > > >
> > > > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > data
> > > > > > > > >
> > > > > > > > > > > >> > >>> > stored
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > in
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > ZK.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> > > discussion
> > > > > on
> > > > > > > > > whether
> > > > > > > > >
> > > > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > tokens
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > should
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > be
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > > > > config/acl/quota,
> > > > > > > > >
> > > > > > > > > > or
> > > > > > > > >
> > > > > > > > > > > >> > through
> > > > > > > > >
> > > > > > > > > > > >> > >>> the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > controller.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller is
> > > only
> > > > > > > designed
> > > > > > > > > for
> > > > > > > > >
> > > > > > > > > > > >> > >>> propagating
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > topic
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to
> send
> > > the
> > > > > > token
> > > > > > > > >
> > > > > > > > > > instead
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > of
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > since
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > it's
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki
> > > seem
> > > > > > > broken.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at
> 10:02
> > > AM,
> > > > > Gwen
> > > > > > > > >
> > > > > > > > > > Shapira <
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
> > remaining
> > > > > > > questions
> > > > > > > > > are:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
> > > renewed?
> > > > By
> > > > > > > > > original
> > > > > > > > >
> > > > > > > > > > > >> > requester
> > > > > > > > >
> > > > > > > > > > > >> > >>> > only?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > or
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > using
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on
> each
> > > > broker
> > > > > > or
> > > > > > > in
> > > > > > > > > ZK?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
> > invalidated /
> > > > > > > expired?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption
> algorithm
> > > is
> > > > > > used?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the
> impersonation
> > > > > proposal
> > > > > > > (it
> > > > > > > > >
> > > > > > > > > > > wasn't
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > > > > >
> > > > > > > > > > > >> > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > but
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > was
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if
> > so -
> > > > for
> > > > > > > what
> > > > > > > > >
> > > > > > > > > > > actions?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48
> > PM,
> > > > > > Harsha
> > > > > > > <
> > > > > > > > >
> > > > > > > > > > > >> > >>> kafka@harsha.io>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > Unfortunately
> > > > > > > > I
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > > > > >
> > > > > > > > > > > >> > >>> attend
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > when
> > > > > > > > delegation
> > > > > > > > >
> > > > > > > > > > > tokens
> > > > > > > > >
> > > > > > > > > > > >> > >>> > discussed.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > if
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > you
> > > > can
> > > > > > > update
> > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > >> > thread if
> > > > > > > > >
> > > > > > > > > > > >> > >>> > you
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > have
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > any
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > further
> > > > > > > > > questions.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at
> > > 11:32
> > > > > AM,
> > > > > > > > Liquan
> > > > > > > > >
> > > > > > > > > > Pei
> > > > > > > > >
> > > > > > > > > > > >> > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the links
> to
> > > > > images
> > > > > > in
> > > > > > > > the
> > > > > > > > >
> > > > > > > > > > KIP
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > > > > >
> > > > > > > > > > > >> > >>> > broken.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at
> > 9:33
> > > > AM,
> > > > > > > parth
> > > > > > > > >
> > > > > > > > > > > >> > brahmbhatt <
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > brahmbhatt.parth@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > > > > getDelegationTokenAs
> > > > > > > > >
> > > > > > > > > > mean?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current
> proposal
> > we
> > > > > only
> > > > > > > > allow
> > > > > > > > > a
> > > > > > > > >
> > > > > > > > > > > user
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> get
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > token
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > > > > authenticated
> > > > > > > > as
> > > > > > > > >
> > > > > > > > > > > using
> > > > > > > > >
> > > > > > > > > > > >> > >>> another
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > mechanism,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate
> using
> > a
> > > > > keytab
> > > > > > > for
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for
> > that
> > > > > user
> > > > > > > > only.
> > > > > > > > > In
> > > > > > > > >
> > > > > > > > > > > >> > future I
> > > > > > > > >
> > > > > > > > > > > >> > >>> > think
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > we
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > will
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > have
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such
> that
> > > we
> > > > > > allow
> > > > > > > > some
> > > > > > > > >
> > > > > > > > > > set
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > > > > >
> > > > > > > > > > > >> > >>> users (
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > kafka-rest-user@EXAMPLE.COM
> > > > ,
> > > > > > > > >
> > > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > acquire
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on
> > > behalf
> > > > of
> > > > > > > other
> > > > > > > > >
> > > > > > > > > > users
> > > > > > > > >
> > > > > > > > > > > >> > whose
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > identity
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > they
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > have
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified
> independently.
> > > > Kafka
> > > > > > > > brokers
> > > > > > > > >
> > > > > > > > > > > will
> > > > > > > > >
> > > > > > > > > > > >> > have
> > > > > > > > >
> > > > > > > > > > > >> > >>> ACLs
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > to
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > control
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > > > > impersonate
> > > > > > > > other
> > > > > > > > >
> > > > > > > > > > > users
> > > > > > > > >
> > > > > > > > > > > >> > and
> > > > > > > > >
> > > > > > > > > > > >> > >>> get
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > tokens
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > on
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> > > Impersonation
> > > > > is a
> > > > > > > > whole
> > > > > > > > >
> > > > > > > > > > > >> > different
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > problem
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > in
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > my
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle
> it
> > > in
> > > > > > > separate
> > > > > > > > >
> > > > > > > > > > KIP.
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the
> typical
> > > rate
> > > > > of
> > > > > > > > > getting
> > > > > > > > >
> > > > > > > > > > > and
> > > > > > > > >
> > > > > > > > > > > >> > >>> renewing
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this should
> be
> > > > very
> > > > > > very
> > > > > > > > > low,
> > > > > > > > >
> > > > > > > > > > 1
> > > > > > > > >
> > > > > > > > > > > >> > request
> > > > > > > > >
> > > > > > > > > > > >> > >>> per
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > minute
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > is a
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
> > estimate.
> > > > > > However
> > > > > > > it
> > > > > > > > >
> > > > > > > > > > > depends
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > > > > >
> > > > > > > > > > > >> > >>> the
> > > > > > > > >
> > > > > > > > > > > >> > >>> > > token
> > > > > > > > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
@Gwen, @Rajini,
As mentioned in the KIP, main motivation for this KIP is to reduce load on
Kerberos
server on large kafka deployments with large number of clients.
Also it looks like we are combining two overlapping concepts
1. Single client sending requests with multiple users/authentications
2. Impersonation
Option 1, is definitely useful in some use cases and can be used to
implement workaround for
impersonation
In Impersonation, a super user can send requests on behalf of another
user(Alice) in a secured way.
superuser has credentials but user Alice doesn't have any. The requests are
required
to run as user Alice and accesses/ACLs on Broker are required to be done as
user Alice.
It is required that user Alice can connect to the Broker on a connection
authenticated with
superuser's credentials. In other words superuser is impersonating the user
Alice.
The approach mentioned by Harsha in previous mail is implemented in hadoop,
storm etc..
Some more details here:
https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Superusers.html
@Rajini
Thanks for your comments on SASL/SCRAM usage. I am thinking to send
tokenHmac (salted-hashed version)
as password for authentication and tokenID for retrial of tokenHmac at
server side.
Does above sound OK?
Thanks,
Manikumar
On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> @Gwen @Mani Not sure why we want to authenticate at every request. Even if
> the token exchange is cheap it still a few calls that need to go through
> round trip. Impersonation doesn't require authentication for every
> request.
>
> "So a centralized app can create few producers, do the metadata request and
> broker discovery with its own user auth, but then use delegation tokens to
> allow performing produce/fetch requests as different users? Instead of
> having to re-connect for each impersonated user?"
>
> Yes. But what we will have is this centralized user as impersonation user
> on behalf of other users. When it authenticates initially we will create a
> "Subject" and from there on wards centralized user can do
> Subject.doAsPrivileged
> on behalf, other users.
> On the server side, we can retrieve two principals out of this one is the
> authenticated user (centralized user) and another is impersonated user. We
> will first check if the authenticated user allowed to impersonate and then
> move on to check if the user Alice has access to the topic "X" to
> read/write.
>
> @Rajini Intention of this KIP is to support token auth via SASL/SCRAM, not
> just with TLS. What you raised is a good point let me take a look and add
> details.
>
> It will be easier to add impersonation once we reach agreement on this KIP.
>
>
> On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Rajini,
> >
> > I think it would definitely be valuable to have a KIP for impersonation.
> >
> > Ismael
> >
> > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rs...@pivotal.io>
> > wrote:
> >
> > > It would clearly be very useful to enable clients to send requests on
> > > behalf of multiple users. A separate KIP makes sense, but it may be
> worth
> > > thinking through some of the implications now, especially if the main
> > > interest in delegation tokens comes from its potential to enable
> > > impersonation.
> > >
> > > I understand that delegation tokens are only expected to be used with
> > TLS.
> > > But the choice of SASL/SCRAM for authentication must be based on a
> > > requirement to protect the tokenHmac - otherwise you could just use
> > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> > on-the-wire,
> > > only a salted-hashed version of it is used in the SASL authentication
> > > exchange. If impersonation is based on sending tokenHmac in requests,
> any
> > > benefit of using SCRAM is lost.
> > >
> > > An alternative may be to allow clients to authenticate multiple times
> > using
> > > SASL and include one of its authenticated principals in each request
> > > (optionally). I haven't thought it through yet, obviously. But if the
> > > approach is of interest and no one is working on a KIP for
> impersonation
> > at
> > > the moment, I am happy to write one. It may provide something for
> > > comparison at least.
> > >
> > > Thoughts?
> > >
> > >
> > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > That's a good idea. Authenticating every request with delegation
> token
> > > will
> > > > be useful for
> > > > impersonation use-cases. But as of now, we are thinking delegation
> > token
> > > as
> > > > just another way
> > > > to authenticate the users. We haven't think through all the use cases
> > > > related to
> > > > impersonation or using delegation token for impersonation. We want to
> > > > handle impersonation
> > > > (KAFKA-3712) as part of separate KIP.
> > > >
> > > > Will that be Ok?
> > > >
> > > >
> > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > > >
> > > > > Thinking out loud here:
> > > > >
> > > > > It looks like authentication with a delegation token is going to be
> > > > > super-cheap, right? We just compare the token to a value in the
> > broker
> > > > > cache?
> > > > >
> > > > > If I understood the KIP correctly, right now it suggests that
> > > > > authentication happens when establishing the client-broker
> connection
> > > (as
> > > > > normal for Kafka. But perhaps we want to consider authenticating
> > every
> > > > > request with delegation token (if exists)?
> > > > >
> > > > > So a centralized app can create few producers, do the metadata
> > request
> > > > and
> > > > > broker discovery with its own user auth, but then use delegation
> > tokens
> > > > to
> > > > > allow performing produce/fetch requests as different users? Instead
> > of
> > > > > having to re-connect for each impersonated user?
> > > > >
> > > > > This may over-complicate things quite a bit (basically adding extra
> > > > > information in every request), but maybe it will be useful for
> > > > > impersonation use-cases (which seem to drive much of the interest
> in
> > > this
> > > > > KIP)?
> > > > > Kafka Connect, NiFi and friends can probably use this to share
> > clients
> > > > > between multiple jobs, tasks, etc.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Gwen
> > > > >
> > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> > manikumar.reddy@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Ashish,
> > > > > >
> > > > > > Thank you for reviewing the KIP. Please see the replies inline.
> > > > > >
> > > > > >
> > > > > > > 1. How to disable delegation token authentication?
> > > > > > >
> > > > > > > This can be achieved in various ways, however I think reusing
> > > > > delegation
> > > > > > > token secret config for this makes sense here. Avoids creating
> > yet
> > > > > > another
> > > > > > > config and forces delegation token users to consciously set the
> > > > secret.
> > > > > > If
> > > > > > > the secret is not set or set to empty string, brokers should
> turn
> > > off
> > > > > > > delegation token support. This will however require a new error
> > > code
> > > > to
> > > > > > > indicate delegation token support is turned off on broker.
> > > > > > >
> > > > > >
> > > > > > Thanks for the suggestion. Option to turnoff delegation token
> > > > > > authentication will be useful.
> > > > > > I'll update the KIP.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > 2. ACLs on delegation token?
> > > > > > >
> > > > > > > Do we need to have ACLs defined for tokens? I do not think it
> > buys
> > > us
> > > > > > > anything, as delegation token can be treated as impersonation
> of
> > > the
> > > > > > owner.
> > > > > > > Any thing the owner has permission to do, delegation tokens
> > should
> > > be
> > > > > > > allowed to do as well. If so, we probably won't need to return
> > > > > > > authorization exception error code while creating delegation
> > token.
> > > > It
> > > > > > > however would make sense to check renew and expire requests are
> > > > coming
> > > > > > from
> > > > > > > owner or renewers of the token, but that does not require
> > explicit
> > > > > acls.
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Yes, We agreed to not have new acl on who can request delegation
> > > token.
> > > > > > I'll update the KIP.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > 3. How to restrict max life time of a token?
> > > > > > >
> > > > > > > Admins might want to restrict max life time of tokens created
> on
> > a
> > > > > > cluster,
> > > > > > > and this can very from cluster to cluster based on use-cases.
> > This
> > > > > might
> > > > > > > warrant a separate broker config.
> > > > > > >
> > > > > > >
> > > > > > Currently we have "delegation.token.max.lifetime.sec" server
> > config
> > > > > > property
> > > > > > May be we can take min(User supplied MaxTime, Server MaxTime) as
> > max
> > > > life
> > > > > > time.
> > > > > > I am open to add new config property.
> > > > > >
> > > > > > Few more comments based on recent KIP update.
> > > > > > >
> > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we
> use
> > > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything
> > before
> > > > > > current
> > > > > > > date?
> > > > > > >
> > > > > >
> > > > > > makes sense. we don't need special request to cancel the token.
> We
> > > can
> > > > > use
> > > > > > ExpireTokenRequest.
> > > > > > I'll update the KIP.
> > > > > >
> > > > > >
> > > > > > > 2. Can we change time field names to indicate their unit is
> > > > > milliseconds,
> > > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > > > >
> > > > > > >
> > > > > > Done.
> > > > > >
> > > > > >
> > > > > > > 3. Can we allow users to renew a token for a specified amount
> of
> > > > time?
> > > > > In
> > > > > > > current version of KIP, renew request does not take time as a
> > > param,
> > > > > not
> > > > > > > sure what is expiry time set to after renewal.
> > > > > > >
> > > > > > >
> > > > > > Yes, we need to specify renew period. I'll update the KIP.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Mankumar
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> > > manikumar.reddy@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I would like to reinitiate the discussion on Delegation token
> > > > support
> > > > > > for
> > > > > > > >
> > > > > > > > Kafka.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Brief summary of the past discussion:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > 1) Broker stores delegation tokens in zookeeper. All brokers
> > > will
> > > > > > have a
> > > > > > > >
> > > > > > > > cache backed by
> > > > > > > >
> > > > > > > > zookeeper so they will all get notified whenever a new
> token
> > > is
> > > > > > > >
> > > > > > > > generated and they will
> > > > > > > >
> > > > > > > > update their local cache whenever token state changes.
> > > > > > > >
> > > > > > > > 2) The current proposal does not support rotation of secret
> > > > > > > >
> > > > > > > > 3) Only allow the renewal by users that authenticated using
> > *non*
> > > > > > > >
> > > > > > > > delegation token mechanism
> > > > > > > >
> > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka
> > > clients
> > > > > can
> > > > > > > >
> > > > > > > > authenticate using
> > > > > > > >
> > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> > > password.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Updated the KIP with the following:
> > > > > > > >
> > > > > > > > 1. Protocol and Config changes
> > > > > > > >
> > > > > > > > 2. format of the data stored in ZK.
> > > > > > > >
> > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > > >
> > > > > > > > 48+Delegation+token+support+for+Kafka
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Jun, Ashish, Gwen,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Pl review the updated KIP.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > >
> > > > > > > > Manikumar
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> > > asingh@cloudera.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > Harsha/ Gwen,
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > How do we proceed here? I am willing to help out with here.
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > > > gwen@confluent.io>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > Is it updated? are all concerns addressed? do you want to
> > > > start a
> > > > > > > vote?
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > Sorry for being pushy, I do appreciate that we are all
> > > > volunteers
> > > > > > and
> > > > > > > >
> > > > > > > > > > finding time is difficult. This feature is important for
> > > > anything
> > > > > > > that
> > > > > > > >
> > > > > > > > > > integrates with Kafka (stream processors, Flume, NiFi,
> etc)
> > > > and I
> > > > > > > >
> > > > > > > > > > don't want to see this getting stuck because we lack
> > > > coordination
> > > > > > > >
> > > > > > > > > > within the community.
> > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > > > > > kafka@harsha.io>
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > > > The only pending update for the KIP is to write up the
> > > > protocol
> > > > > > > > changes
> > > > > > > >
> > > > > > > > > > like
> > > > > > > >
> > > > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > > > asingh@cloudera.com>
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> I think we decided to not support secret rotation, I
> > guess
> > > > > this
> > > > > > > can
> > > > > > > > be
> > > > > > > >
> > > > > > > > > > >> stated clearly on the KIP. Also, more details on how
> > > clients
> > > > > > will
> > > > > > > >
> > > > > > > > > > perform
> > > > > > > >
> > > > > > > > > > >> token distribution and how CLI will look like will be
> > > > helpful.
> > > > > > > >
> > > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > > > gwen@confluent.io>
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> > Hi Guys,
> > > > > > > >
> > > > > > > > > > >> >
> > > > > > > >
> > > > > > > > > > >> > This discussion was dead for a while. Are there
> still
> > > > > > > contentious
> > > > > > > >
> > > > > > > > > > >> > points? If not, why are there no votes?
> > > > > > > >
> > > > > > > > > > >> >
> > > > > > > >
> > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > > > jun@confluent.io>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > > >> > > Ashish,
> > > > > > > >
> > > > > > > > > > >> > >
> > > > > > > >
> > > > > > > > > > >> > > Yes, I will send out a KIP invite for next week to
> > > > discuss
> > > > > > > > KIP-48
> > > > > > > >
> > > > > > > > > > and
> > > > > > > >
> > > > > > > > > > >> > other
> > > > > > > >
> > > > > > > > > > >> > > remaining KIPs.
> > > > > > > >
> > > > > > > > > > >> > >
> > > > > > > >
> > > > > > > > > > >> > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >
> > > > > > > >
> > > > > > > > > > >> > > Jun
> > > > > > > >
> > > > > > > > > > >> > >
> > > > > > > >
> > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > > > > > >
> > > > > > > > > asingh@cloudera.com>
> > > > > > > >
> > > > > > > > > > >> > wrote:
> > > > > > > >
> > > > > > > > > > >> > >
> > > > > > > >
> > > > > > > > > > >> > >> Thanks Harsha!
> > > > > > > >
> > > > > > > > > > >> > >>
> > > > > > > >
> > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
> > agenda.
> > > > > Also,
> > > > > > we
> > > > > > > > did
> > > > > > > >
> > > > > > > > > > not
> > > > > > > >
> > > > > > > > > > >> > >> actually make a call on when we should have next
> > KIP
> > > > > call.
> > > > > > As
> > > > > > > >
> > > > > > > > > there
> > > > > > > >
> > > > > > > > > > >> > >> are
> > > > > > > >
> > > > > > > > > > >> > a
> > > > > > > >
> > > > > > > > > > >> > >> few outstanding KIPs that could not be discussed
> > this
> > > > > week,
> > > > > > > can
> > > > > > > >
> > > > > > > > > we
> > > > > > > >
> > > > > > > > > > >> > >> have
> > > > > > > >
> > > > > > > > > > >> > a
> > > > > > > >
> > > > > > > > > > >> > >> KIP hangout call next week?
> > > > > > > >
> > > > > > > > > > >> > >>
> > > > > > > >
> > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> > Chintalapani
> > > > > > > >
> > > > > > > > > > >> > >> <ka...@harsha.io>
> > > > > > > >
> > > > > > > > > > >> > >> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>
> > > > > > > >
> > > > > > > > > > >> > >>> Ashish,
> > > > > > > >
> > > > > > > > > > >> > >>> Yes we are working on it. Lets discuss
> in
> > > the
> > > > > next
> > > > > > > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> meeting.
> > > > > > > >
> > > > > > > > > > >> > >>> I'll join.
> > > > > > > >
> > > > > > > > > > >> > >>> -Harsha
> > > > > > > >
> > > > > > > > > > >> > >>>
> > > > > > > >
> > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > > > > > > >
> > > > > > > > > > asingh@cloudera.com>
> > > > > > > >
> > > > > > > > > > >> > >>> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>>
> > > > > > > >
> > > > > > > > > > >> > >>> > Hello Harsha,
> > > > > > > >
> > > > > > > > > > >> > >>> >
> > > > > > > >
> > > > > > > > > > >> > >>> > Are you still working on this? Wondering if we
> > can
> > > > > > discuss
> > > > > > > >
> > > > > > > > > this
> > > > > > > >
> > > > > > > > > > in
> > > > > > > >
> > > > > > > > > > >> > next
> > > > > > > >
> > > > > > > > > > >> > >>> KIP
> > > > > > > >
> > > > > > > > > > >> > >>> > meeting, if you can join.
> > > > > > > >
> > > > > > > > > > >> > >>> >
> > > > > > > >
> > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > > > Chintalapani <
> > > > > > > >
> > > > > > > > > > >> > kafka@harsha.io>
> > > > > > > >
> > > > > > > > > > >> > >>> > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> >
> > > > > > > >
> > > > > > > > > > >> > >>> > > Hi Grant,
> > > > > > > >
> > > > > > > > > > >> > >>> > > We are working on it. Will add the
> > > > details
> > > > > > to
> > > > > > > > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> > > about
> > > > > > > >
> > > > > > > > > > >> > the
> > > > > > > >
> > > > > > > > > > >> > >>> > > request protocol.
> > > > > > > >
> > > > > > > > > > >> > >>> > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > Harsha
> > > > > > > >
> > > > > > > > > > >> > >>> > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > > > > > > >
> > > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > > > >
> > > > > > > > > > >> > >>> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > Hi Parth,
> > > > > > > >
> > > > > > > > > > >> > >>> > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > Are you still working on this? If you need
> > any
> > > > > help
> > > > > > > > please
> > > > > > > >
> > > > > > > > > > >> > >>> > > > don't
> > > > > > > >
> > > > > > > > > > >> > >>> > hesitate
> > > > > > > >
> > > > > > > > > > >> > >>> > > > to ask.
> > > > > > > >
> > > > > > > > > > >> > >>> > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > Grant
> > > > > > > >
> > > > > > > > > > >> > >>> > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > > > > > > >
> > > > > > > > > jun@confluent.io>
> > > > > > > >
> > > > > > > > > > >> > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > Parth,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > It makes sense to only allow the renewal
> > by
> > > > > users
> > > > > > > that
> > > > > > > >
> > > > > > > > > > >> > >>> authenticated
> > > > > > > >
> > > > > > > > > > >> > >>> > > > using
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism. Then,
> > > should
> > > > > we
> > > > > > > make
> > > > > > > >
> > > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > >>> renewal a
> > > > > > > >
> > > > > > > > > > >> > >>> > > > list?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > For example, in the case of rest proxy,
> it
> > > > will
> > > > > be
> > > > > > > >
> > > > > > > > > useful
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > for
> > > > > > > >
> > > > > > > > > > >> > >>> every
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to
> renew
> > > the
> > > > > > > tokens.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > It would be clearer if we can document
> the
> > > > > request
> > > > > > > >
> > > > > > > > > > protocol
> > > > > > > >
> > > > > > > > > > >> > like
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > uence/display/KAFKA/KIP-
> > > > > > > >
> > > > > > > > > > >> > >>> > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> > > centralized+administrative+
> > > > > > > >
> > > > > > > > > > operations#KIP-4-
> > > > > > > >
> > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > > > istrativeoperations-
> > > > > > > >
> > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > > > 2945):(VotedandPlannedforin0.
> > > > > > > >
> > > > > > > > > > 10.1.0)
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > .
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > It would also be useful to document the
> > > client
> > > > > > APIs.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > Jun
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> > > > > brahmbhatt
> > > > > > <
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Hi,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only
> allow
> > > the
> > > > > > > renewal
> > > > > > > > by
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > users
> > > > > > > >
> > > > > > > > > > >> > >>> that
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > authenticated using *non* delegation
> > token
> > > > > > > > mechanism.
> > > > > > > >
> > > > > > > > > > For
> > > > > > > >
> > > > > > > > > > >> > >>> example,
> > > > > > > >
> > > > > > > > > > >> > >>> > If
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > user
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos and
> > > > > requested
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > >
> > > > > > > > > > >> > >>> tokens,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > only
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> > > delegation
> > > > > > token
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > mechanism
> > > > > > > >
> > > > > > > > > > >> > can
> > > > > > > >
> > > > > > > > > > >> > >>> > > renew.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Clients that have access to
> delegation
> > > > tokens
> > > > > > can
> > > > > > > > not
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > issue
> > > > > > > >
> > > > > > > > > > >> > >>> > renewal
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > request for renewing their own token
> and
> > > > this
> > > > > is
> > > > > > > >
> > > > > > > > > > primarily
> > > > > > > >
> > > > > > > > > > >> > >>> > important
> > > > > > > >
> > > > > > > > > > >> > >>> > > to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > > > compromised
> > > > > > > token
> > > > > > > >
> > > > > > > > > > will
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > be
> > > > > > > >
> > > > > > > > > > >> > >>> valid.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user
> > can
> > > > > > request
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > >
> > > > > > > > > > >> > >>> > tokens
> > > > > > > >
> > > > > > > > > > >> > >>> > > > but
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > even here I would recommend to avoid
> > > > creating
> > > > > a
> > > > > > > > chain
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > where a
> > > > > > > >
> > > > > > > > > > >> > >>> > client
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > authenticated via delegation token
> > request
> > > > for
> > > > > > > more
> > > > > > > >
> > > > > > > > > > >> > delegation
> > > > > > > >
> > > > > > > > > > >> > >>> > > tokens.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Basically anyone can request
> delegation
> > > > token,
> > > > > > as
> > > > > > > > long
> > > > > > > >
> > > > > > > > > > as
> > > > > > > >
> > > > > > > > > > >> > they
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > authenticate
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > <
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > > uence/display/KAFKA/KIP-
> > > > > > > >
> > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > > > >
> > > > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > > > >
> > > > > > > > > > >> > >>> upportforKaf
> > > > > > > >
> > > > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > sufficient?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Thanks
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > Parth
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun
> Rao
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > > > >
> > > > > > > > > > >> > >>> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > Parth,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of
> > > comments
> > > > > > > inline
> > > > > > > >
> > > > > > > > > > below.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM,
> > parth
> > > > > > > > brahmbhatt <
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed?
> By
> > > > > original
> > > > > > > >
> > > > > > > > > > requester
> > > > > > > >
> > > > > > > > > > >> > >>> only? or
> > > > > > > >
> > > > > > > > > > >> > >>> > > > using
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do this
> only
> > > > using
> > > > > > > >
> > > > > > > > > Kerberos
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > auth
> > > > > > > >
> > > > > > > > > > >> > and
> > > > > > > >
> > > > > > > > > > >> > >>> > only
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > threw
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > renewer specified during the
> > > acquisition
> > > > > > > > request.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this.
> Are
> > > you
> > > > > > saying
> > > > > > > >
> > > > > > > > > that
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > any
> > > > > > > >
> > > > > > > > > > >> > >>> client
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > authenticated with the delegation
> > token
> > > > can
> > > > > > > renew,
> > > > > > > >
> > > > > > > > > > i.e.
> > > > > > > >
> > > > > > > > > > >> > there
> > > > > > > >
> > > > > > > > > > >> > >>> is
> > > > > > > >
> > > > > > > > > > >> > >>> > no
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > renewer
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > needed?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> > > authenticated
> > > > > > client
> > > > > > > >
> > > > > > > > > > (either
> > > > > > > >
> > > > > > > > > > >> > >>> through
> > > > > > > >
> > > > > > > > > > >> > >>> > > SASL
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > or
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation token
> > for
> > > > the
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > authenticated
> > > > > > > >
> > > > > > > > > > >> > >>> user,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > right?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each
> broker
> > or
> > > > in
> > > > > > ZK?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to
> store
> > in
> > > > ZK
> > > > > or
> > > > > > > not
> > > > > > > >
> > > > > > > > > > store
> > > > > > > >
> > > > > > > > > > >> > them
> > > > > > > >
> > > > > > > > > > >> > >>> at
> > > > > > > >
> > > > > > > > > > >> > >>> > > all.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > The
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > whole controller based
> distribution
> > is
> > > > too
> > > > > > > much
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > overhead
> > > > > > > >
> > > > > > > > > > >> > >>> with
> > > > > > > >
> > > > > > > > > > >> > >>> > not
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > much
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated /
> > > expired?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Either by expiration time out or
> > > through
> > > > > an
> > > > > > > >
> > > > > > > > > explicit
> > > > > > > >
> > > > > > > > > > >> > >>> request to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is
> > used?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
> > proposal
> > > > (it
> > > > > > > wasn't
> > > > > > > >
> > > > > > > > > in
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> but
> > > > > > > >
> > > > > > > > > > >> > >>> > > was
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > discussed
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > There is no imperonation
> proposal. I
> > > > tried
> > > > > > and
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > explained
> > > > > > > >
> > > > > > > > > > >> > how
> > > > > > > >
> > > > > > > > > > >> > >>> > its
> > > > > > > >
> > > > > > > > > > >> > >>> > > a
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > different problem and why its not
> > > really
> > > > > > > > necessary
> > > > > > > >
> > > > > > > > > > to
> > > > > > > >
> > > > > > > > > > >> > >>> discuss
> > > > > > > >
> > > > > > > > > > >> > >>> > > that
> > > > > > > >
> > > > > > > > > > >> > >>> > > > as
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > part
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not
> > > support
> > > > > any
> > > > > > > >
> > > > > > > > > > >> > impersonation,
> > > > > > > >
> > > > > > > > > > >> > >>> it
> > > > > > > >
> > > > > > > > > > >> > >>> > > will
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > just
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > be
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so -
> for
> > > what
> > > > > > > > actions?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > Could we document the format of the
> > new
> > > > > > > >
> > > > > > > > > > request/response
> > > > > > > >
> > > > > > > > > > >> > and
> > > > > > > >
> > > > > > > > > > >> > >>> > their
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > associated Resource and Operation
> for
> > > ACL?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation token
> be
> > > > > > > configured
> > > > > > > > in
> > > > > > > >
> > > > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > >>> client?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Should be through config. I wasn't
> > > > > planning
> > > > > > on
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > supporting
> > > > > > > >
> > > > > > > > > > >> > >>> JAAS
> > > > > > > >
> > > > > > > > > > >> > >>> > > for
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > tokens.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does this
> > > either.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Parth
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM,
> Jun
> > > > Rao <
> > > > > > > >
> > > > > > > > > > >> > jun@confluent.io>
> > > > > > > >
> > > > > > > > > > >> > >>> > > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation
> token
> > be
> > > > > > > > configured
> > > > > > > >
> > > > > > > > > in
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > >>> > client?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > The
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > standard
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > way is to do this through JAAS.
> > > > However,
> > > > > > we
> > > > > > > > will
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > need
> > > > > > > >
> > > > > > > > > > >> > to
> > > > > > > >
> > > > > > > > > > >> > >>> > think
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > through
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > if
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > > > > > environment.
> > > > > > > > For
> > > > > > > >
> > > > > > > > > > >> > example,
> > > > > > > >
> > > > > > > > > > >> > >>> > when a
> > > > > > > >
> > > > > > > > > > >> > >>> > > > new
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > task
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > is
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker node, do
> > we
> > > > need
> > > > > > to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > > > >
> > > > > > > > > > >> > >>> add a
> > > > > > > >
> > > > > > > > > > >> > >>> > > new
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > section
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> > > > > > convenient
> > > > > > > if
> > > > > > > >
> > > > > > > > > we
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > can
> > > > > > > >
> > > > > > > > > > >> > >>> pass in
> > > > > > > >
> > > > > > > > > > >> > >>> > > the
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > token
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > through the config directly w/o
> > > going
> > > > > > > through
> > > > > > > >
> > > > > > > > > > JAAS.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still actively
> > > > working
> > > > > on
> > > > > > > > this
> > > > > > > >
> > > > > > > > > > KIP?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM,
> > Jun
> > > > > Rao <
> > > > > > > >
> > > > > > > > > > >> > >>> jun@confluent.io>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to
> document
> > > the
> > > > > > format
> > > > > > > > of
> > > > > > > >
> > > > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > data
> > > > > > > >
> > > > > > > > > > >> > >>> > stored
> > > > > > > >
> > > > > > > > > > >> > >>> > > > in
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > ZK.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> > discussion
> > > > on
> > > > > > > > whether
> > > > > > > >
> > > > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > tokens
> > > > > > > >
> > > > > > > > > > >> > >>> > > should
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > be
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > > > config/acl/quota,
> > > > > > > >
> > > > > > > > > or
> > > > > > > >
> > > > > > > > > > >> > through
> > > > > > > >
> > > > > > > > > > >> > >>> the
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > controller.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller is
> > only
> > > > > > designed
> > > > > > > > for
> > > > > > > >
> > > > > > > > > > >> > >>> propagating
> > > > > > > >
> > > > > > > > > > >> > >>> > > > topic
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send
> > the
> > > > > token
> > > > > > > >
> > > > > > > > > instead
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > of
> > > > > > > >
> > > > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > since
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > it's
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki
> > seem
> > > > > > broken.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02
> > AM,
> > > > Gwen
> > > > > > > >
> > > > > > > > > Shapira <
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see,
> remaining
> > > > > > questions
> > > > > > > > are:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
> > renewed?
> > > By
> > > > > > > > original
> > > > > > > >
> > > > > > > > > > >> > requester
> > > > > > > >
> > > > > > > > > > >> > >>> > only?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > or
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > using
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each
> > > broker
> > > > > or
> > > > > > in
> > > > > > > > ZK?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens
> invalidated /
> > > > > > expired?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm
> > is
> > > > > used?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation
> > > > proposal
> > > > > > (it
> > > > > > > >
> > > > > > > > > > wasn't
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > > > >
> > > > > > > > > > >> > the
> > > > > > > >
> > > > > > > > > > >> > >>> > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> > > > but
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > was
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if
> so -
> > > for
> > > > > > what
> > > > > > > >
> > > > > > > > > > actions?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48
> PM,
> > > > > Harsha
> > > > > > <
> > > > > > > >
> > > > > > > > > > >> > >>> kafka@harsha.io>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > Unfortunately
> > > > > > > I
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > > > >
> > > > > > > > > > >> > >>> attend
> > > > > > > >
> > > > > > > > > > >> > >>> > > the
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >
> when
> > > > > > > delegation
> > > > > > > >
> > > > > > > > > > tokens
> > > > > > > >
> > > > > > > > > > >> > >>> > discussed.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > if
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >
> you
> > > can
> > > > > > update
> > > > > > > > the
> > > > > > > >
> > > > > > > > > > >> > thread if
> > > > > > > >
> > > > > > > > > > >> > >>> > you
> > > > > > > >
> > > > > > > > > > >> > >>> > > > have
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > any
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > further
> > > > > > > > questions.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at
> > 11:32
> > > > AM,
> > > > > > > Liquan
> > > > > > > >
> > > > > > > > > Pei
> > > > > > > >
> > > > > > > > > > >> > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the links to
> > > > images
> > > > > in
> > > > > > > the
> > > > > > > >
> > > > > > > > > KIP
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > > > >
> > > > > > > > > > >> > >>> > broken.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at
> 9:33
> > > AM,
> > > > > > parth
> > > > > > > >
> > > > > > > > > > >> > brahmbhatt <
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >>
> brahmbhatt.parth@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > > > getDelegationTokenAs
> > > > > > > >
> > > > > > > > > mean?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current proposal
> we
> > > > only
> > > > > > > allow
> > > > > > > > a
> > > > > > > >
> > > > > > > > > > user
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > > > >
> > > > > > > > > > >> > >>> get
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > token
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > > > authenticated
> > > > > > > as
> > > > > > > >
> > > > > > > > > > using
> > > > > > > >
> > > > > > > > > > >> > >>> another
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > mechanism,
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate using
> a
> > > > keytab
> > > > > > for
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for
> that
> > > > user
> > > > > > > only.
> > > > > > > > In
> > > > > > > >
> > > > > > > > > > >> > future I
> > > > > > > >
> > > > > > > > > > >> > >>> > think
> > > > > > > >
> > > > > > > > > > >> > >>> > > > we
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > will
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > have
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such that
> > we
> > > > > allow
> > > > > > > some
> > > > > > > >
> > > > > > > > > set
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > > > >
> > > > > > > > > > >> > >>> users (
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > kafka-rest-user@EXAMPLE.COM
> > > ,
> > > > > > > >
> > > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > > > >
> > > > > > > > > > >> > >>> > > > to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > acquire
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on
> > behalf
> > > of
> > > > > > other
> > > > > > > >
> > > > > > > > > users
> > > > > > > >
> > > > > > > > > > >> > whose
> > > > > > > >
> > > > > > > > > > >> > >>> > > identity
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > they
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > have
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified independently.
> > > Kafka
> > > > > > > brokers
> > > > > > > >
> > > > > > > > > > will
> > > > > > > >
> > > > > > > > > > >> > have
> > > > > > > >
> > > > > > > > > > >> > >>> ACLs
> > > > > > > >
> > > > > > > > > > >> > >>> > > to
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > control
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > > > impersonate
> > > > > > > other
> > > > > > > >
> > > > > > > > > > users
> > > > > > > >
> > > > > > > > > > >> > and
> > > > > > > >
> > > > > > > > > > >> > >>> get
> > > > > > > >
> > > > > > > > > > >> > >>> > > > tokens
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > on
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> > Impersonation
> > > > is a
> > > > > > > whole
> > > > > > > >
> > > > > > > > > > >> > different
> > > > > > > >
> > > > > > > > > > >> > >>> > > problem
> > > > > > > >
> > > > > > > > > > >> > >>> > > > in
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > my
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it
> > in
> > > > > > separate
> > > > > > > >
> > > > > > > > > KIP.
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical
> > rate
> > > > of
> > > > > > > > getting
> > > > > > > >
> > > > > > > > > > and
> > > > > > > >
> > > > > > > > > > >> > >>> renewing
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > delegation
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this should be
> > > very
> > > > > very
> > > > > > > > low,
> > > > > > > >
> > > > > > > > > 1
> > > > > > > >
> > > > > > > > > > >> > request
> > > > > > > >
> > > > > > > > > > >> > >>> per
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > minute
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > is a
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high
> estimate.
> > > > > However
> > > > > > it
> > > > > > > >
> > > > > > > > > > depends
> > > > > > > >
> > > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > > > >
> > > > > > > > > > >> > >>> the
> > > > > > > >
> > > > > > > > > > >> > >>> > > token
> > > > > > > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Harsha Chintalapani <ka...@harsha.io>.
@Gwen @Mani Not sure why we want to authenticate at every request. Even if
the token exchange is cheap it still a few calls that need to go through
round trip. Impersonation doesn't require authentication for every
request.
"So a centralized app can create few producers, do the metadata request and
broker discovery with its own user auth, but then use delegation tokens to
allow performing produce/fetch requests as different users? Instead of
having to re-connect for each impersonated user?"
Yes. But what we will have is this centralized user as impersonation user
on behalf of other users. When it authenticates initially we will create a
"Subject" and from there on wards centralized user can do
Subject.doAsPrivileged
on behalf, other users.
On the server side, we can retrieve two principals out of this one is the
authenticated user (centralized user) and another is impersonated user. We
will first check if the authenticated user allowed to impersonate and then
move on to check if the user Alice has access to the topic "X" to
read/write.
@Rajini Intention of this KIP is to support token auth via SASL/SCRAM, not
just with TLS. What you raised is a good point let me take a look and add
details.
It will be easier to add impersonation once we reach agreement on this KIP.
On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <is...@juma.me.uk> wrote:
> Hi Rajini,
>
> I think it would definitely be valuable to have a KIP for impersonation.
>
> Ismael
>
> On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rs...@pivotal.io>
> wrote:
>
> > It would clearly be very useful to enable clients to send requests on
> > behalf of multiple users. A separate KIP makes sense, but it may be worth
> > thinking through some of the implications now, especially if the main
> > interest in delegation tokens comes from its potential to enable
> > impersonation.
> >
> > I understand that delegation tokens are only expected to be used with
> TLS.
> > But the choice of SASL/SCRAM for authentication must be based on a
> > requirement to protect the tokenHmac - otherwise you could just use
> > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated
> on-the-wire,
> > only a salted-hashed version of it is used in the SASL authentication
> > exchange. If impersonation is based on sending tokenHmac in requests, any
> > benefit of using SCRAM is lost.
> >
> > An alternative may be to allow clients to authenticate multiple times
> using
> > SASL and include one of its authenticated principals in each request
> > (optionally). I haven't thought it through yet, obviously. But if the
> > approach is of interest and no one is working on a KIP for impersonation
> at
> > the moment, I am happy to write one. It may provide something for
> > comparison at least.
> >
> > Thoughts?
> >
> >
> > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > That's a good idea. Authenticating every request with delegation token
> > will
> > > be useful for
> > > impersonation use-cases. But as of now, we are thinking delegation
> token
> > as
> > > just another way
> > > to authenticate the users. We haven't think through all the use cases
> > > related to
> > > impersonation or using delegation token for impersonation. We want to
> > > handle impersonation
> > > (KAFKA-3712) as part of separate KIP.
> > >
> > > Will that be Ok?
> > >
> > >
> > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > Thinking out loud here:
> > > >
> > > > It looks like authentication with a delegation token is going to be
> > > > super-cheap, right? We just compare the token to a value in the
> broker
> > > > cache?
> > > >
> > > > If I understood the KIP correctly, right now it suggests that
> > > > authentication happens when establishing the client-broker connection
> > (as
> > > > normal for Kafka. But perhaps we want to consider authenticating
> every
> > > > request with delegation token (if exists)?
> > > >
> > > > So a centralized app can create few producers, do the metadata
> request
> > > and
> > > > broker discovery with its own user auth, but then use delegation
> tokens
> > > to
> > > > allow performing produce/fetch requests as different users? Instead
> of
> > > > having to re-connect for each impersonated user?
> > > >
> > > > This may over-complicate things quite a bit (basically adding extra
> > > > information in every request), but maybe it will be useful for
> > > > impersonation use-cases (which seem to drive much of the interest in
> > this
> > > > KIP)?
> > > > Kafka Connect, NiFi and friends can probably use this to share
> clients
> > > > between multiple jobs, tasks, etc.
> > > >
> > > > What do you think?
> > > >
> > > > Gwen
> > > >
> > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <
> manikumar.reddy@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Ashish,
> > > > >
> > > > > Thank you for reviewing the KIP. Please see the replies inline.
> > > > >
> > > > >
> > > > > > 1. How to disable delegation token authentication?
> > > > > >
> > > > > > This can be achieved in various ways, however I think reusing
> > > > delegation
> > > > > > token secret config for this makes sense here. Avoids creating
> yet
> > > > > another
> > > > > > config and forces delegation token users to consciously set the
> > > secret.
> > > > > If
> > > > > > the secret is not set or set to empty string, brokers should turn
> > off
> > > > > > delegation token support. This will however require a new error
> > code
> > > to
> > > > > > indicate delegation token support is turned off on broker.
> > > > > >
> > > > >
> > > > > Thanks for the suggestion. Option to turnoff delegation token
> > > > > authentication will be useful.
> > > > > I'll update the KIP.
> > > > >
> > > > >
> > > > > >
> > > > > > 2. ACLs on delegation token?
> > > > > >
> > > > > > Do we need to have ACLs defined for tokens? I do not think it
> buys
> > us
> > > > > > anything, as delegation token can be treated as impersonation of
> > the
> > > > > owner.
> > > > > > Any thing the owner has permission to do, delegation tokens
> should
> > be
> > > > > > allowed to do as well. If so, we probably won't need to return
> > > > > > authorization exception error code while creating delegation
> token.
> > > It
> > > > > > however would make sense to check renew and expire requests are
> > > coming
> > > > > from
> > > > > > owner or renewers of the token, but that does not require
> explicit
> > > > acls.
> > > > > >
> > > > >
> > > > >
> > > > > Yes, We agreed to not have new acl on who can request delegation
> > token.
> > > > > I'll update the KIP.
> > > > >
> > > > >
> > > > > >
> > > > > > 3. How to restrict max life time of a token?
> > > > > >
> > > > > > Admins might want to restrict max life time of tokens created on
> a
> > > > > cluster,
> > > > > > and this can very from cluster to cluster based on use-cases.
> This
> > > > might
> > > > > > warrant a separate broker config.
> > > > > >
> > > > > >
> > > > > Currently we have "delegation.token.max.lifetime.sec" server
> config
> > > > > property
> > > > > May be we can take min(User supplied MaxTime, Server MaxTime) as
> max
> > > life
> > > > > time.
> > > > > I am open to add new config property.
> > > > >
> > > > > Few more comments based on recent KIP update.
> > > > > >
> > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything
> before
> > > > > current
> > > > > > date?
> > > > > >
> > > > >
> > > > > makes sense. we don't need special request to cancel the token. We
> > can
> > > > use
> > > > > ExpireTokenRequest.
> > > > > I'll update the KIP.
> > > > >
> > > > >
> > > > > > 2. Can we change time field names to indicate their unit is
> > > > milliseconds,
> > > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > > >
> > > > > >
> > > > > Done.
> > > > >
> > > > >
> > > > > > 3. Can we allow users to renew a token for a specified amount of
> > > time?
> > > > In
> > > > > > current version of KIP, renew request does not take time as a
> > param,
> > > > not
> > > > > > sure what is expiry time set to after renewal.
> > > > > >
> > > > > >
> > > > > Yes, we need to specify renew period. I'll update the KIP.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Mankumar
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> > manikumar.reddy@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I would like to reinitiate the discussion on Delegation token
> > > support
> > > > > for
> > > > > > >
> > > > > > > Kafka.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Brief summary of the past discussion:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > 1) Broker stores delegation tokens in zookeeper. All brokers
> > will
> > > > > have a
> > > > > > >
> > > > > > > cache backed by
> > > > > > >
> > > > > > > zookeeper so they will all get notified whenever a new token
> > is
> > > > > > >
> > > > > > > generated and they will
> > > > > > >
> > > > > > > update their local cache whenever token state changes.
> > > > > > >
> > > > > > > 2) The current proposal does not support rotation of secret
> > > > > > >
> > > > > > > 3) Only allow the renewal by users that authenticated using
> *non*
> > > > > > >
> > > > > > > delegation token mechanism
> > > > > > >
> > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka
> > clients
> > > > can
> > > > > > >
> > > > > > > authenticate using
> > > > > > >
> > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> > password.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Updated the KIP with the following:
> > > > > > >
> > > > > > > 1. Protocol and Config changes
> > > > > > >
> > > > > > > 2. format of the data stored in ZK.
> > > > > > >
> > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > >
> > > > > > > 48+Delegation+token+support+for+Kafka
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Jun, Ashish, Gwen,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Pl review the updated KIP.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > Manikumar
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> > asingh@cloudera.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Harsha/ Gwen,
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > How do we proceed here? I am willing to help out with here.
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > > gwen@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > Is it updated? are all concerns addressed? do you want to
> > > start a
> > > > > > vote?
> > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > > > > Sorry for being pushy, I do appreciate that we are all
> > > volunteers
> > > > > and
> > > > > > >
> > > > > > > > > finding time is difficult. This feature is important for
> > > anything
> > > > > > that
> > > > > > >
> > > > > > > > > integrates with Kafka (stream processors, Flume, NiFi, etc)
> > > and I
> > > > > > >
> > > > > > > > > don't want to see this getting stuck because we lack
> > > coordination
> > > > > > >
> > > > > > > > > within the community.
> > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > > > > kafka@harsha.io>
> > > > > > >
> > > > > > > > > wrote:
> > > > > > >
> > > > > > > > > > The only pending update for the KIP is to write up the
> > > protocol
> > > > > > > changes
> > > > > > >
> > > > > > > > > like
> > > > > > >
> > > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > > asingh@cloudera.com>
> > > > > > >
> > > > > > > > > wrote:
> > > > > > >
> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> I think we decided to not support secret rotation, I
> guess
> > > > this
> > > > > > can
> > > > > > > be
> > > > > > >
> > > > > > > > > >> stated clearly on the KIP. Also, more details on how
> > clients
> > > > > will
> > > > > > >
> > > > > > > > > perform
> > > > > > >
> > > > > > > > > >> token distribution and how CLI will look like will be
> > > helpful.
> > > > > > >
> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > > gwen@confluent.io>
> > > > > > >
> > > > > > > > > wrote:
> > > > > > >
> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> > Hi Guys,
> > > > > > >
> > > > > > > > > >> >
> > > > > > >
> > > > > > > > > >> > This discussion was dead for a while. Are there still
> > > > > > contentious
> > > > > > >
> > > > > > > > > >> > points? If not, why are there no votes?
> > > > > > >
> > > > > > > > > >> >
> > > > > > >
> > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > > jun@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > > >> > > Ashish,
> > > > > > >
> > > > > > > > > >> > >
> > > > > > >
> > > > > > > > > >> > > Yes, I will send out a KIP invite for next week to
> > > discuss
> > > > > > > KIP-48
> > > > > > >
> > > > > > > > > and
> > > > > > >
> > > > > > > > > >> > other
> > > > > > >
> > > > > > > > > >> > > remaining KIPs.
> > > > > > >
> > > > > > > > > >> > >
> > > > > > >
> > > > > > > > > >> > > Thanks,
> > > > > > >
> > > > > > > > > >> > >
> > > > > > >
> > > > > > > > > >> > > Jun
> > > > > > >
> > > > > > > > > >> > >
> > > > > > >
> > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > > > > >
> > > > > > > > asingh@cloudera.com>
> > > > > > >
> > > > > > > > > >> > wrote:
> > > > > > >
> > > > > > > > > >> > >
> > > > > > >
> > > > > > > > > >> > >> Thanks Harsha!
> > > > > > >
> > > > > > > > > >> > >>
> > > > > > >
> > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's
> agenda.
> > > > Also,
> > > > > we
> > > > > > > did
> > > > > > >
> > > > > > > > > not
> > > > > > >
> > > > > > > > > >> > >> actually make a call on when we should have next
> KIP
> > > > call.
> > > > > As
> > > > > > >
> > > > > > > > there
> > > > > > >
> > > > > > > > > >> > >> are
> > > > > > >
> > > > > > > > > >> > a
> > > > > > >
> > > > > > > > > >> > >> few outstanding KIPs that could not be discussed
> this
> > > > week,
> > > > > > can
> > > > > > >
> > > > > > > > we
> > > > > > >
> > > > > > > > > >> > >> have
> > > > > > >
> > > > > > > > > >> > a
> > > > > > >
> > > > > > > > > >> > >> KIP hangout call next week?
> > > > > > >
> > > > > > > > > >> > >>
> > > > > > >
> > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha
> Chintalapani
> > > > > > >
> > > > > > > > > >> > >> <ka...@harsha.io>
> > > > > > >
> > > > > > > > > >> > >> wrote:
> > > > > > >
> > > > > > > > > >> > >>
> > > > > > >
> > > > > > > > > >> > >>> Ashish,
> > > > > > >
> > > > > > > > > >> > >>> Yes we are working on it. Lets discuss in
> > the
> > > > next
> > > > > > KIP
> > > > > > >
> > > > > > > > > >> > >>> meeting.
> > > > > > >
> > > > > > > > > >> > >>> I'll join.
> > > > > > >
> > > > > > > > > >> > >>> -Harsha
> > > > > > >
> > > > > > > > > >> > >>>
> > > > > > >
> > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > > > > > >
> > > > > > > > > asingh@cloudera.com>
> > > > > > >
> > > > > > > > > >> > >>> wrote:
> > > > > > >
> > > > > > > > > >> > >>>
> > > > > > >
> > > > > > > > > >> > >>> > Hello Harsha,
> > > > > > >
> > > > > > > > > >> > >>> >
> > > > > > >
> > > > > > > > > >> > >>> > Are you still working on this? Wondering if we
> can
> > > > > discuss
> > > > > > >
> > > > > > > > this
> > > > > > >
> > > > > > > > > in
> > > > > > >
> > > > > > > > > >> > next
> > > > > > >
> > > > > > > > > >> > >>> KIP
> > > > > > >
> > > > > > > > > >> > >>> > meeting, if you can join.
> > > > > > >
> > > > > > > > > >> > >>> >
> > > > > > >
> > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > > Chintalapani <
> > > > > > >
> > > > > > > > > >> > kafka@harsha.io>
> > > > > > >
> > > > > > > > > >> > >>> > wrote:
> > > > > > >
> > > > > > > > > >> > >>> >
> > > > > > >
> > > > > > > > > >> > >>> > > Hi Grant,
> > > > > > >
> > > > > > > > > >> > >>> > > We are working on it. Will add the
> > > details
> > > > > to
> > > > > > > KIP
> > > > > > >
> > > > > > > > > >> > >>> > > about
> > > > > > >
> > > > > > > > > >> > the
> > > > > > >
> > > > > > > > > >> > >>> > > request protocol.
> > > > > > >
> > > > > > > > > >> > >>> > >
> > > > > > >
> > > > > > > > > >> > >>> > > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > Harsha
> > > > > > >
> > > > > > > > > >> > >>> > >
> > > > > > >
> > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > > > > > >
> > > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > > >
> > > > > > > > > >> > >>> wrote:
> > > > > > >
> > > > > > > > > >> > >>> > >
> > > > > > >
> > > > > > > > > >> > >>> > > > Hi Parth,
> > > > > > >
> > > > > > > > > >> > >>> > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > Are you still working on this? If you need
> any
> > > > help
> > > > > > > please
> > > > > > >
> > > > > > > > > >> > >>> > > > don't
> > > > > > >
> > > > > > > > > >> > >>> > hesitate
> > > > > > >
> > > > > > > > > >> > >>> > > > to ask.
> > > > > > >
> > > > > > > > > >> > >>> > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > > Grant
> > > > > > >
> > > > > > > > > >> > >>> > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > > > > > >
> > > > > > > > jun@confluent.io>
> > > > > > >
> > > > > > > > > >> > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > Parth,
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > It makes sense to only allow the renewal
> by
> > > > users
> > > > > > that
> > > > > > >
> > > > > > > > > >> > >>> authenticated
> > > > > > >
> > > > > > > > > >> > >>> > > > using
> > > > > > >
> > > > > > > > > >> > >>> > > > > *non* delegation token mechanism. Then,
> > should
> > > > we
> > > > > > make
> > > > > > >
> > > > > > > > the
> > > > > > >
> > > > > > > > > >> > >>> renewal a
> > > > > > >
> > > > > > > > > >> > >>> > > > list?
> > > > > > >
> > > > > > > > > >> > >>> > > > > For example, in the case of rest proxy, it
> > > will
> > > > be
> > > > > > >
> > > > > > > > useful
> > > > > > >
> > > > > > > > > >> > >>> > > > > for
> > > > > > >
> > > > > > > > > >> > >>> every
> > > > > > >
> > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to renew
> > the
> > > > > > tokens.
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > It would be clearer if we can document the
> > > > request
> > > > > > >
> > > > > > > > > protocol
> > > > > > >
> > > > > > > > > >> > like
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > uence/display/KAFKA/KIP-
> > > > > > >
> > > > > > > > > >> > >>> > >
> > > > > > >
> > > > > > > > > >> > >>> > > 4+-+Command+line+and+
> > centralized+administrative+
> > > > > > >
> > > > > > > > > operations#KIP-4-
> > > > > > >
> > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > > istrativeoperations-
> > > > > > >
> > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > > 2945):(VotedandPlannedforin0.
> > > > > > >
> > > > > > > > > 10.1.0)
> > > > > > >
> > > > > > > > > >> > >>> > > > > .
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > It would also be useful to document the
> > client
> > > > > APIs.
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > Jun
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> > > > brahmbhatt
> > > > > <
> > > > > > >
> > > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Hi,
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > I am suggesting that we will only allow
> > the
> > > > > > renewal
> > > > > > > by
> > > > > > >
> > > > > > > > > >> > >>> > > > > > users
> > > > > > >
> > > > > > > > > >> > >>> that
> > > > > > >
> > > > > > > > > >> > >>> > > > > > authenticated using *non* delegation
> token
> > > > > > > mechanism.
> > > > > > >
> > > > > > > > > For
> > > > > > >
> > > > > > > > > >> > >>> example,
> > > > > > >
> > > > > > > > > >> > >>> > If
> > > > > > >
> > > > > > > > > >> > >>> > > > > user
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos and
> > > > requested
> > > > > > >
> > > > > > > > > >> > >>> > > > > > delegation
> > > > > > >
> > > > > > > > > >> > >>> tokens,
> > > > > > >
> > > > > > > > > >> > >>> > > > only
> > > > > > >
> > > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> > delegation
> > > > > token
> > > > > > >
> > > > > > > > > >> > >>> > > > > > mechanism
> > > > > > >
> > > > > > > > > >> > can
> > > > > > >
> > > > > > > > > >> > >>> > > renew.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Clients that have access to delegation
> > > tokens
> > > > > can
> > > > > > > not
> > > > > > >
> > > > > > > > > >> > >>> > > > > > issue
> > > > > > >
> > > > > > > > > >> > >>> > renewal
> > > > > > >
> > > > > > > > > >> > >>> > > > > > request for renewing their own token and
> > > this
> > > > is
> > > > > > >
> > > > > > > > > primarily
> > > > > > >
> > > > > > > > > >> > >>> > important
> > > > > > >
> > > > > > > > > >> > >>> > > to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > > compromised
> > > > > > token
> > > > > > >
> > > > > > > > > will
> > > > > > >
> > > > > > > > > >> > >>> > > > > > be
> > > > > > >
> > > > > > > > > >> > >>> valid.
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user
> can
> > > > > request
> > > > > > >
> > > > > > > > > >> > >>> > > > > > delegation
> > > > > > >
> > > > > > > > > >> > >>> > tokens
> > > > > > >
> > > > > > > > > >> > >>> > > > but
> > > > > > >
> > > > > > > > > >> > >>> > > > > > even here I would recommend to avoid
> > > creating
> > > > a
> > > > > > > chain
> > > > > > >
> > > > > > > > > >> > >>> > > > > > where a
> > > > > > >
> > > > > > > > > >> > >>> > client
> > > > > > >
> > > > > > > > > >> > >>> > > > > > authenticated via delegation token
> request
> > > for
> > > > > > more
> > > > > > >
> > > > > > > > > >> > delegation
> > > > > > >
> > > > > > > > > >> > >>> > > tokens.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Basically anyone can request delegation
> > > token,
> > > > > as
> > > > > > > long
> > > > > > >
> > > > > > > > > as
> > > > > > >
> > > > > > > > > >> > they
> > > > > > >
> > > > > > > > > >> > >>> > > > > authenticate
> > > > > > >
> > > > > > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > > >
> > > > > > > > > >> > >>> > > > > > <
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > > uence/display/KAFKA/KIP-
> > > > > > >
> > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > > >
> > > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > > >
> > > > > > > > > >> > >>> upportforKaf
> > > > > > >
> > > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > sufficient?
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Thanks
> > > > > > >
> > > > > > > > > >> > >>> > > > > > Parth
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > > > > > >
> > > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > > >
> > > > > > > > > >> > >>> wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > Parth,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of
> > comments
> > > > > > inline
> > > > > > >
> > > > > > > > > below.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM,
> parth
> > > > > > > brahmbhatt <
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By
> > > > original
> > > > > > >
> > > > > > > > > requester
> > > > > > >
> > > > > > > > > >> > >>> only? or
> > > > > > >
> > > > > > > > > >> > >>> > > > using
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > My recommendation is to do this only
> > > using
> > > > > > >
> > > > > > > > Kerberos
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > auth
> > > > > > >
> > > > > > > > > >> > and
> > > > > > >
> > > > > > > > > >> > >>> > only
> > > > > > >
> > > > > > > > > >> > >>> > > > > threw
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > the
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > renewer specified during the
> > acquisition
> > > > > > > request.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are
> > you
> > > > > saying
> > > > > > >
> > > > > > > > that
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > any
> > > > > > >
> > > > > > > > > >> > >>> client
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > authenticated with the delegation
> token
> > > can
> > > > > > renew,
> > > > > > >
> > > > > > > > > i.e.
> > > > > > >
> > > > > > > > > >> > there
> > > > > > >
> > > > > > > > > >> > >>> is
> > > > > > >
> > > > > > > > > >> > >>> > no
> > > > > > >
> > > > > > > > > >> > >>> > > > > > renewer
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > needed?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> > authenticated
> > > > > client
> > > > > > >
> > > > > > > > > (either
> > > > > > >
> > > > > > > > > >> > >>> through
> > > > > > >
> > > > > > > > > >> > >>> > > SASL
> > > > > > >
> > > > > > > > > >> > >>> > > > > or
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation token
> for
> > > the
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > authenticated
> > > > > > >
> > > > > > > > > >> > >>> user,
> > > > > > >
> > > > > > > > > >> > >>> > > > right?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker
> or
> > > in
> > > > > ZK?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > My recommendation is still to store
> in
> > > ZK
> > > > or
> > > > > > not
> > > > > > >
> > > > > > > > > store
> > > > > > >
> > > > > > > > > >> > them
> > > > > > >
> > > > > > > > > >> > >>> at
> > > > > > >
> > > > > > > > > >> > >>> > > all.
> > > > > > >
> > > > > > > > > >> > >>> > > > > The
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > whole controller based distribution
> is
> > > too
> > > > > > much
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > overhead
> > > > > > >
> > > > > > > > > >> > >>> with
> > > > > > >
> > > > > > > > > >> > >>> > not
> > > > > > >
> > > > > > > > > >> > >>> > > > > much
> > > > > > >
> > > > > > > > > >> > >>> > > > > > to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated /
> > expired?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Either by expiration time out or
> > through
> > > > an
> > > > > > >
> > > > > > > > explicit
> > > > > > >
> > > > > > > > > >> > >>> request to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is
> used?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation
> proposal
> > > (it
> > > > > > wasn't
> > > > > > >
> > > > > > > > in
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > the
> > > > > > >
> > > > > > > > > >> > KIP
> > > > > > >
> > > > > > > > > >> > >>> but
> > > > > > >
> > > > > > > > > >> > >>> > > was
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > discussed
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > There is no imperonation proposal. I
> > > tried
> > > > > and
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > explained
> > > > > > >
> > > > > > > > > >> > how
> > > > > > >
> > > > > > > > > >> > >>> > its
> > > > > > >
> > > > > > > > > >> > >>> > > a
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > different problem and why its not
> > really
> > > > > > > necessary
> > > > > > >
> > > > > > > > > to
> > > > > > >
> > > > > > > > > >> > >>> discuss
> > > > > > >
> > > > > > > > > >> > >>> > > that
> > > > > > >
> > > > > > > > > >> > >>> > > > as
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > part
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not
> > support
> > > > any
> > > > > > >
> > > > > > > > > >> > impersonation,
> > > > > > >
> > > > > > > > > >> > >>> it
> > > > > > >
> > > > > > > > > >> > >>> > > will
> > > > > > >
> > > > > > > > > >> > >>> > > > > just
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > be
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for
> > what
> > > > > > > actions?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > Could we document the format of the
> new
> > > > > > >
> > > > > > > > > request/response
> > > > > > >
> > > > > > > > > >> > and
> > > > > > >
> > > > > > > > > >> > >>> > their
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > associated Resource and Operation for
> > ACL?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation token be
> > > > > > configured
> > > > > > > in
> > > > > > >
> > > > > > > > > the
> > > > > > >
> > > > > > > > > >> > >>> client?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Should be through config. I wasn't
> > > > planning
> > > > > on
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > supporting
> > > > > > >
> > > > > > > > > >> > >>> JAAS
> > > > > > >
> > > > > > > > > >> > >>> > > for
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > tokens.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does this
> > either.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Parth
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun
> > > Rao <
> > > > > > >
> > > > > > > > > >> > jun@confluent.io>
> > > > > > >
> > > > > > > > > >> > >>> > > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation token
> be
> > > > > > > configured
> > > > > > >
> > > > > > > > in
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > the
> > > > > > >
> > > > > > > > > >> > >>> > client?
> > > > > > >
> > > > > > > > > >> > >>> > > > The
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > standard
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > way is to do this through JAAS.
> > > However,
> > > > > we
> > > > > > > will
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > need
> > > > > > >
> > > > > > > > > >> > to
> > > > > > >
> > > > > > > > > >> > >>> > think
> > > > > > >
> > > > > > > > > >> > >>> > > > > > through
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > if
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > > > > environment.
> > > > > > > For
> > > > > > >
> > > > > > > > > >> > example,
> > > > > > >
> > > > > > > > > >> > >>> > when a
> > > > > > >
> > > > > > > > > >> > >>> > > > new
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > task
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > is
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker node, do
> we
> > > need
> > > > > to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > > >
> > > > > > > > > >> > >>> add a
> > > > > > >
> > > > > > > > > >> > >>> > > new
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > section
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> > > > > convenient
> > > > > > if
> > > > > > >
> > > > > > > > we
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > can
> > > > > > >
> > > > > > > > > >> > >>> pass in
> > > > > > >
> > > > > > > > > >> > >>> > > the
> > > > > > >
> > > > > > > > > >> > >>> > > > > > token
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > through the config directly w/o
> > going
> > > > > > through
> > > > > > >
> > > > > > > > > JAAS.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still actively
> > > working
> > > > on
> > > > > > > this
> > > > > > >
> > > > > > > > > KIP?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM,
> Jun
> > > > Rao <
> > > > > > >
> > > > > > > > > >> > >>> jun@confluent.io>
> > > > > > >
> > > > > > > > > >> > >>> > > > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to document
> > the
> > > > > format
> > > > > > > of
> > > > > > >
> > > > > > > > > the
> > > > > > >
> > > > > > > > > >> > data
> > > > > > >
> > > > > > > > > >> > >>> > stored
> > > > > > >
> > > > > > > > > >> > >>> > > > in
> > > > > > >
> > > > > > > > > >> > >>> > > > > > ZK.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a
> discussion
> > > on
> > > > > > > whether
> > > > > > >
> > > > > > > > > the
> > > > > > >
> > > > > > > > > >> > tokens
> > > > > > >
> > > > > > > > > >> > >>> > > should
> > > > > > >
> > > > > > > > > >> > >>> > > > > be
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > > config/acl/quota,
> > > > > > >
> > > > > > > > or
> > > > > > >
> > > > > > > > > >> > through
> > > > > > >
> > > > > > > > > >> > >>> the
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > controller.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller is
> only
> > > > > designed
> > > > > > > for
> > > > > > >
> > > > > > > > > >> > >>> propagating
> > > > > > >
> > > > > > > > > >> > >>> > > > topic
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send
> the
> > > > token
> > > > > > >
> > > > > > > > instead
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > of
> > > > > > >
> > > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > > >
> > > > > > > > > >> > >>> > > > > > since
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > it's
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki
> seem
> > > > > broken.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02
> AM,
> > > Gwen
> > > > > > >
> > > > > > > > Shapira <
> > > > > > >
> > > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> From what I can see, remaining
> > > > > questions
> > > > > > > are:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens
> renewed?
> > By
> > > > > > > original
> > > > > > >
> > > > > > > > > >> > requester
> > > > > > >
> > > > > > > > > >> > >>> > only?
> > > > > > >
> > > > > > > > > >> > >>> > > > or
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > using
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each
> > broker
> > > > or
> > > > > in
> > > > > > > ZK?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated /
> > > > > expired?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm
> is
> > > > used?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation
> > > proposal
> > > > > (it
> > > > > > >
> > > > > > > > > wasn't
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > > >
> > > > > > > > > >> > the
> > > > > > >
> > > > > > > > > >> > >>> > KIP
> > > > > > >
> > > > > > > > > >> > >>> > > > but
> > > > > > >
> > > > > > > > > >> > >>> > > > > > was
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so -
> > for
> > > > > what
> > > > > > >
> > > > > > > > > actions?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM,
> > > > Harsha
> > > > > <
> > > > > > >
> > > > > > > > > >> > >>> kafka@harsha.io>
> > > > > > >
> > > > > > > > > >> > >>> > > > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > Unfortunately
> > > > > > I
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > > >
> > > > > > > > > >> > >>> attend
> > > > > > >
> > > > > > > > > >> > >>> > > the
> > > > > > >
> > > > > > > > > >> > >>> > > > > KIP
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > when
> > > > > > delegation
> > > > > > >
> > > > > > > > > tokens
> > > > > > >
> > > > > > > > > >> > >>> > discussed.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > if
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > you
> > can
> > > > > update
> > > > > > > the
> > > > > > >
> > > > > > > > > >> > thread if
> > > > > > >
> > > > > > > > > >> > >>> > you
> > > > > > >
> > > > > > > > > >> > >>> > > > have
> > > > > > >
> > > > > > > > > >> > >>> > > > > > any
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >
> > further
> > > > > > > questions.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at
> 11:32
> > > AM,
> > > > > > Liquan
> > > > > > >
> > > > > > > > Pei
> > > > > > >
> > > > > > > > > >> > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the links to
> > > images
> > > > in
> > > > > > the
> > > > > > >
> > > > > > > > KIP
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > > >
> > > > > > > > > >> > >>> > broken.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33
> > AM,
> > > > > parth
> > > > > > >
> > > > > > > > > >> > brahmbhatt <
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > > getDelegationTokenAs
> > > > > > >
> > > > > > > > mean?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current proposal we
> > > only
> > > > > > allow
> > > > > > > a
> > > > > > >
> > > > > > > > > user
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > > >
> > > > > > > > > >> > >>> get
> > > > > > >
> > > > > > > > > >> > >>> > > > > > delegation
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > token
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > > authenticated
> > > > > > as
> > > > > > >
> > > > > > > > > using
> > > > > > >
> > > > > > > > > >> > >>> another
> > > > > > >
> > > > > > > > > >> > >>> > > > > > mechanism,
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate using a
> > > keytab
> > > > > for
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that
> > > user
> > > > > > only.
> > > > > > > In
> > > > > > >
> > > > > > > > > >> > future I
> > > > > > >
> > > > > > > > > >> > >>> > think
> > > > > > >
> > > > > > > > > >> > >>> > > > we
> > > > > > >
> > > > > > > > > >> > >>> > > > > > will
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > have
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such that
> we
> > > > allow
> > > > > > some
> > > > > > >
> > > > > > > > set
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > > >
> > > > > > > > > >> > >>> users (
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> >
> kafka-rest-user@EXAMPLE.COM
> > ,
> > > > > > >
> > > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > > >
> > > > > > > > > >> > >>> > > > to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > acquire
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on
> behalf
> > of
> > > > > other
> > > > > > >
> > > > > > > > users
> > > > > > >
> > > > > > > > > >> > whose
> > > > > > >
> > > > > > > > > >> > >>> > > identity
> > > > > > >
> > > > > > > > > >> > >>> > > > > > they
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > have
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > verified independently.
> > Kafka
> > > > > > brokers
> > > > > > >
> > > > > > > > > will
> > > > > > >
> > > > > > > > > >> > have
> > > > > > >
> > > > > > > > > >> > >>> ACLs
> > > > > > >
> > > > > > > > > >> > >>> > > to
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > control
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > > impersonate
> > > > > > other
> > > > > > >
> > > > > > > > > users
> > > > > > >
> > > > > > > > > >> > and
> > > > > > >
> > > > > > > > > >> > >>> get
> > > > > > >
> > > > > > > > > >> > >>> > > > tokens
> > > > > > >
> > > > > > > > > >> > >>> > > > > > on
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall
> Impersonation
> > > is a
> > > > > > whole
> > > > > > >
> > > > > > > > > >> > different
> > > > > > >
> > > > > > > > > >> > >>> > > problem
> > > > > > >
> > > > > > > > > >> > >>> > > > in
> > > > > > >
> > > > > > > > > >> > >>> > > > > > my
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it
> in
> > > > > separate
> > > > > > >
> > > > > > > > KIP.
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical
> rate
> > > of
> > > > > > > getting
> > > > > > >
> > > > > > > > > and
> > > > > > >
> > > > > > > > > >> > >>> renewing
> > > > > > >
> > > > > > > > > >> > >>> > > > > > delegation
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this should be
> > very
> > > > very
> > > > > > > low,
> > > > > > >
> > > > > > > > 1
> > > > > > >
> > > > > > > > > >> > request
> > > > > > >
> > > > > > > > > >> > >>> per
> > > > > > >
> > > > > > > > > >> > >>> > > > > minute
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > is a
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high estimate.
> > > > However
> > > > > it
> > > > > > >
> > > > > > > > > depends
> > > > > > >
> > > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > > >
> > > > > > > > > >> > >>> the
> > > > > > >
> > > > > > > > > >> > >>> > > token
> > > > > > >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Rajini,
I think it would definitely be valuable to have a KIP for impersonation.
Ismael
On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rs...@pivotal.io> wrote:
> It would clearly be very useful to enable clients to send requests on
> behalf of multiple users. A separate KIP makes sense, but it may be worth
> thinking through some of the implications now, especially if the main
> interest in delegation tokens comes from its potential to enable
> impersonation.
>
> I understand that delegation tokens are only expected to be used with TLS.
> But the choice of SASL/SCRAM for authentication must be based on a
> requirement to protect the tokenHmac - otherwise you could just use
> SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated on-the-wire,
> only a salted-hashed version of it is used in the SASL authentication
> exchange. If impersonation is based on sending tokenHmac in requests, any
> benefit of using SCRAM is lost.
>
> An alternative may be to allow clients to authenticate multiple times using
> SASL and include one of its authenticated principals in each request
> (optionally). I haven't thought it through yet, obviously. But if the
> approach is of interest and no one is working on a KIP for impersonation at
> the moment, I am happy to write one. It may provide something for
> comparison at least.
>
> Thoughts?
>
>
> On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > That's a good idea. Authenticating every request with delegation token
> will
> > be useful for
> > impersonation use-cases. But as of now, we are thinking delegation token
> as
> > just another way
> > to authenticate the users. We haven't think through all the use cases
> > related to
> > impersonation or using delegation token for impersonation. We want to
> > handle impersonation
> > (KAFKA-3712) as part of separate KIP.
> >
> > Will that be Ok?
> >
> >
> > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Thinking out loud here:
> > >
> > > It looks like authentication with a delegation token is going to be
> > > super-cheap, right? We just compare the token to a value in the broker
> > > cache?
> > >
> > > If I understood the KIP correctly, right now it suggests that
> > > authentication happens when establishing the client-broker connection
> (as
> > > normal for Kafka. But perhaps we want to consider authenticating every
> > > request with delegation token (if exists)?
> > >
> > > So a centralized app can create few producers, do the metadata request
> > and
> > > broker discovery with its own user auth, but then use delegation tokens
> > to
> > > allow performing produce/fetch requests as different users? Instead of
> > > having to re-connect for each impersonated user?
> > >
> > > This may over-complicate things quite a bit (basically adding extra
> > > information in every request), but maybe it will be useful for
> > > impersonation use-cases (which seem to drive much of the interest in
> this
> > > KIP)?
> > > Kafka Connect, NiFi and friends can probably use this to share clients
> > > between multiple jobs, tasks, etc.
> > >
> > > What do you think?
> > >
> > > Gwen
> > >
> > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <manikumar.reddy@gmail.com
> >
> > > wrote:
> > >
> > > > Ashish,
> > > >
> > > > Thank you for reviewing the KIP. Please see the replies inline.
> > > >
> > > >
> > > > > 1. How to disable delegation token authentication?
> > > > >
> > > > > This can be achieved in various ways, however I think reusing
> > > delegation
> > > > > token secret config for this makes sense here. Avoids creating yet
> > > > another
> > > > > config and forces delegation token users to consciously set the
> > secret.
> > > > If
> > > > > the secret is not set or set to empty string, brokers should turn
> off
> > > > > delegation token support. This will however require a new error
> code
> > to
> > > > > indicate delegation token support is turned off on broker.
> > > > >
> > > >
> > > > Thanks for the suggestion. Option to turnoff delegation token
> > > > authentication will be useful.
> > > > I'll update the KIP.
> > > >
> > > >
> > > > >
> > > > > 2. ACLs on delegation token?
> > > > >
> > > > > Do we need to have ACLs defined for tokens? I do not think it buys
> us
> > > > > anything, as delegation token can be treated as impersonation of
> the
> > > > owner.
> > > > > Any thing the owner has permission to do, delegation tokens should
> be
> > > > > allowed to do as well. If so, we probably won't need to return
> > > > > authorization exception error code while creating delegation token.
> > It
> > > > > however would make sense to check renew and expire requests are
> > coming
> > > > from
> > > > > owner or renewers of the token, but that does not require explicit
> > > acls.
> > > > >
> > > >
> > > >
> > > > Yes, We agreed to not have new acl on who can request delegation
> token.
> > > > I'll update the KIP.
> > > >
> > > >
> > > > >
> > > > > 3. How to restrict max life time of a token?
> > > > >
> > > > > Admins might want to restrict max life time of tokens created on a
> > > > cluster,
> > > > > and this can very from cluster to cluster based on use-cases. This
> > > might
> > > > > warrant a separate broker config.
> > > > >
> > > > >
> > > > Currently we have "delegation.token.max.lifetime.sec" server config
> > > > property
> > > > May be we can take min(User supplied MaxTime, Server MaxTime) as max
> > life
> > > > time.
> > > > I am open to add new config property.
> > > >
> > > > Few more comments based on recent KIP update.
> > > > >
> > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> > > > > {{ExpireTokenRequest}} with with expiryDate set to anything before
> > > > current
> > > > > date?
> > > > >
> > > >
> > > > makes sense. we don't need special request to cancel the token. We
> can
> > > use
> > > > ExpireTokenRequest.
> > > > I'll update the KIP.
> > > >
> > > >
> > > > > 2. Can we change time field names to indicate their unit is
> > > milliseconds,
> > > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > > >
> > > > >
> > > > Done.
> > > >
> > > >
> > > > > 3. Can we allow users to renew a token for a specified amount of
> > time?
> > > In
> > > > > current version of KIP, renew request does not take time as a
> param,
> > > not
> > > > > sure what is expiry time set to after renewal.
> > > > >
> > > > >
> > > > Yes, we need to specify renew period. I'll update the KIP.
> > > >
> > > >
> > > > Thanks,
> > > > Mankumar
> > > >
> > > >
> > > >
> > > > >
> > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <
> manikumar.reddy@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > >
> > > > > >
> > > > > > I would like to reinitiate the discussion on Delegation token
> > support
> > > > for
> > > > > >
> > > > > > Kafka.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Brief summary of the past discussion:
> > > > > >
> > > > > >
> > > > > >
> > > > > > 1) Broker stores delegation tokens in zookeeper. All brokers
> will
> > > > have a
> > > > > >
> > > > > > cache backed by
> > > > > >
> > > > > > zookeeper so they will all get notified whenever a new token
> is
> > > > > >
> > > > > > generated and they will
> > > > > >
> > > > > > update their local cache whenever token state changes.
> > > > > >
> > > > > > 2) The current proposal does not support rotation of secret
> > > > > >
> > > > > > 3) Only allow the renewal by users that authenticated using *non*
> > > > > >
> > > > > > delegation token mechanism
> > > > > >
> > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka
> clients
> > > can
> > > > > >
> > > > > > authenticate using
> > > > > >
> > > > > > SCRAM-SHA-256, providing the delegation token HMAC as
> password.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Updated the KIP with the following:
> > > > > >
> > > > > > 1. Protocol and Config changes
> > > > > >
> > > > > > 2. format of the data stored in ZK.
> > > > > >
> > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > > >
> > > > > >
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > >
> > > > > > 48+Delegation+token+support+for+Kafka
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Jun, Ashish, Gwen,
> > > > > >
> > > > > >
> > > > > >
> > > > > > Pl review the updated KIP.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Manikumar
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <
> asingh@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Harsha/ Gwen,
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > How do we proceed here? I am willing to help out with here.
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> > gwen@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > > Is it updated? are all concerns addressed? do you want to
> > start a
> > > > > vote?
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > Sorry for being pushy, I do appreciate that we are all
> > volunteers
> > > > and
> > > > > >
> > > > > > > > finding time is difficult. This feature is important for
> > anything
> > > > > that
> > > > > >
> > > > > > > > integrates with Kafka (stream processors, Flume, NiFi, etc)
> > and I
> > > > > >
> > > > > > > > don't want to see this getting stuck because we lack
> > coordination
> > > > > >
> > > > > > > > within the community.
> > > > > >
> > > > > > > >
> > > > > >
> > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > > > kafka@harsha.io>
> > > > > >
> > > > > > > > wrote:
> > > > > >
> > > > > > > > > The only pending update for the KIP is to write up the
> > protocol
> > > > > > changes
> > > > > >
> > > > > > > > like
> > > > > >
> > > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > >
> > > > > >
> > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > > asingh@cloudera.com>
> > > > > >
> > > > > > > > wrote:
> > > > > >
> > > > > > > > >>
> > > > > >
> > > > > > > > >> I think we decided to not support secret rotation, I guess
> > > this
> > > > > can
> > > > > > be
> > > > > >
> > > > > > > > >> stated clearly on the KIP. Also, more details on how
> clients
> > > > will
> > > > > >
> > > > > > > > perform
> > > > > >
> > > > > > > > >> token distribution and how CLI will look like will be
> > helpful.
> > > > > >
> > > > > > > > >>
> > > > > >
> > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > > gwen@confluent.io>
> > > > > >
> > > > > > > > wrote:
> > > > > >
> > > > > > > > >>
> > > > > >
> > > > > > > > >> > Hi Guys,
> > > > > >
> > > > > > > > >> >
> > > > > >
> > > > > > > > >> > This discussion was dead for a while. Are there still
> > > > > contentious
> > > > > >
> > > > > > > > >> > points? If not, why are there no votes?
> > > > > >
> > > > > > > > >> >
> > > > > >
> > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> > jun@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > > >> > > Ashish,
> > > > > >
> > > > > > > > >> > >
> > > > > >
> > > > > > > > >> > > Yes, I will send out a KIP invite for next week to
> > discuss
> > > > > > KIP-48
> > > > > >
> > > > > > > > and
> > > > > >
> > > > > > > > >> > other
> > > > > >
> > > > > > > > >> > > remaining KIPs.
> > > > > >
> > > > > > > > >> > >
> > > > > >
> > > > > > > > >> > > Thanks,
> > > > > >
> > > > > > > > >> > >
> > > > > >
> > > > > > > > >> > > Jun
> > > > > >
> > > > > > > > >> > >
> > > > > >
> > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > > > >
> > > > > > > asingh@cloudera.com>
> > > > > >
> > > > > > > > >> > wrote:
> > > > > >
> > > > > > > > >> > >
> > > > > >
> > > > > > > > >> > >> Thanks Harsha!
> > > > > >
> > > > > > > > >> > >>
> > > > > >
> > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda.
> > > Also,
> > > > we
> > > > > > did
> > > > > >
> > > > > > > > not
> > > > > >
> > > > > > > > >> > >> actually make a call on when we should have next KIP
> > > call.
> > > > As
> > > > > >
> > > > > > > there
> > > > > >
> > > > > > > > >> > >> are
> > > > > >
> > > > > > > > >> > a
> > > > > >
> > > > > > > > >> > >> few outstanding KIPs that could not be discussed this
> > > week,
> > > > > can
> > > > > >
> > > > > > > we
> > > > > >
> > > > > > > > >> > >> have
> > > > > >
> > > > > > > > >> > a
> > > > > >
> > > > > > > > >> > >> KIP hangout call next week?
> > > > > >
> > > > > > > > >> > >>
> > > > > >
> > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> > > > > >
> > > > > > > > >> > >> <ka...@harsha.io>
> > > > > >
> > > > > > > > >> > >> wrote:
> > > > > >
> > > > > > > > >> > >>
> > > > > >
> > > > > > > > >> > >>> Ashish,
> > > > > >
> > > > > > > > >> > >>> Yes we are working on it. Lets discuss in
> the
> > > next
> > > > > KIP
> > > > > >
> > > > > > > > >> > >>> meeting.
> > > > > >
> > > > > > > > >> > >>> I'll join.
> > > > > >
> > > > > > > > >> > >>> -Harsha
> > > > > >
> > > > > > > > >> > >>>
> > > > > >
> > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > > > > >
> > > > > > > > asingh@cloudera.com>
> > > > > >
> > > > > > > > >> > >>> wrote:
> > > > > >
> > > > > > > > >> > >>>
> > > > > >
> > > > > > > > >> > >>> > Hello Harsha,
> > > > > >
> > > > > > > > >> > >>> >
> > > > > >
> > > > > > > > >> > >>> > Are you still working on this? Wondering if we can
> > > > discuss
> > > > > >
> > > > > > > this
> > > > > >
> > > > > > > > in
> > > > > >
> > > > > > > > >> > next
> > > > > >
> > > > > > > > >> > >>> KIP
> > > > > >
> > > > > > > > >> > >>> > meeting, if you can join.
> > > > > >
> > > > > > > > >> > >>> >
> > > > > >
> > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> > Chintalapani <
> > > > > >
> > > > > > > > >> > kafka@harsha.io>
> > > > > >
> > > > > > > > >> > >>> > wrote:
> > > > > >
> > > > > > > > >> > >>> >
> > > > > >
> > > > > > > > >> > >>> > > Hi Grant,
> > > > > >
> > > > > > > > >> > >>> > > We are working on it. Will add the
> > details
> > > > to
> > > > > > KIP
> > > > > >
> > > > > > > > >> > >>> > > about
> > > > > >
> > > > > > > > >> > the
> > > > > >
> > > > > > > > >> > >>> > > request protocol.
> > > > > >
> > > > > > > > >> > >>> > >
> > > > > >
> > > > > > > > >> > >>> > > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > Harsha
> > > > > >
> > > > > > > > >> > >>> > >
> > > > > >
> > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > > > > >
> > > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > > >
> > > > > > > > >> > >>> wrote:
> > > > > >
> > > > > > > > >> > >>> > >
> > > > > >
> > > > > > > > >> > >>> > > > Hi Parth,
> > > > > >
> > > > > > > > >> > >>> > > >
> > > > > >
> > > > > > > > >> > >>> > > > Are you still working on this? If you need any
> > > help
> > > > > > please
> > > > > >
> > > > > > > > >> > >>> > > > don't
> > > > > >
> > > > > > > > >> > >>> > hesitate
> > > > > >
> > > > > > > > >> > >>> > > > to ask.
> > > > > >
> > > > > > > > >> > >>> > > >
> > > > > >
> > > > > > > > >> > >>> > > > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > > Grant
> > > > > >
> > > > > > > > >> > >>> > > >
> > > > > >
> > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > > > > >
> > > > > > > jun@confluent.io>
> > > > > >
> > > > > > > > >> > wrote:
> > > > > >
> > > > > > > > >> > >>> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > Parth,
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > It makes sense to only allow the renewal by
> > > users
> > > > > that
> > > > > >
> > > > > > > > >> > >>> authenticated
> > > > > >
> > > > > > > > >> > >>> > > > using
> > > > > >
> > > > > > > > >> > >>> > > > > *non* delegation token mechanism. Then,
> should
> > > we
> > > > > make
> > > > > >
> > > > > > > the
> > > > > >
> > > > > > > > >> > >>> renewal a
> > > > > >
> > > > > > > > >> > >>> > > > list?
> > > > > >
> > > > > > > > >> > >>> > > > > For example, in the case of rest proxy, it
> > will
> > > be
> > > > > >
> > > > > > > useful
> > > > > >
> > > > > > > > >> > >>> > > > > for
> > > > > >
> > > > > > > > >> > >>> every
> > > > > >
> > > > > > > > >> > >>> > > > > instance of rest proxy to be able to renew
> the
> > > > > tokens.
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > It would be clearer if we can document the
> > > request
> > > > > >
> > > > > > > > protocol
> > > > > >
> > > > > > > > >> > like
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > uence/display/KAFKA/KIP-
> > > > > >
> > > > > > > > >> > >>> > >
> > > > > >
> > > > > > > > >> > >>> > > 4+-+Command+line+and+
> centralized+administrative+
> > > > > >
> > > > > > > > operations#KIP-4-
> > > > > >
> > > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> > istrativeoperations-
> > > > > >
> > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > > 2945):(VotedandPlannedforin0.
> > > > > >
> > > > > > > > 10.1.0)
> > > > > >
> > > > > > > > >> > >>> > > > > .
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > It would also be useful to document the
> client
> > > > APIs.
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > Jun
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> > > brahmbhatt
> > > > <
> > > > > >
> > > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > Hi,
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > I am suggesting that we will only allow
> the
> > > > > renewal
> > > > > > by
> > > > > >
> > > > > > > > >> > >>> > > > > > users
> > > > > >
> > > > > > > > >> > >>> that
> > > > > >
> > > > > > > > >> > >>> > > > > > authenticated using *non* delegation token
> > > > > > mechanism.
> > > > > >
> > > > > > > > For
> > > > > >
> > > > > > > > >> > >>> example,
> > > > > >
> > > > > > > > >> > >>> > If
> > > > > >
> > > > > > > > >> > >>> > > > > user
> > > > > >
> > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos and
> > > requested
> > > > > >
> > > > > > > > >> > >>> > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> tokens,
> > > > > >
> > > > > > > > >> > >>> > > > only
> > > > > >
> > > > > > > > >> > >>> > > > > > user Alice authenticated via non
> delegation
> > > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > mechanism
> > > > > >
> > > > > > > > >> > can
> > > > > >
> > > > > > > > >> > >>> > > renew.
> > > > > >
> > > > > > > > >> > >>> > > > > > Clients that have access to delegation
> > tokens
> > > > can
> > > > > > not
> > > > > >
> > > > > > > > >> > >>> > > > > > issue
> > > > > >
> > > > > > > > >> > >>> > renewal
> > > > > >
> > > > > > > > >> > >>> > > > > > request for renewing their own token and
> > this
> > > is
> > > > > >
> > > > > > > > primarily
> > > > > >
> > > > > > > > >> > >>> > important
> > > > > >
> > > > > > > > >> > >>> > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > reduce the time window for which a
> > compromised
> > > > > token
> > > > > >
> > > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > be
> > > > > >
> > > > > > > > >> > >>> valid.
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user can
> > > > request
> > > > > >
> > > > > > > > >> > >>> > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > tokens
> > > > > >
> > > > > > > > >> > >>> > > > but
> > > > > >
> > > > > > > > >> > >>> > > > > > even here I would recommend to avoid
> > creating
> > > a
> > > > > > chain
> > > > > >
> > > > > > > > >> > >>> > > > > > where a
> > > > > >
> > > > > > > > >> > >>> > client
> > > > > >
> > > > > > > > >> > >>> > > > > > authenticated via delegation token request
> > for
> > > > > more
> > > > > >
> > > > > > > > >> > delegation
> > > > > >
> > > > > > > > >> > >>> > > tokens.
> > > > > >
> > > > > > > > >> > >>> > > > > > Basically anyone can request delegation
> > token,
> > > > as
> > > > > > long
> > > > > >
> > > > > > > > as
> > > > > >
> > > > > > > > >> > they
> > > > > >
> > > > > > > > >> > >>> > > > > authenticate
> > > > > >
> > > > > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > > >
> > > > > > > > >> > >>> > > > > > <
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > > uence/display/KAFKA/KIP-
> > > > > >
> > > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > > >
> > > > > > > r+Kafka#KIP-48Delegationtokens
> > > > > >
> > > > > > > > >> > >>> upportforKaf
> > > > > >
> > > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > sufficient?
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > Thanks
> > > > > >
> > > > > > > > >> > >>> > > > > > Parth
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > > > > >
> > > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > > >
> > > > > > > > >> > >>> wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > Parth,
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of
> comments
> > > > > inline
> > > > > >
> > > > > > > > below.
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> > > > > > brahmbhatt <
> > > > > >
> > > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By
> > > original
> > > > > >
> > > > > > > > requester
> > > > > >
> > > > > > > > >> > >>> only? or
> > > > > >
> > > > > > > > >> > >>> > > > using
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Kerberos
> > > > > >
> > > > > > > > >> > >>> > > > > > > > auth only?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > My recommendation is to do this only
> > using
> > > > > >
> > > > > > > Kerberos
> > > > > >
> > > > > > > > >> > >>> > > > > > > > auth
> > > > > >
> > > > > > > > >> > and
> > > > > >
> > > > > > > > >> > >>> > only
> > > > > >
> > > > > > > > >> > >>> > > > > threw
> > > > > >
> > > > > > > > >> > >>> > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > renewer specified during the
> acquisition
> > > > > > request.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are
> you
> > > > saying
> > > > > >
> > > > > > > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > any
> > > > > >
> > > > > > > > >> > >>> client
> > > > > >
> > > > > > > > >> > >>> > > > > > > authenticated with the delegation token
> > can
> > > > > renew,
> > > > > >
> > > > > > > > i.e.
> > > > > >
> > > > > > > > >> > there
> > > > > >
> > > > > > > > >> > >>> is
> > > > > >
> > > > > > > > >> > >>> > no
> > > > > >
> > > > > > > > >> > >>> > > > > > renewer
> > > > > >
> > > > > > > > >> > >>> > > > > > > needed?
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > Also, just to be clear, any
> authenticated
> > > > client
> > > > > >
> > > > > > > > (either
> > > > > >
> > > > > > > > >> > >>> through
> > > > > >
> > > > > > > > >> > >>> > > SASL
> > > > > >
> > > > > > > > >> > >>> > > > > or
> > > > > >
> > > > > > > > >> > >>> > > > > > > SSL) can request a delegation token for
> > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > authenticated
> > > > > >
> > > > > > > > >> > >>> user,
> > > > > >
> > > > > > > > >> > >>> > > > right?
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or
> > in
> > > > ZK?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > My recommendation is still to store in
> > ZK
> > > or
> > > > > not
> > > > > >
> > > > > > > > store
> > > > > >
> > > > > > > > >> > them
> > > > > >
> > > > > > > > >> > >>> at
> > > > > >
> > > > > > > > >> > >>> > > all.
> > > > > >
> > > > > > > > >> > >>> > > > > The
> > > > > >
> > > > > > > > >> > >>> > > > > > > > whole controller based distribution is
> > too
> > > > > much
> > > > > >
> > > > > > > > >> > >>> > > > > > > > overhead
> > > > > >
> > > > > > > > >> > >>> with
> > > > > >
> > > > > > > > >> > >>> > not
> > > > > >
> > > > > > > > >> > >>> > > > > much
> > > > > >
> > > > > > > > >> > >>> > > > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > achieve.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated /
> expired?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Either by expiration time out or
> through
> > > an
> > > > > >
> > > > > > > explicit
> > > > > >
> > > > > > > > >> > >>> request to
> > > > > >
> > > > > > > > >> > >>> > > > > > > invalidate.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > SCRAM
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation proposal
> > (it
> > > > > wasn't
> > > > > >
> > > > > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > the
> > > > > >
> > > > > > > > >> > KIP
> > > > > >
> > > > > > > > >> > >>> but
> > > > > >
> > > > > > > > >> > >>> > > was
> > > > > >
> > > > > > > > >> > >>> > > > > > > > discussed
> > > > > >
> > > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > There is no imperonation proposal. I
> > tried
> > > > and
> > > > > >
> > > > > > > > >> > >>> > > > > > > > explained
> > > > > >
> > > > > > > > >> > how
> > > > > >
> > > > > > > > >> > >>> > its
> > > > > >
> > > > > > > > >> > >>> > > a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > different problem and why its not
> really
> > > > > > necessary
> > > > > >
> > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> discuss
> > > > > >
> > > > > > > > >> > >>> > > that
> > > > > >
> > > > > > > > >> > >>> > > > as
> > > > > >
> > > > > > > > >> > >>> > > > > > > part
> > > > > >
> > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not
> support
> > > any
> > > > > >
> > > > > > > > >> > impersonation,
> > > > > >
> > > > > > > > >> > >>> it
> > > > > >
> > > > > > > > >> > >>> > > will
> > > > > >
> > > > > > > > >> > >>> > > > > just
> > > > > >
> > > > > > > > >> > >>> > > > > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for
> what
> > > > > > actions?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > Could we document the format of the new
> > > > > >
> > > > > > > > request/response
> > > > > >
> > > > > > > > >> > and
> > > > > >
> > > > > > > > >> > >>> > their
> > > > > >
> > > > > > > > >> > >>> > > > > > > associated Resource and Operation for
> ACL?
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > 7. How would the delegation token be
> > > > > configured
> > > > > > in
> > > > > >
> > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> client?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Should be through config. I wasn't
> > > planning
> > > > on
> > > > > >
> > > > > > > > >> > >>> > > > > > > > supporting
> > > > > >
> > > > > > > > >> > >>> JAAS
> > > > > >
> > > > > > > > >> > >>> > > for
> > > > > >
> > > > > > > > >> > >>> > > > > > > tokens.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does this
> either.
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Thanks
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Parth
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun
> > Rao <
> > > > > >
> > > > > > > > >> > jun@confluent.io>
> > > > > >
> > > > > > > > >> > >>> > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > Another question.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation token be
> > > > > > configured
> > > > > >
> > > > > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > client?
> > > > > >
> > > > > > > > >> > >>> > > > The
> > > > > >
> > > > > > > > >> > >>> > > > > > > > standard
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > way is to do this through JAAS.
> > However,
> > > > we
> > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > need
> > > > > >
> > > > > > > > >> > to
> > > > > >
> > > > > > > > >> > >>> > think
> > > > > >
> > > > > > > > >> > >>> > > > > > through
> > > > > >
> > > > > > > > >> > >>> > > > > > > if
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > > > environment.
> > > > > > For
> > > > > >
> > > > > > > > >> > example,
> > > > > >
> > > > > > > > >> > >>> > when a
> > > > > >
> > > > > > > > >> > >>> > > > new
> > > > > >
> > > > > > > > >> > >>> > > > > > > task
> > > > > >
> > > > > > > > >> > >>> > > > > > > > is
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > added to a Storm worker node, do we
> > need
> > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > dynamically
> > > > > >
> > > > > > > > >> > >>> add a
> > > > > >
> > > > > > > > >> > >>> > > new
> > > > > >
> > > > > > > > >> > >>> > > > > > > section
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> > > > convenient
> > > > > if
> > > > > >
> > > > > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > can
> > > > > >
> > > > > > > > >> > >>> pass in
> > > > > >
> > > > > > > > >> > >>> > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > through the config directly w/o
> going
> > > > > through
> > > > > >
> > > > > > > > JAAS.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > Are you or Parth still actively
> > working
> > > on
> > > > > > this
> > > > > >
> > > > > > > > KIP?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > Jun
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun
> > > Rao <
> > > > > >
> > > > > > > > >> > >>> jun@confluent.io>
> > > > > >
> > > > > > > > >> > >>> > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to document
> the
> > > > format
> > > > > > of
> > > > > >
> > > > > > > > the
> > > > > >
> > > > > > > > >> > data
> > > > > >
> > > > > > > > >> > >>> > stored
> > > > > >
> > > > > > > > >> > >>> > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > ZK.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion
> > on
> > > > > > whether
> > > > > >
> > > > > > > > the
> > > > > >
> > > > > > > > >> > tokens
> > > > > >
> > > > > > > > >> > >>> > > should
> > > > > >
> > > > > > > > >> > >>> > > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > > config/acl/quota,
> > > > > >
> > > > > > > or
> > > > > >
> > > > > > > > >> > through
> > > > > >
> > > > > > > > >> > >>> the
> > > > > >
> > > > > > > > >> > >>> > > > > > > controller.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > Currently, the controller is only
> > > > designed
> > > > > > for
> > > > > >
> > > > > > > > >> > >>> propagating
> > > > > >
> > > > > > > > >> > >>> > > > topic
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > metadata,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the
> > > token
> > > > > >
> > > > > > > instead
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > of
> > > > > >
> > > > > > > > >> > >>> > > DIGEST-MD5
> > > > > >
> > > > > > > > >> > >>> > > > > > since
> > > > > >
> > > > > > > > >> > >>> > > > > > > > it's
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem
> > > > broken.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > Jun
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM,
> > Gwen
> > > > > >
> > > > > > > Shapira <
> > > > > >
> > > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> From what I can see, remaining
> > > > questions
> > > > > > are:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed?
> By
> > > > > > original
> > > > > >
> > > > > > > > >> > requester
> > > > > >
> > > > > > > > >> > >>> > only?
> > > > > >
> > > > > > > > >> > >>> > > > or
> > > > > >
> > > > > > > > >> > >>> > > > > > > using
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each
> broker
> > > or
> > > > in
> > > > > > ZK?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated /
> > > > expired?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is
> > > used?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation
> > proposal
> > > > (it
> > > > > >
> > > > > > > > wasn't
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > >
> > > > > > > > >> > the
> > > > > >
> > > > > > > > >> > >>> > KIP
> > > > > >
> > > > > > > > >> > >>> > > > but
> > > > > >
> > > > > > > > >> > >>> > > > > > was
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so -
> for
> > > > what
> > > > > >
> > > > > > > > actions?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM,
> > > Harsha
> > > > <
> > > > > >
> > > > > > > > >> > >>> kafka@harsha.io>
> > > > > >
> > > > > > > > >> > >>> > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >
> > > > Unfortunately
> > > > > I
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > > >
> > > > > > > > >> > >>> attend
> > > > > >
> > > > > > > > >> > >>> > > the
> > > > > >
> > > > > > > > >> > >>> > > > > KIP
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > meeting
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > when
> > > > > delegation
> > > > > >
> > > > > > > > tokens
> > > > > >
> > > > > > > > >> > >>> > discussed.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Appreciate
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > if
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > you
> can
> > > > update
> > > > > > the
> > > > > >
> > > > > > > > >> > thread if
> > > > > >
> > > > > > > > >> > >>> > you
> > > > > >
> > > > > > > > >> > >>> > > > have
> > > > > >
> > > > > > > > >> > >>> > > > > > any
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >
> further
> > > > > > questions.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32
> > AM,
> > > > > Liquan
> > > > > >
> > > > > > > Pei
> > > > > >
> > > > > > > > >> > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the links to
> > images
> > > in
> > > > > the
> > > > > >
> > > > > > > KIP
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > > >
> > > > > > > > >> > >>> > broken.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33
> AM,
> > > > parth
> > > > > >
> > > > > > > > >> > brahmbhatt <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > > getDelegationTokenAs
> > > > > >
> > > > > > > mean?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > In the current proposal we
> > only
> > > > > allow
> > > > > > a
> > > > > >
> > > > > > > > user
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > > >
> > > > > > > > >> > >>> get
> > > > > >
> > > > > > > > >> > >>> > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > > authenticated
> > > > > as
> > > > > >
> > > > > > > > using
> > > > > >
> > > > > > > > >> > >>> another
> > > > > >
> > > > > > > > >> > >>> > > > > > mechanism,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > i.e.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> A user
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate using a
> > keytab
> > > > for
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > > >
> > > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> will get
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that
> > user
> > > > > only.
> > > > > > In
> > > > > >
> > > > > > > > >> > future I
> > > > > >
> > > > > > > > >> > >>> > think
> > > > > >
> > > > > > > > >> > >>> > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > have
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such that we
> > > allow
> > > > > some
> > > > > >
> > > > > > > set
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > > >
> > > > > > > > >> > >>> users (
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM
> ,
> > > > > >
> > > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > > >
> > > > > > > > >> > >>> > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > acquire
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf
> of
> > > > other
> > > > > >
> > > > > > > users
> > > > > >
> > > > > > > > >> > whose
> > > > > >
> > > > > > > > >> > >>> > > identity
> > > > > >
> > > > > > > > >> > >>> > > > > > they
> > > > > >
> > > > > > > > >> > >>> > > > > > > > have
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > verified independently.
> Kafka
> > > > > brokers
> > > > > >
> > > > > > > > will
> > > > > >
> > > > > > > > >> > have
> > > > > >
> > > > > > > > >> > >>> ACLs
> > > > > >
> > > > > > > > >> > >>> > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > control
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> which
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> > impersonate
> > > > > other
> > > > > >
> > > > > > > > users
> > > > > >
> > > > > > > > >> > and
> > > > > >
> > > > > > > > >> > >>> get
> > > > > >
> > > > > > > > >> > >>> > > > tokens
> > > > > >
> > > > > > > > >> > >>> > > > > > on
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation
> > is a
> > > > > whole
> > > > > >
> > > > > > > > >> > different
> > > > > >
> > > > > > > > >> > >>> > > problem
> > > > > >
> > > > > > > > >> > >>> > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > my
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in
> > > > separate
> > > > > >
> > > > > > > KIP.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate
> > of
> > > > > > getting
> > > > > >
> > > > > > > > and
> > > > > >
> > > > > > > > >> > >>> renewing
> > > > > >
> > > > > > > > >> > >>> > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this should be
> very
> > > very
> > > > > > low,
> > > > > >
> > > > > > > 1
> > > > > >
> > > > > > > > >> > request
> > > > > >
> > > > > > > > >> > >>> per
> > > > > >
> > > > > > > > >> > >>> > > > > minute
> > > > > >
> > > > > > > > >> > >>> > > > > > > is a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high estimate.
> > > However
> > > > it
> > > > > >
> > > > > > > > depends
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > > >
> > > > > > > > >> > >>> the
> > > > > >
> > > > > > > > >> > >>> > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> expiration. I am
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > less worried about the extra
> > > load
> > > > it
> > > > > >
> > > > > > > puts
> > > > > >
> > > > > > > > on
> > > > > >
> > > > > > > > >> > >>> > controller
> > > > > >
> > > > > > > > >> > >>> > > > vs
> > > > > >
> > > > > > > > >> > >>> > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > added
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > complexity and the value it
> > > > offers.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > Thanks
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > Parth
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30
> > AM,
> > > > > > Ismael
> > > > > >
> > > > > > > > Juma
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > <
> > > > > >
> > > > > > > > >> > >>> > > > > > > ismael@juma.me.uk>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would
> > > probably
> > > > > >
> > > > > > > > require a
> > > > > >
> > > > > > > > >> > >>> separate
> > > > > >
> > > > > > > > >> > >>> > > KIP
> > > > > >
> > > > > > > > >> > >>> > > > > as
> > > > > >
> > > > > > > > >> > >>> > > > > > it
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > introduce user visible
> > > changes.
> > > > We
> > > > > >
> > > > > > > could
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > also
> > > > > >
> > > > > > > > >> > >>> > update
> > > > > >
> > > > > > > > >> > >>> > > > > KIP-48
> > > > > >
> > > > > > > > >> > >>> > > > > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> have this
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > information, but it seems
> > > > cleaner
> > > > > to
> > > > > >
> > > > > > > do
> > > > > >
> > > > > > > > it
> > > > > >
> > > > > > > > >> > >>> > > separately.
> > > > > >
> > > > > > > > >> > >>> > > > We
> > > > > >
> > > > > > > > >> > >>> > > > > > can
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> discuss
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > Ismael
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at
> 3:19
> > > PM,
> > > > > >
> > > > > > > Rajini
> > > > > >
> > > > > > > > >> > Sivaram
> > > > > >
> > > > > > > > >> > >>> <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > >
> > rajinisivaram@googlemail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > Ismael,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > > > > >
> > > > > > > > >> > jira/browse/KAFKA-3751)
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a
> SASL
> > > > > >
> > > > > > > mechanism.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > Would
> > > > > >
> > > > > > > > >> > >>> that
> > > > > >
> > > > > > > > >> > >>> > > need
> > > > > >
> > > > > > > > >> > >>> > > > > > > another
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> KIP? If
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this
> > > > mechanism,
> > > > > > can
> > > > > >
> > > > > > > > this
> > > > > >
> > > > > > > > >> > just
> > > > > >
> > > > > > > > >> > >>> be
> > > > > >
> > > > > > > > >> > >>> > a
> > > > > >
> > > > > > > > >> > >>> > > > JIRA
> > > > > >
> > > > > > > > >> > >>> > > > > > > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > gets
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > reviewed
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > Thank you,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > Rajini
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at
> > 2:46
> > > > PM,
> > > > > >
> > > > > > > > Ismael
> > > > > >
> > > > > > > > >> > Juma <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > ismael@juma.me.uk>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM
> > seems
> > > > > like
> > > > > > a
> > > > > >
> > > > > > > > good
> > > > > >
> > > > > > > > >> > >>> > candidate.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently
> > > > > mentioned
> > > > > >
> > > > > > > > this
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > as
> > > > > >
> > > > > > > > >> > a
> > > > > >
> > > > > > > > >> > >>> SASL
> > > > > >
> > > > > > > > >> > >>> > > > > > mechanism
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> might
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I
> > > have
> > > > > been
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > meaning
> > > > > >
> > > > > > > > >> > to
> > > > > >
> > > > > > > > >> > >>> > check
> > > > > >
> > > > > > > > >> > >>> > > > it
> > > > > >
> > > > > > > > >> > >>> > > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > more
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> detail.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > Good
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > to know that you are
> > > willing
> > > > > to
> > > > > >
> > > > > > > > >> > contribute
> > > > > >
> > > > > > > > >> > >>> an
> > > > > >
> > > > > > > > >> > >>> > > > > > > > implementation.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> Maybe
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > should file a separate
> > > JIRA
> > > > > for
> > > > > >
> > > > > > > > this?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > Ismael
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016
> at
> > > 2:12
> > > > > PM,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > Rajini
> > > > > >
> > > > > > > > >> > >>> > Sivaram <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > rajinisivaram@googlemail.com>
> > > > > >
> > > > > > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted
> > Challenge
> > > > > > Response
> > > > > >
> > > > > > > > >> > >>> > Authentication
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Mechanism)
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> is a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > better
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than
> > > Digest-MD5.
> > > > > > Java
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> > > > > >
> > > > > > > > >> > >>> come
> > > > > >
> > > > > > > > >> > >>> > > > with a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > built-in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> SCRAM
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or
> > > SaslClient,
> > > > > but
> > > > > > I
> > > > > >
> > > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > be
> > > > > >
> > > > > > > > >> > >>> happy
> > > > > >
> > > > > > > > >> > >>> > > to
> > > > > >
> > > > > > > > >> > >>> > > > > add
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > support
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > since
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful
> > > > > mechanism
> > > > > >
> > > > > > > to
> > > > > >
> > > > > > > > >> > support
> > > > > >
> > > > > > > > >> > >>> > > anyway.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > > https://tools.ietf.org/html/
> > > > > >
> > > > > > > > rfc7677
> > > > > >
> > > > > > > > >> > >>> > describes
> > > > > >
> > > > > > > > >> > >>> > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > protocol
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> for
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016
> > at
> > > > 2:37
> > > > > > AM,
> > > > > >
> > > > > > > > Jun
> > > > > >
> > > > > > > > >> > Rao <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > jun@confluent.io
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the
> > > > > explanation.
> > > > > > A
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > couple
> > > > > >
> > > > > > > > >> > of
> > > > > >
> > > > > > > > >> > >>> > more
> > > > > >
> > > > > > > > >> > >>> > > > > > > questions.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> getDelegationTokenAs
> > > > > >
> > > > > > > > >> > >>> mean?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the
> > > typical
> > > > > rate
> > > > > >
> > > > > > > of
> > > > > >
> > > > > > > > >> > getting
> > > > > >
> > > > > > > > >> > >>> and
> > > > > >
> > > > > > > > >> > >>> > > > > > renewing
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > tokens?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an
> > > impact
> > > > on
> > > > > >
> > > > > > > > whether
> > > > > >
> > > > > > > > >> > they
> > > > > >
> > > > > > > > >> > >>> > > should
> > > > > >
> > > > > > > > >> > >>> > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > directed
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> to the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23,
> 2016
> > > at
> > > > > 1:19
> > > > > >
> > > > > > > PM,
> > > > > >
> > > > > > > > >> > parth
> > > > > >
> > > > > > > > >> > >>> > > > > brahmbhatt <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > brahmbhatt.parth@gmail.com>
> > > > > >
> > > > > > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for
> > > reviewing.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a
> > > > Cluster
> > > > > >
> > > > > > > > action
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > > >
> > > > > > > > >> > add
> > > > > >
> > > > > > > > >> > >>> > acls
> > > > > >
> > > > > > > > >> > >>> > > > on
> > > > > >
> > > > > > > > >> > >>> > > > > > who
> > > > > >
> > > > > > > > >> > >>> > > > > > > > can
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> request
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't
> > see
> > > > the
> > > > > > use
> > > > > >
> > > > > > > > case
> > > > > >
> > > > > > > > >> > for
> > > > > >
> > > > > > > > >> > >>> that
> > > > > >
> > > > > > > > >> > >>> > > yet
> > > > > >
> > > > > > > > >> > >>> > > > > but
> > > > > >
> > > > > > > > >> > >>> > > > > > > > down
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> the line
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > when
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > > > > >
> > > > > > > > >> > getDelegationTokenAs
> > > > > >
> > > > > > > > >> > >>> it
> > > > > >
> > > > > > > > >> > >>> > > will
> > > > > >
> > > > > > > > >> > >>> > > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> necessary.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we
> recommend
> > > > > tokens
> > > > > > to
> > > > > >
> > > > > > > > be
> > > > > >
> > > > > > > > >> > only
> > > > > >
> > > > > > > > >> > >>> > > > > > > used/distributed
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> over
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > secure
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on
> > what
> > > > > design
> > > > > >
> > > > > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > end
> > > > > >
> > > > > > > > >> > up
> > > > > >
> > > > > > > > >> > >>> > > choosing
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> Invalidation will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility
> of
> > > > every
> > > > > >
> > > > > > > broker
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > or
> > > > > >
> > > > > > > > >> > >>> > > controller.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure
> > if I
> > > > > >
> > > > > > > > documented
> > > > > >
> > > > > > > > >> > >>> somewhere
> > > > > >
> > > > > > > > >> > >>> > > > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> invalidation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > directly
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > go through
> > zookeeper
> > > > but
> > > > > >
> > > > > > > that
> > > > > >
> > > > > > > > is
> > > > > >
> > > > > > > > >> > not
> > > > > >
> > > > > > > > >> > >>> the
> > > > > >
> > > > > > > > >> > >>> > > > > intent.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> Invalidation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > either
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based
> > or
> > > > due
> > > > > to
> > > > > >
> > > > > > > > >> > >>> expiration. No
> > > > > >
> > > > > > > > >> > >>> > > > > direct
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> zookeeper
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > interaction
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > from
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also
> > > stores
> > > > > the
> > > > > >
> > > > > > > > >> > >>> DelegationToken
> > > > > >
> > > > > > > > >> > >>> > > > > without
> > > > > >
> > > > > > > > >> > >>> > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> hmac in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." :
> > Sorry
> > > > > about
> > > > > >
> > > > > > > the
> > > > > >
> > > > > > > > >> > >>> confusion.
> > > > > >
> > > > > > > > >> > >>> > > The
> > > > > >
> > > > > > > > >> > >>> > > > > sole
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> purpose of
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is
> as
> > > > > >
> > > > > > > distribution
> > > > > >
> > > > > > > > >> > channel
> > > > > >
> > > > > > > > >> > >>> > for
> > > > > >
> > > > > > > > >> > >>> > > > > tokens
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> between all
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > brokers
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that
> ensures
> > > > only
> > > > > >
> > > > > > > tokens
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > that
> > > > > >
> > > > > > > > >> > >>> were
> > > > > >
> > > > > > > > >> > >>> > > > > > generated
> > > > > >
> > > > > > > > >> > >>> > > > > > > by
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> making a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > request
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > to a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be
> > > > accepted
> > > > > >
> > > > > > > (more
> > > > > >
> > > > > > > > on
> > > > > >
> > > > > > > > >> > this
> > > > > >
> > > > > > > > >> > >>> in
> > > > > >
> > > > > > > > >> > >>> > > > second
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> paragraph). The
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few
> > > > elements
> > > > > >
> > > > > > > > (owner,
> > > > > >
> > > > > > > > >> > >>> renewer,
> > > > > >
> > > > > > > > >> > >>> > > > uuid
> > > > > >
> > > > > > > > >> > >>> > > > > ,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> expiration,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > hmac)
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > ,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > one
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > of
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the
> > finally
> > > > > >
> > > > > > > generated
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> > > > > >
> > > > > > > > >> > >>> but
> > > > > >
> > > > > > > > >> > >>> > > hmac
> > > > > >
> > > > > > > > >> > >>> > > > it
> > > > > >
> > > > > > > > >> > >>> > > > > > > self
> > > > > >
> > > > > > > > >> > >>> > > > > > > > is
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > derivable
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > if
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > you
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the
> other
> > > > > > elements
> > > > > >
> > > > > > > of
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> token
> > > > > >
> > > > > > > > >> > >>> > +
> > > > > >
> > > > > > > > >> > >>> > > > > secret
> > > > > >
> > > > > > > > >> > >>> > > > > > > key
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > generate
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > hmac.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper
> > does
> > > > not
> > > > > >
> > > > > > > > provide
> > > > > >
> > > > > > > > >> > SSL
> > > > > >
> > > > > > > > >> > >>> > > support
> > > > > >
> > > > > > > > >> > >>> > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > do
> > > > > >
> > > > > > > > >> > >>> > > > > > > > not
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> want the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > entire
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> > > > > > transferred
> > > > > >
> > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> zookeeper
> > > > > >
> > > > > > > > >> > >>> > > as
> > > > > >
> > > > > > > > >> > >>> > > > > that
> > > > > >
> > > > > > > > >> > >>> > > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> be an
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > insecure
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > wire
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer.
> Instead
> > we
> > > > > only
> > > > > >
> > > > > > > > store
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > all
> > > > > >
> > > > > > > > >> > >>> the
> > > > > >
> > > > > > > > >> > >>> > > other
> > > > > >
> > > > > > > > >> > >>> > > > > > > > elements
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> of a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers
> > can
> > > > read
> > > > > >
> > > > > > > these
> > > > > >
> > > > > > > > >> > >>> elements
> > > > > >
> > > > > > > > >> > >>> > and
> > > > > >
> > > > > > > > >> > >>> > > > > > because
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > they
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> also
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > have
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > access
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key
> they
> > > > will
> > > > > be
> > > > > >
> > > > > > > > able
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> > generate
> > > > > >
> > > > > > > > >> > >>> > > > > hmac
> > > > > >
> > > > > > > > >> > >>> > > > > > on
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> their end.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the
> > > alternative
> > > > > >
> > > > > > > > proposed
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > is
> > > > > >
> > > > > > > > >> > to
> > > > > >
> > > > > > > > >> > >>> > avoid
> > > > > >
> > > > > > > > >> > >>> > > > > > > zookeeper
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > altogether. A
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Client
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker
> > > with
> > > > > >
> > > > > > > required
> > > > > >
> > > > > > > > >> > >>> > information
> > > > > >
> > > > > > > > >> > >>> > > > > > (owner,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> renwer,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > expiration)
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed
> > > hmac,
> > > > > >
> > > > > > > uuid).
> > > > > >
> > > > > > > > >> > Broker
> > > > > >
> > > > > > > > >> > >>> > won't
> > > > > >
> > > > > > > > >> > >>> > > > > store
> > > > > >
> > > > > > > > >> > >>> > > > > > > this
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > zookeeper.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > From
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a
> > client
> > > > can
> > > > > >
> > > > > > > > contact
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > > > >
> > > > > > > > >> > >>> > broker
> > > > > >
> > > > > > > > >> > >>> > > > with
> > > > > >
> > > > > > > > >> > >>> > > > > > all
> > > > > >
> > > > > > > > >> > >>> > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > token
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner,
> > rewner,
> > > > > >
> > > > > > > > expiration,
> > > > > >
> > > > > > > > >> > hmac,
> > > > > >
> > > > > > > > >> > >>> > > uuid)
> > > > > >
> > > > > > > > >> > >>> > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > borker
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > regenerate
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long
> > as
> > > it
> > > > > >
> > > > > > > matches
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > with
> > > > > >
> > > > > > > > >> > >>> hmac
> > > > > >
> > > > > > > > >> > >>> > > > > > presented
> > > > > >
> > > > > > > > >> > >>> > > > > > > by
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> client ,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > broker
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the
> request
> > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> > > > > >
> > > > > > > > >> > >>> Only
> > > > > >
> > > > > > > > >> > >>> > > > > problem
> > > > > >
> > > > > > > > >> > >>> > > > > > > with
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> this
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > approach
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > is
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > if
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key
> is
> > > > > >
> > > > > > > compromised
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > > > >
> > > > > > > > >> > >>> client
> > > > > >
> > > > > > > > >> > >>> > > can
> > > > > >
> > > > > > > > >> > >>> > > > > now
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > generate
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > random
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > tokens
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still
> be
> > > > able
> > > > > to
> > > > > >
> > > > > > > > >> > >>> authenticate
> > > > > >
> > > > > > > > >> > >>> > as
> > > > > >
> > > > > > > > >> > >>> > > > any
> > > > > >
> > > > > > > > >> > >>> > > > > > user
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > they
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> like.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > with
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we
> > > guarantee
> > > > > > that
> > > > > >
> > > > > > > > only
> > > > > >
> > > > > > > > >> > >>> tokens
> > > > > >
> > > > > > > > >> > >>> > > > > acquired
> > > > > >
> > > > > > > > >> > >>> > > > > > > via
> > > > > >
> > > > > > > > >> > >>> > > > > > > > a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> broker
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > (using
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > some
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme
> other
> > > than
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> token)
> > > > > >
> > > > > > > > >> > >>> > > will
> > > > > >
> > > > > > > > >> > >>> > > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> accepted. We
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > need
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > to
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which
> > > proposal
> > > > > > makes
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > more
> > > > > >
> > > > > > > > >> > >>> sense
> > > > > >
> > > > > > > > >> > >>> > and
> > > > > >
> > > > > > > > >> > >>> > > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > can
> > > > > >
> > > > > > > > >> > >>> > > > > > > go
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> over it
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > in
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you
> > > forward
> > > > > the
> > > > > >
> > > > > > > > invite
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> me?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23,
> > 2016
> > > > at
> > > > > >
> > > > > > > 10:35
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> > > > > >
> > > > > > > > >> > Jun
> > > > > >
> > > > > > > > >> > >>> > Rao <
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> jun@confluent.io>
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > wrote:
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the
> > > KIP.
> > > > A
> > > > > > few
> > > > > >
> > > > > > > > >> > comments.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This
> > > > potentially
> > > > > > can
> > > > > >
> > > > > > > be
> > > > > >
> > > > > > > > >> > useful
> > > > > >
> > > > > > > > >> > >>> for
> > > > > >
> > > > > > > > >> > >>> > > > Kafka
> > > > > >
> > > > > > > > >> > >>> > > > > > > > Connect
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> and
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > rest
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker
> > > agent
> > > > > > will
> > > > > >
> > > > > > > > need
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> > > > > >
> > > > > > > > >> > >>> run a
> > > > > >
> > > > > > > > >> > >>> > > > task
> > > > > >
> > > > > > > > >> > >>> > > > > on
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > behalf
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> of a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > client.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > We
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to
> > > > change
> > > > > > how
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> > > > > >
> > > > > > > > >> > >>> > services
> > > > > >
> > > > > > > > >> > >>> > > > use
> > > > > >
> > > > > > > > >> > >>> > > > > > > Kafka
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> clients
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > (producer/consumer).
> > > > > >
> > > > > > > Instead
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> > > > > >
> > > > > > > > >> > >>> > shared
> > > > > >
> > > > > > > > >> > >>> > > > > client
> > > > > >
> > > > > > > > >> > >>> > > > > > > per
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> worker,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > will
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > need
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > a
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per
> user
> > > task
> > > > > > since
> > > > > >
> > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > authentication
> > > > > >
> > > > > > > > >> > >>> > > > > > > > happens
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> at the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > connection
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For
> Kafka
> > > > > > Connect,
> > > > > >
> > > > > > > > the
> > > > > >
> > > > > > > > >> > >>> renewer
> > > > > >
> > > > > > > > >> > >>> > > will
> > > > > >
> > > > > > > > >> > >>> > > > be
> > > > > >
> > > > > > > > >> > >>> > > > > > the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> workers.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > So,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > we
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow
> > > > multiple
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> > > > > >
> > > > > > > > >> > For
> > > > > >
> > > > > > > > >> > >>> > > Kafka
> > > > > >
> > > > > > > > >> > >>> > > > > rest
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > proxy,
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> the
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > renewer
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > can
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just
> be
> > > the
> > > > > >
> > > > > > > creator
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> > > > > >
> > > > > > > > >> > the
> > > > > >
> > > > > > > > >> > >>> > > token.
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we
> need
> > > new
> > > > > acl
> > > > > > on
> > > > > >
> > > > > > > > who
> > > > > >
> > > > > > > > >> > can
> > > > > >
> > > > > > > > >> > >>> > > request
> > > > > >
> > > > > > > > >> > >>> > > > > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we
> > > recommend
> > > > > >
> > > > > > > people
> > > > > >
> > > > > > > > to
> > > > > >
> > > > > > > > >> > send
> > > > > >
> > > > > > > > >> > >>> > > > > delegation
> > > > > >
> > > > > > > > >> > >>> > > > > > > > tokens
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> in an
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > encrypted
> > > > > >
> > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > > > > >
> > > > > > > > >> > >>>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *Gwen Shapira*
> > > Product Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > <http://www.confluent.io/blog>
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Rajini Sivaram <rs...@pivotal.io>.
It would clearly be very useful to enable clients to send requests on
behalf of multiple users. A separate KIP makes sense, but it may be worth
thinking through some of the implications now, especially if the main
interest in delegation tokens comes from its potential to enable
impersonation.
I understand that delegation tokens are only expected to be used with TLS.
But the choice of SASL/SCRAM for authentication must be based on a
requirement to protect the tokenHmac - otherwise you could just use
SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated on-the-wire,
only a salted-hashed version of it is used in the SASL authentication
exchange. If impersonation is based on sending tokenHmac in requests, any
benefit of using SCRAM is lost.
An alternative may be to allow clients to authenticate multiple times using
SASL and include one of its authenticated principals in each request
(optionally). I haven't thought it through yet, obviously. But if the
approach is of interest and no one is working on a KIP for impersonation at
the moment, I am happy to write one. It may provide something for
comparison at least.
Thoughts?
On Wed, Dec 14, 2016 at 9:53 AM, Manikumar <ma...@gmail.com>
wrote:
> That's a good idea. Authenticating every request with delegation token will
> be useful for
> impersonation use-cases. But as of now, we are thinking delegation token as
> just another way
> to authenticate the users. We haven't think through all the use cases
> related to
> impersonation or using delegation token for impersonation. We want to
> handle impersonation
> (KAFKA-3712) as part of separate KIP.
>
> Will that be Ok?
>
>
> On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thinking out loud here:
> >
> > It looks like authentication with a delegation token is going to be
> > super-cheap, right? We just compare the token to a value in the broker
> > cache?
> >
> > If I understood the KIP correctly, right now it suggests that
> > authentication happens when establishing the client-broker connection (as
> > normal for Kafka. But perhaps we want to consider authenticating every
> > request with delegation token (if exists)?
> >
> > So a centralized app can create few producers, do the metadata request
> and
> > broker discovery with its own user auth, but then use delegation tokens
> to
> > allow performing produce/fetch requests as different users? Instead of
> > having to re-connect for each impersonated user?
> >
> > This may over-complicate things quite a bit (basically adding extra
> > information in every request), but maybe it will be useful for
> > impersonation use-cases (which seem to drive much of the interest in this
> > KIP)?
> > Kafka Connect, NiFi and friends can probably use this to share clients
> > between multiple jobs, tasks, etc.
> >
> > What do you think?
> >
> > Gwen
> >
> > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Ashish,
> > >
> > > Thank you for reviewing the KIP. Please see the replies inline.
> > >
> > >
> > > > 1. How to disable delegation token authentication?
> > > >
> > > > This can be achieved in various ways, however I think reusing
> > delegation
> > > > token secret config for this makes sense here. Avoids creating yet
> > > another
> > > > config and forces delegation token users to consciously set the
> secret.
> > > If
> > > > the secret is not set or set to empty string, brokers should turn off
> > > > delegation token support. This will however require a new error code
> to
> > > > indicate delegation token support is turned off on broker.
> > > >
> > >
> > > Thanks for the suggestion. Option to turnoff delegation token
> > > authentication will be useful.
> > > I'll update the KIP.
> > >
> > >
> > > >
> > > > 2. ACLs on delegation token?
> > > >
> > > > Do we need to have ACLs defined for tokens? I do not think it buys us
> > > > anything, as delegation token can be treated as impersonation of the
> > > owner.
> > > > Any thing the owner has permission to do, delegation tokens should be
> > > > allowed to do as well. If so, we probably won't need to return
> > > > authorization exception error code while creating delegation token.
> It
> > > > however would make sense to check renew and expire requests are
> coming
> > > from
> > > > owner or renewers of the token, but that does not require explicit
> > acls.
> > > >
> > >
> > >
> > > Yes, We agreed to not have new acl on who can request delegation token.
> > > I'll update the KIP.
> > >
> > >
> > > >
> > > > 3. How to restrict max life time of a token?
> > > >
> > > > Admins might want to restrict max life time of tokens created on a
> > > cluster,
> > > > and this can very from cluster to cluster based on use-cases. This
> > might
> > > > warrant a separate broker config.
> > > >
> > > >
> > > Currently we have "delegation.token.max.lifetime.sec" server config
> > > property
> > > May be we can take min(User supplied MaxTime, Server MaxTime) as max
> life
> > > time.
> > > I am open to add new config property.
> > >
> > > Few more comments based on recent KIP update.
> > > >
> > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> > > > {{ExpireTokenRequest}} with with expiryDate set to anything before
> > > current
> > > > date?
> > > >
> > >
> > > makes sense. we don't need special request to cancel the token. We can
> > use
> > > ExpireTokenRequest.
> > > I'll update the KIP.
> > >
> > >
> > > > 2. Can we change time field names to indicate their unit is
> > milliseconds,
> > > > like, IssueDateMs, ExpiryDateMs, etc.?
> > > >
> > > >
> > > Done.
> > >
> > >
> > > > 3. Can we allow users to renew a token for a specified amount of
> time?
> > In
> > > > current version of KIP, renew request does not take time as a param,
> > not
> > > > sure what is expiry time set to after renewal.
> > > >
> > > >
> > > Yes, we need to specify renew period. I'll update the KIP.
> > >
> > >
> > > Thanks,
> > > Mankumar
> > >
> > >
> > >
> > > >
> > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <manikumar.reddy@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > >
> > > > >
> > > > > I would like to reinitiate the discussion on Delegation token
> support
> > > for
> > > > >
> > > > > Kafka.
> > > > >
> > > > >
> > > > >
> > > > > Brief summary of the past discussion:
> > > > >
> > > > >
> > > > >
> > > > > 1) Broker stores delegation tokens in zookeeper. All brokers will
> > > have a
> > > > >
> > > > > cache backed by
> > > > >
> > > > > zookeeper so they will all get notified whenever a new token is
> > > > >
> > > > > generated and they will
> > > > >
> > > > > update their local cache whenever token state changes.
> > > > >
> > > > > 2) The current proposal does not support rotation of secret
> > > > >
> > > > > 3) Only allow the renewal by users that authenticated using *non*
> > > > >
> > > > > delegation token mechanism
> > > > >
> > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients
> > can
> > > > >
> > > > > authenticate using
> > > > >
> > > > > SCRAM-SHA-256, providing the delegation token HMAC as password.
> > > > >
> > > > >
> > > > >
> > > > > Updated the KIP with the following:
> > > > >
> > > > > 1. Protocol and Config changes
> > > > >
> > > > > 2. format of the data stored in ZK.
> > > > >
> > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > > >
> > > > >
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > >
> > > > > 48+Delegation+token+support+for+Kafka
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Jun, Ashish, Gwen,
> > > > >
> > > > >
> > > > >
> > > > > Pl review the updated KIP.
> > > > >
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > Manikumar
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <asingh@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > >
> > > > >
> > > > > > Harsha/ Gwen,
> > > > >
> > > > > >
> > > > >
> > > > > > How do we proceed here? I am willing to help out with here.
> > > > >
> > > > > >
> > > > >
> > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <
> gwen@confluent.io>
> > > > > wrote:
> > > > >
> > > > > >
> > > > >
> > > > > > > Is it updated? are all concerns addressed? do you want to
> start a
> > > > vote?
> > > > >
> > > > > > >
> > > > >
> > > > > > > Sorry for being pushy, I do appreciate that we are all
> volunteers
> > > and
> > > > >
> > > > > > > finding time is difficult. This feature is important for
> anything
> > > > that
> > > > >
> > > > > > > integrates with Kafka (stream processors, Flume, NiFi, etc)
> and I
> > > > >
> > > > > > > don't want to see this getting stuck because we lack
> coordination
> > > > >
> > > > > > > within the community.
> > > > >
> > > > > > >
> > > > >
> > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > > kafka@harsha.io>
> > > > >
> > > > > > > wrote:
> > > > >
> > > > > > > > The only pending update for the KIP is to write up the
> protocol
> > > > > changes
> > > > >
> > > > > > > like
> > > > >
> > > > > > > > we've it KIP-4. I'll update the wiki.
> > > > >
> > > > > > > >
> > > > >
> > > > > > > >
> > > > >
> > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > > asingh@cloudera.com>
> > > > >
> > > > > > > wrote:
> > > > >
> > > > > > > >>
> > > > >
> > > > > > > >> I think we decided to not support secret rotation, I guess
> > this
> > > > can
> > > > > be
> > > > >
> > > > > > > >> stated clearly on the KIP. Also, more details on how clients
> > > will
> > > > >
> > > > > > > perform
> > > > >
> > > > > > > >> token distribution and how CLI will look like will be
> helpful.
> > > > >
> > > > > > > >>
> > > > >
> > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > > gwen@confluent.io>
> > > > >
> > > > > > > wrote:
> > > > >
> > > > > > > >>
> > > > >
> > > > > > > >> > Hi Guys,
> > > > >
> > > > > > > >> >
> > > > >
> > > > > > > >> > This discussion was dead for a while. Are there still
> > > > contentious
> > > > >
> > > > > > > >> > points? If not, why are there no votes?
> > > > >
> > > > > > > >> >
> > > > >
> > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <
> jun@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > > >> > > Ashish,
> > > > >
> > > > > > > >> > >
> > > > >
> > > > > > > >> > > Yes, I will send out a KIP invite for next week to
> discuss
> > > > > KIP-48
> > > > >
> > > > > > > and
> > > > >
> > > > > > > >> > other
> > > > >
> > > > > > > >> > > remaining KIPs.
> > > > >
> > > > > > > >> > >
> > > > >
> > > > > > > >> > > Thanks,
> > > > >
> > > > > > > >> > >
> > > > >
> > > > > > > >> > > Jun
> > > > >
> > > > > > > >> > >
> > > > >
> > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > > >
> > > > > > asingh@cloudera.com>
> > > > >
> > > > > > > >> > wrote:
> > > > >
> > > > > > > >> > >
> > > > >
> > > > > > > >> > >> Thanks Harsha!
> > > > >
> > > > > > > >> > >>
> > > > >
> > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda.
> > Also,
> > > we
> > > > > did
> > > > >
> > > > > > > not
> > > > >
> > > > > > > >> > >> actually make a call on when we should have next KIP
> > call.
> > > As
> > > > >
> > > > > > there
> > > > >
> > > > > > > >> > >> are
> > > > >
> > > > > > > >> > a
> > > > >
> > > > > > > >> > >> few outstanding KIPs that could not be discussed this
> > week,
> > > > can
> > > > >
> > > > > > we
> > > > >
> > > > > > > >> > >> have
> > > > >
> > > > > > > >> > a
> > > > >
> > > > > > > >> > >> KIP hangout call next week?
> > > > >
> > > > > > > >> > >>
> > > > >
> > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> > > > >
> > > > > > > >> > >> <ka...@harsha.io>
> > > > >
> > > > > > > >> > >> wrote:
> > > > >
> > > > > > > >> > >>
> > > > >
> > > > > > > >> > >>> Ashish,
> > > > >
> > > > > > > >> > >>> Yes we are working on it. Lets discuss in the
> > next
> > > > KIP
> > > > >
> > > > > > > >> > >>> meeting.
> > > > >
> > > > > > > >> > >>> I'll join.
> > > > >
> > > > > > > >> > >>> -Harsha
> > > > >
> > > > > > > >> > >>>
> > > > >
> > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > > > >
> > > > > > > asingh@cloudera.com>
> > > > >
> > > > > > > >> > >>> wrote:
> > > > >
> > > > > > > >> > >>>
> > > > >
> > > > > > > >> > >>> > Hello Harsha,
> > > > >
> > > > > > > >> > >>> >
> > > > >
> > > > > > > >> > >>> > Are you still working on this? Wondering if we can
> > > discuss
> > > > >
> > > > > > this
> > > > >
> > > > > > > in
> > > > >
> > > > > > > >> > next
> > > > >
> > > > > > > >> > >>> KIP
> > > > >
> > > > > > > >> > >>> > meeting, if you can join.
> > > > >
> > > > > > > >> > >>> >
> > > > >
> > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha
> Chintalapani <
> > > > >
> > > > > > > >> > kafka@harsha.io>
> > > > >
> > > > > > > >> > >>> > wrote:
> > > > >
> > > > > > > >> > >>> >
> > > > >
> > > > > > > >> > >>> > > Hi Grant,
> > > > >
> > > > > > > >> > >>> > > We are working on it. Will add the
> details
> > > to
> > > > > KIP
> > > > >
> > > > > > > >> > >>> > > about
> > > > >
> > > > > > > >> > the
> > > > >
> > > > > > > >> > >>> > > request protocol.
> > > > >
> > > > > > > >> > >>> > >
> > > > >
> > > > > > > >> > >>> > > Thanks,
> > > > >
> > > > > > > >> > >>> > > Harsha
> > > > >
> > > > > > > >> > >>> > >
> > > > >
> > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > > > >
> > > > > > > >> > >>> > > <gh...@cloudera.com>
> > > > >
> > > > > > > >> > >>> wrote:
> > > > >
> > > > > > > >> > >>> > >
> > > > >
> > > > > > > >> > >>> > > > Hi Parth,
> > > > >
> > > > > > > >> > >>> > > >
> > > > >
> > > > > > > >> > >>> > > > Are you still working on this? If you need any
> > help
> > > > > please
> > > > >
> > > > > > > >> > >>> > > > don't
> > > > >
> > > > > > > >> > >>> > hesitate
> > > > >
> > > > > > > >> > >>> > > > to ask.
> > > > >
> > > > > > > >> > >>> > > >
> > > > >
> > > > > > > >> > >>> > > > Thanks,
> > > > >
> > > > > > > >> > >>> > > > Grant
> > > > >
> > > > > > > >> > >>> > > >
> > > > >
> > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > > > >
> > > > > > jun@confluent.io>
> > > > >
> > > > > > > >> > wrote:
> > > > >
> > > > > > > >> > >>> > > >
> > > > >
> > > > > > > >> > >>> > > > > Parth,
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > Thanks for the reply.
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > It makes sense to only allow the renewal by
> > users
> > > > that
> > > > >
> > > > > > > >> > >>> authenticated
> > > > >
> > > > > > > >> > >>> > > > using
> > > > >
> > > > > > > >> > >>> > > > > *non* delegation token mechanism. Then, should
> > we
> > > > make
> > > > >
> > > > > > the
> > > > >
> > > > > > > >> > >>> renewal a
> > > > >
> > > > > > > >> > >>> > > > list?
> > > > >
> > > > > > > >> > >>> > > > > For example, in the case of rest proxy, it
> will
> > be
> > > > >
> > > > > > useful
> > > > >
> > > > > > > >> > >>> > > > > for
> > > > >
> > > > > > > >> > >>> every
> > > > >
> > > > > > > >> > >>> > > > > instance of rest proxy to be able to renew the
> > > > tokens.
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > It would be clearer if we can document the
> > request
> > > > >
> > > > > > > protocol
> > > > >
> > > > > > > >> > like
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > uence/display/KAFKA/KIP-
> > > > >
> > > > > > > >> > >>> > >
> > > > >
> > > > > > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> > > > >
> > > > > > > operations#KIP-4-
> > > > >
> > > > > > > >> > >>> > > Commandlineandcentralizedadmin
> istrativeoperations-
> > > > >
> > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > > 2945):(VotedandPlannedforin0.
> > > > >
> > > > > > > 10.1.0)
> > > > >
> > > > > > > >> > >>> > > > > .
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > It would also be useful to document the client
> > > APIs.
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > Thanks,
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > Jun
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> > brahmbhatt
> > > <
> > > > >
> > > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > > > Hi,
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > I am suggesting that we will only allow the
> > > > renewal
> > > > > by
> > > > >
> > > > > > > >> > >>> > > > > > users
> > > > >
> > > > > > > >> > >>> that
> > > > >
> > > > > > > >> > >>> > > > > > authenticated using *non* delegation token
> > > > > mechanism.
> > > > >
> > > > > > > For
> > > > >
> > > > > > > >> > >>> example,
> > > > >
> > > > > > > >> > >>> > If
> > > > >
> > > > > > > >> > >>> > > > > user
> > > > >
> > > > > > > >> > >>> > > > > > Alice authenticated using kerberos and
> > requested
> > > > >
> > > > > > > >> > >>> > > > > > delegation
> > > > >
> > > > > > > >> > >>> tokens,
> > > > >
> > > > > > > >> > >>> > > > only
> > > > >
> > > > > > > >> > >>> > > > > > user Alice authenticated via non delegation
> > > token
> > > > >
> > > > > > > >> > >>> > > > > > mechanism
> > > > >
> > > > > > > >> > can
> > > > >
> > > > > > > >> > >>> > > renew.
> > > > >
> > > > > > > >> > >>> > > > > > Clients that have access to delegation
> tokens
> > > can
> > > > > not
> > > > >
> > > > > > > >> > >>> > > > > > issue
> > > > >
> > > > > > > >> > >>> > renewal
> > > > >
> > > > > > > >> > >>> > > > > > request for renewing their own token and
> this
> > is
> > > > >
> > > > > > > primarily
> > > > >
> > > > > > > >> > >>> > important
> > > > >
> > > > > > > >> > >>> > > to
> > > > >
> > > > > > > >> > >>> > > > > > reduce the time window for which a
> compromised
> > > > token
> > > > >
> > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > be
> > > > >
> > > > > > > >> > >>> valid.
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user can
> > > request
> > > > >
> > > > > > > >> > >>> > > > > > delegation
> > > > >
> > > > > > > >> > >>> > tokens
> > > > >
> > > > > > > >> > >>> > > > but
> > > > >
> > > > > > > >> > >>> > > > > > even here I would recommend to avoid
> creating
> > a
> > > > > chain
> > > > >
> > > > > > > >> > >>> > > > > > where a
> > > > >
> > > > > > > >> > >>> > client
> > > > >
> > > > > > > >> > >>> > > > > > authenticated via delegation token request
> for
> > > > more
> > > > >
> > > > > > > >> > delegation
> > > > >
> > > > > > > >> > >>> > > tokens.
> > > > >
> > > > > > > >> > >>> > > > > > Basically anyone can request delegation
> token,
> > > as
> > > > > long
> > > > >
> > > > > > > as
> > > > >
> > > > > > > >> > they
> > > > >
> > > > > > > >> > >>> > > > > authenticate
> > > > >
> > > > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > Aren't classes listed here
> > > > >
> > > > > > > >> > >>> > > > > > <
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > >
> > > > >
> > > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > > uence/display/KAFKA/KIP-
> > > > >
> > > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > > >
> > > > > > r+Kafka#KIP-48Delegationtokens
> > > > >
> > > > > > > >> > >>> upportforKaf
> > > > >
> > > > > > > >> > >>> > > ka-PublicInterfaces
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > sufficient?
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > Thanks
> > > > >
> > > > > > > >> > >>> > > > > > Parth
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > > > >
> > > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > > >
> > > > > > > >> > >>> wrote:
> > > > >
> > > > > > > >> > >>> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > Parth,
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of comments
> > > > inline
> > > > >
> > > > > > > below.
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> > > > > brahmbhatt <
> > > > >
> > > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By
> > original
> > > > >
> > > > > > > requester
> > > > >
> > > > > > > >> > >>> only? or
> > > > >
> > > > > > > >> > >>> > > > using
> > > > >
> > > > > > > >> > >>> > > > > > > > Kerberos
> > > > >
> > > > > > > >> > >>> > > > > > > > auth only?
> > > > >
> > > > > > > >> > >>> > > > > > > > My recommendation is to do this only
> using
> > > > >
> > > > > > Kerberos
> > > > >
> > > > > > > >> > >>> > > > > > > > auth
> > > > >
> > > > > > > >> > and
> > > > >
> > > > > > > >> > >>> > only
> > > > >
> > > > > > > >> > >>> > > > > threw
> > > > >
> > > > > > > >> > >>> > > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > renewer specified during the acquisition
> > > > > request.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you
> > > saying
> > > > >
> > > > > > that
> > > > >
> > > > > > > >> > >>> > > > > > > any
> > > > >
> > > > > > > >> > >>> client
> > > > >
> > > > > > > >> > >>> > > > > > > authenticated with the delegation token
> can
> > > > renew,
> > > > >
> > > > > > > i.e.
> > > > >
> > > > > > > >> > there
> > > > >
> > > > > > > >> > >>> is
> > > > >
> > > > > > > >> > >>> > no
> > > > >
> > > > > > > >> > >>> > > > > > renewer
> > > > >
> > > > > > > >> > >>> > > > > > > needed?
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > Also, just to be clear, any authenticated
> > > client
> > > > >
> > > > > > > (either
> > > > >
> > > > > > > >> > >>> through
> > > > >
> > > > > > > >> > >>> > > SASL
> > > > >
> > > > > > > >> > >>> > > > > or
> > > > >
> > > > > > > >> > >>> > > > > > > SSL) can request a delegation token for
> the
> > > > >
> > > > > > > >> > >>> > > > > > > authenticated
> > > > >
> > > > > > > >> > >>> user,
> > > > >
> > > > > > > >> > >>> > > > right?
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or
> in
> > > ZK?
> > > > >
> > > > > > > >> > >>> > > > > > > > My recommendation is still to store in
> ZK
> > or
> > > > not
> > > > >
> > > > > > > store
> > > > >
> > > > > > > >> > them
> > > > >
> > > > > > > >> > >>> at
> > > > >
> > > > > > > >> > >>> > > all.
> > > > >
> > > > > > > >> > >>> > > > > The
> > > > >
> > > > > > > >> > >>> > > > > > > > whole controller based distribution is
> too
> > > > much
> > > > >
> > > > > > > >> > >>> > > > > > > > overhead
> > > > >
> > > > > > > >> > >>> with
> > > > >
> > > > > > > >> > >>> > not
> > > > >
> > > > > > > >> > >>> > > > > much
> > > > >
> > > > > > > >> > >>> > > > > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > achieve.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> > > > >
> > > > > > > >> > >>> > > > > > > > Either by expiration time out or through
> > an
> > > > >
> > > > > > explicit
> > > > >
> > > > > > > >> > >>> request to
> > > > >
> > > > > > > >> > >>> > > > > > > invalidate.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > > > >
> > > > > > > >> > >>> > > > > > > > SCRAM
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 5. What is the impersonation proposal
> (it
> > > > wasn't
> > > > >
> > > > > > in
> > > > >
> > > > > > > >> > >>> > > > > > > > the
> > > > >
> > > > > > > >> > KIP
> > > > >
> > > > > > > >> > >>> but
> > > > >
> > > > > > > >> > >>> > > was
> > > > >
> > > > > > > >> > >>> > > > > > > > discussed
> > > > >
> > > > > > > >> > >>> > > > > > > > in this thread)?
> > > > >
> > > > > > > >> > >>> > > > > > > > There is no imperonation proposal. I
> tried
> > > and
> > > > >
> > > > > > > >> > >>> > > > > > > > explained
> > > > >
> > > > > > > >> > how
> > > > >
> > > > > > > >> > >>> > its
> > > > >
> > > > > > > >> > >>> > > a
> > > > >
> > > > > > > >> > >>> > > > > > > > different problem and why its not really
> > > > > necessary
> > > > >
> > > > > > > to
> > > > >
> > > > > > > >> > >>> discuss
> > > > >
> > > > > > > >> > >>> > > that
> > > > >
> > > > > > > >> > >>> > > > as
> > > > >
> > > > > > > >> > >>> > > > > > > part
> > > > >
> > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not support
> > any
> > > > >
> > > > > > > >> > impersonation,
> > > > >
> > > > > > > >> > >>> it
> > > > >
> > > > > > > >> > >>> > > will
> > > > >
> > > > > > > >> > >>> > > > > just
> > > > >
> > > > > > > >> > >>> > > > > > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what
> > > > > actions?
> > > > >
> > > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > Could we document the format of the new
> > > > >
> > > > > > > request/response
> > > > >
> > > > > > > >> > and
> > > > >
> > > > > > > >> > >>> > their
> > > > >
> > > > > > > >> > >>> > > > > > > associated Resource and Operation for ACL?
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > 7. How would the delegation token be
> > > > configured
> > > > > in
> > > > >
> > > > > > > the
> > > > >
> > > > > > > >> > >>> client?
> > > > >
> > > > > > > >> > >>> > > > > > > > Should be through config. I wasn't
> > planning
> > > on
> > > > >
> > > > > > > >> > >>> > > > > > > > supporting
> > > > >
> > > > > > > >> > >>> JAAS
> > > > >
> > > > > > > >> > >>> > > for
> > > > >
> > > > > > > >> > >>> > > > > > > tokens.
> > > > >
> > > > > > > >> > >>> > > > > > > > I don't believe hadoop does this either.
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > Thanks
> > > > >
> > > > > > > >> > >>> > > > > > > > Parth
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun
> Rao <
> > > > >
> > > > > > > >> > jun@confluent.io>
> > > > >
> > > > > > > >> > >>> > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > Harsha,
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > Another question.
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > 9. How would the delegation token be
> > > > > configured
> > > > >
> > > > > > in
> > > > >
> > > > > > > >> > >>> > > > > > > > > the
> > > > >
> > > > > > > >> > >>> > client?
> > > > >
> > > > > > > >> > >>> > > > The
> > > > >
> > > > > > > >> > >>> > > > > > > > standard
> > > > >
> > > > > > > >> > >>> > > > > > > > > way is to do this through JAAS.
> However,
> > > we
> > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > need
> > > > >
> > > > > > > >> > to
> > > > >
> > > > > > > >> > >>> > think
> > > > >
> > > > > > > >> > >>> > > > > > through
> > > > >
> > > > > > > >> > >>> > > > > > > if
> > > > >
> > > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > > environment.
> > > > > For
> > > > >
> > > > > > > >> > example,
> > > > >
> > > > > > > >> > >>> > when a
> > > > >
> > > > > > > >> > >>> > > > new
> > > > >
> > > > > > > >> > >>> > > > > > > task
> > > > >
> > > > > > > >> > >>> > > > > > > > is
> > > > >
> > > > > > > >> > >>> > > > > > > > > added to a Storm worker node, do we
> need
> > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > > dynamically
> > > > >
> > > > > > > >> > >>> add a
> > > > >
> > > > > > > >> > >>> > > new
> > > > >
> > > > > > > >> > >>> > > > > > > section
> > > > >
> > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> > > convenient
> > > > if
> > > > >
> > > > > > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > can
> > > > >
> > > > > > > >> > >>> pass in
> > > > >
> > > > > > > >> > >>> > > the
> > > > >
> > > > > > > >> > >>> > > > > > token
> > > > >
> > > > > > > >> > >>> > > > > > > > > through the config directly w/o going
> > > > through
> > > > >
> > > > > > > JAAS.
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > Are you or Parth still actively
> working
> > on
> > > > > this
> > > > >
> > > > > > > KIP?
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > Thanks,
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > Jun
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun
> > Rao <
> > > > >
> > > > > > > >> > >>> jun@confluent.io>
> > > > >
> > > > > > > >> > >>> > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > 2. It would be good to document the
> > > format
> > > > > of
> > > > >
> > > > > > > the
> > > > >
> > > > > > > >> > data
> > > > >
> > > > > > > >> > >>> > stored
> > > > >
> > > > > > > >> > >>> > > > in
> > > > >
> > > > > > > >> > >>> > > > > > ZK.
> > > > >
> > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion
> on
> > > > > whether
> > > > >
> > > > > > > the
> > > > >
> > > > > > > >> > tokens
> > > > >
> > > > > > > >> > >>> > > should
> > > > >
> > > > > > > >> > >>> > > > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > > config/acl/quota,
> > > > >
> > > > > > or
> > > > >
> > > > > > > >> > through
> > > > >
> > > > > > > >> > >>> the
> > > > >
> > > > > > > >> > >>> > > > > > > controller.
> > > > >
> > > > > > > >> > >>> > > > > > > > > > Currently, the controller is only
> > > designed
> > > > > for
> > > > >
> > > > > > > >> > >>> propagating
> > > > >
> > > > > > > >> > >>> > > > topic
> > > > >
> > > > > > > >> > >>> > > > > > > > > metadata,
> > > > >
> > > > > > > >> > >>> > > > > > > > > > but not other data.
> > > > >
> > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the
> > token
> > > > >
> > > > > > instead
> > > > >
> > > > > > > >> > >>> > > > > > > > > > of
> > > > >
> > > > > > > >> > >>> > > DIGEST-MD5
> > > > >
> > > > > > > >> > >>> > > > > > since
> > > > >
> > > > > > > >> > >>> > > > > > > > it's
> > > > >
> > > > > > > >> > >>> > > > > > > > > > deprecated?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem
> > > broken.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > Thanks,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > Jun
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM,
> Gwen
> > > > >
> > > > > > Shapira <
> > > > >
> > > > > > > >> > >>> > > > > gwen@confluent.io>
> > > > >
> > > > > > > >> > >>> > > > > > > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> From what I can see, remaining
> > > questions
> > > > > are:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By
> > > > > original
> > > > >
> > > > > > > >> > requester
> > > > >
> > > > > > > >> > >>> > only?
> > > > >
> > > > > > > >> > >>> > > > or
> > > > >
> > > > > > > >> > >>> > > > > > > using
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker
> > or
> > > in
> > > > > ZK?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated /
> > > expired?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is
> > used?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation
> proposal
> > > (it
> > > > >
> > > > > > > wasn't
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> in
> > > > >
> > > > > > > >> > the
> > > > >
> > > > > > > >> > >>> > KIP
> > > > >
> > > > > > > >> > >>> > > > but
> > > > >
> > > > > > > >> > >>> > > > > > was
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for
> > > what
> > > > >
> > > > > > > actions?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> Gwen
> > > > >
> > > > > > > >> > >>> > > > > > > > > >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM,
> > Harsha
> > > <
> > > > >
> > > > > > > >> > >>> kafka@harsha.io>
> > > > >
> > > > > > > >> > >>> > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >
> > > Unfortunately
> > > > I
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > > >
> > > > > > > >> > >>> attend
> > > > >
> > > > > > > >> > >>> > > the
> > > > >
> > > > > > > >> > >>> > > > > KIP
> > > > >
> > > > > > > >> > >>> > > > > > > > > meeting
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > when
> > > > delegation
> > > > >
> > > > > > > tokens
> > > > >
> > > > > > > >> > >>> > discussed.
> > > > >
> > > > > > > >> > >>> > > > > > > > Appreciate
> > > > >
> > > > > > > >> > >>> > > > > > > > > if
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > you can
> > > update
> > > > > the
> > > > >
> > > > > > > >> > thread if
> > > > >
> > > > > > > >> > >>> > you
> > > > >
> > > > > > > >> > >>> > > > have
> > > > >
> > > > > > > >> > >>> > > > > > any
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > further
> > > > > questions.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32
> AM,
> > > > Liquan
> > > > >
> > > > > > Pei
> > > > >
> > > > > > > >> > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> It seems that the links to
> images
> > in
> > > > the
> > > > >
> > > > > > KIP
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> are
> > > > >
> > > > > > > >> > >>> > broken.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM,
> > > parth
> > > > >
> > > > > > > >> > brahmbhatt <
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com>
> > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > > getDelegationTokenAs
> > > > >
> > > > > > mean?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > In the current proposal we
> only
> > > > allow
> > > > > a
> > > > >
> > > > > > > user
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > to
> > > > >
> > > > > > > >> > >>> get
> > > > >
> > > > > > > >> > >>> > > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > token
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> for
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> > authenticated
> > > > as
> > > > >
> > > > > > > using
> > > > >
> > > > > > > >> > >>> another
> > > > >
> > > > > > > >> > >>> > > > > > mechanism,
> > > > >
> > > > > > > >> > >>> > > > > > > > i.e.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> A user
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate using a
> keytab
> > > for
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > > >
> > > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> will get
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that
> user
> > > > only.
> > > > > In
> > > > >
> > > > > > > >> > future I
> > > > >
> > > > > > > >> > >>> > think
> > > > >
> > > > > > > >> > >>> > > > we
> > > > >
> > > > > > > >> > >>> > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > have
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> to
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > extend support such that we
> > allow
> > > > some
> > > > >
> > > > > > set
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > of
> > > > >
> > > > > > > >> > >>> users (
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > > > >
> > > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > > >
> > > > > > > >> > >>> > > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > acquire
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of
> > > other
> > > > >
> > > > > > users
> > > > >
> > > > > > > >> > whose
> > > > >
> > > > > > > >> > >>> > > identity
> > > > >
> > > > > > > >> > >>> > > > > > they
> > > > >
> > > > > > > >> > >>> > > > > > > > have
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka
> > > > brokers
> > > > >
> > > > > > > will
> > > > >
> > > > > > > >> > have
> > > > >
> > > > > > > >> > >>> ACLs
> > > > >
> > > > > > > >> > >>> > > to
> > > > >
> > > > > > > >> > >>> > > > > > > control
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> which
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to
> impersonate
> > > > other
> > > > >
> > > > > > > users
> > > > >
> > > > > > > >> > and
> > > > >
> > > > > > > >> > >>> get
> > > > >
> > > > > > > >> > >>> > > > tokens
> > > > >
> > > > > > > >> > >>> > > > > > on
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> behalf of
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation
> is a
> > > > whole
> > > > >
> > > > > > > >> > different
> > > > >
> > > > > > > >> > >>> > > problem
> > > > >
> > > > > > > >> > >>> > > > in
> > > > >
> > > > > > > >> > >>> > > > > > my
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> opinion and
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in
> > > separate
> > > > >
> > > > > > KIP.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate
> of
> > > > > getting
> > > > >
> > > > > > > and
> > > > >
> > > > > > > >> > >>> renewing
> > > > >
> > > > > > > >> > >>> > > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > Typically this should be very
> > very
> > > > > low,
> > > > >
> > > > > > 1
> > > > >
> > > > > > > >> > request
> > > > >
> > > > > > > >> > >>> per
> > > > >
> > > > > > > >> > >>> > > > > minute
> > > > >
> > > > > > > >> > >>> > > > > > > is a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > relatively high estimate.
> > However
> > > it
> > > > >
> > > > > > > depends
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > on
> > > > >
> > > > > > > >> > >>> the
> > > > >
> > > > > > > >> > >>> > > token
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> expiration. I am
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > less worried about the extra
> > load
> > > it
> > > > >
> > > > > > puts
> > > > >
> > > > > > > on
> > > > >
> > > > > > > >> > >>> > controller
> > > > >
> > > > > > > >> > >>> > > > vs
> > > > >
> > > > > > > >> > >>> > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > added
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > complexity and the value it
> > > offers.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > Thanks
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > Parth
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30
> AM,
> > > > > Ismael
> > > > >
> > > > > > > Juma
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > <
> > > > >
> > > > > > > >> > >>> > > > > > > ismael@juma.me.uk>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would
> > probably
> > > > >
> > > > > > > require a
> > > > >
> > > > > > > >> > >>> separate
> > > > >
> > > > > > > >> > >>> > > KIP
> > > > >
> > > > > > > >> > >>> > > > > as
> > > > >
> > > > > > > >> > >>> > > > > > it
> > > > >
> > > > > > > >> > >>> > > > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > introduce user visible
> > changes.
> > > We
> > > > >
> > > > > > could
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > also
> > > > >
> > > > > > > >> > >>> > update
> > > > >
> > > > > > > >> > >>> > > > > KIP-48
> > > > >
> > > > > > > >> > >>> > > > > > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> have this
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > information, but it seems
> > > cleaner
> > > > to
> > > > >
> > > > > > do
> > > > >
> > > > > > > it
> > > > >
> > > > > > > >> > >>> > > separately.
> > > > >
> > > > > > > >> > >>> > > > We
> > > > >
> > > > > > > >> > >>> > > > > > can
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> discuss
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > that
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > Ismael
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19
> > PM,
> > > > >
> > > > > > Rajini
> > > > >
> > > > > > > >> > Sivaram
> > > > >
> > > > > > > >> > >>> <
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > >
> rajinisivaram@googlemail.com>
> > > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > Ismael,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > > > >
> > > > > > > >> > jira/browse/KAFKA-3751)
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
> > > > >
> > > > > > mechanism.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > Would
> > > > >
> > > > > > > >> > >>> that
> > > > >
> > > > > > > >> > >>> > > need
> > > > >
> > > > > > > >> > >>> > > > > > > another
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> KIP? If
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this
> > > mechanism,
> > > > > can
> > > > >
> > > > > > > this
> > > > >
> > > > > > > >> > just
> > > > >
> > > > > > > >> > >>> be
> > > > >
> > > > > > > >> > >>> > a
> > > > >
> > > > > > > >> > >>> > > > JIRA
> > > > >
> > > > > > > >> > >>> > > > > > > that
> > > > >
> > > > > > > >> > >>> > > > > > > > > gets
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > reviewed
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > Thank you,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > Rajini
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at
> 2:46
> > > PM,
> > > > >
> > > > > > > Ismael
> > > > >
> > > > > > > >> > Juma <
> > > > >
> > > > > > > >> > >>> > > > > > > > > ismael@juma.me.uk>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM
> seems
> > > > like
> > > > > a
> > > > >
> > > > > > > good
> > > > >
> > > > > > > >> > >>> > candidate.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently
> > > > mentioned
> > > > >
> > > > > > > this
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > as
> > > > >
> > > > > > > >> > a
> > > > >
> > > > > > > >> > >>> SASL
> > > > >
> > > > > > > >> > >>> > > > > > mechanism
> > > > >
> > > > > > > >> > >>> > > > > > > > > that
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> might
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I
> > have
> > > > been
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > meaning
> > > > >
> > > > > > > >> > to
> > > > >
> > > > > > > >> > >>> > check
> > > > >
> > > > > > > >> > >>> > > > it
> > > > >
> > > > > > > >> > >>> > > > > in
> > > > >
> > > > > > > >> > >>> > > > > > > > more
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> detail.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > Good
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > to know that you are
> > willing
> > > > to
> > > > >
> > > > > > > >> > contribute
> > > > >
> > > > > > > >> > >>> an
> > > > >
> > > > > > > >> > >>> > > > > > > > implementation.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> Maybe
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > should file a separate
> > JIRA
> > > > for
> > > > >
> > > > > > > this?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Ismael
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at
> > 2:12
> > > > PM,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > Rajini
> > > > >
> > > > > > > >> > >>> > Sivaram <
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > rajinisivaram@googlemail.com>
> > > > >
> > > > > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted
> Challenge
> > > > > Response
> > > > >
> > > > > > > >> > >>> > Authentication
> > > > >
> > > > > > > >> > >>> > > > > > > > Mechanism)
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> is a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > better
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than
> > Digest-MD5.
> > > > > Java
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> > > > >
> > > > > > > >> > >>> come
> > > > >
> > > > > > > >> > >>> > > > with a
> > > > >
> > > > > > > >> > >>> > > > > > > > > built-in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> SCRAM
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or
> > SaslClient,
> > > > but
> > > > > I
> > > > >
> > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > be
> > > > >
> > > > > > > >> > >>> happy
> > > > >
> > > > > > > >> > >>> > > to
> > > > >
> > > > > > > >> > >>> > > > > add
> > > > >
> > > > > > > >> > >>> > > > > > > > > support
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > since
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful
> > > > mechanism
> > > > >
> > > > > > to
> > > > >
> > > > > > > >> > support
> > > > >
> > > > > > > >> > >>> > > anyway.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > https://tools.ietf.org/html/
> > > > >
> > > > > > > rfc7677
> > > > >
> > > > > > > >> > >>> > describes
> > > > >
> > > > > > > >> > >>> > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > protocol
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> for
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016
> at
> > > 2:37
> > > > > AM,
> > > > >
> > > > > > > Jun
> > > > >
> > > > > > > >> > Rao <
> > > > >
> > > > > > > >> > >>> > > > > > > > jun@confluent.io
> > > > >
> > > > > > > >> > >>> > > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the
> > > > explanation.
> > > > > A
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > couple
> > > > >
> > > > > > > >> > of
> > > > >
> > > > > > > >> > >>> > more
> > > > >
> > > > > > > >> > >>> > > > > > > questions.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> > > > >
> > > > > > > >> > >>> mean?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the
> > typical
> > > > rate
> > > > >
> > > > > > of
> > > > >
> > > > > > > >> > getting
> > > > >
> > > > > > > >> > >>> and
> > > > >
> > > > > > > >> > >>> > > > > > renewing
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > tokens?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an
> > impact
> > > on
> > > > >
> > > > > > > whether
> > > > >
> > > > > > > >> > they
> > > > >
> > > > > > > >> > >>> > > should
> > > > >
> > > > > > > >> > >>> > > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > directed
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> to the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016
> > at
> > > > 1:19
> > > > >
> > > > > > PM,
> > > > >
> > > > > > > >> > parth
> > > > >
> > > > > > > >> > >>> > > > > brahmbhatt <
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > brahmbhatt.parth@gmail.com>
> > > > >
> > > > > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for
> > reviewing.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a
> > > Cluster
> > > > >
> > > > > > > action
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > >
> > > > > > > >> > add
> > > > >
> > > > > > > >> > >>> > acls
> > > > >
> > > > > > > >> > >>> > > > on
> > > > >
> > > > > > > >> > >>> > > > > > who
> > > > >
> > > > > > > >> > >>> > > > > > > > can
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> request
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't
> see
> > > the
> > > > > use
> > > > >
> > > > > > > case
> > > > >
> > > > > > > >> > for
> > > > >
> > > > > > > >> > >>> that
> > > > >
> > > > > > > >> > >>> > > yet
> > > > >
> > > > > > > >> > >>> > > > > but
> > > > >
> > > > > > > >> > >>> > > > > > > > down
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> the line
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > when
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > > > >
> > > > > > > >> > getDelegationTokenAs
> > > > >
> > > > > > > >> > >>> it
> > > > >
> > > > > > > >> > >>> > > will
> > > > >
> > > > > > > >> > >>> > > > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> necessary.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend
> > > > tokens
> > > > > to
> > > > >
> > > > > > > be
> > > > >
> > > > > > > >> > only
> > > > >
> > > > > > > >> > >>> > > > > > > used/distributed
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> over
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > secure
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on
> what
> > > > design
> > > > >
> > > > > > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > end
> > > > >
> > > > > > > >> > up
> > > > >
> > > > > > > >> > >>> > > choosing
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> Invalidation will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of
> > > every
> > > > >
> > > > > > broker
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > or
> > > > >
> > > > > > > >> > >>> > > controller.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure
> if I
> > > > >
> > > > > > > documented
> > > > >
> > > > > > > >> > >>> somewhere
> > > > >
> > > > > > > >> > >>> > > > that
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> invalidation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > directly
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > go through
> zookeeper
> > > but
> > > > >
> > > > > > that
> > > > >
> > > > > > > is
> > > > >
> > > > > > > >> > not
> > > > >
> > > > > > > >> > >>> the
> > > > >
> > > > > > > >> > >>> > > > > intent.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> Invalidation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > either
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based
> or
> > > due
> > > > to
> > > > >
> > > > > > > >> > >>> expiration. No
> > > > >
> > > > > > > >> > >>> > > > > direct
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> zookeeper
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > interaction
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > from
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also
> > stores
> > > > the
> > > > >
> > > > > > > >> > >>> DelegationToken
> > > > >
> > > > > > > >> > >>> > > > > without
> > > > >
> > > > > > > >> > >>> > > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> hmac in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." :
> Sorry
> > > > about
> > > > >
> > > > > > the
> > > > >
> > > > > > > >> > >>> confusion.
> > > > >
> > > > > > > >> > >>> > > The
> > > > >
> > > > > > > >> > >>> > > > > sole
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> purpose of
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
> > > > >
> > > > > > distribution
> > > > >
> > > > > > > >> > channel
> > > > >
> > > > > > > >> > >>> > for
> > > > >
> > > > > > > >> > >>> > > > > tokens
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> between all
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > brokers
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures
> > > only
> > > > >
> > > > > > tokens
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > that
> > > > >
> > > > > > > >> > >>> were
> > > > >
> > > > > > > >> > >>> > > > > > generated
> > > > >
> > > > > > > >> > >>> > > > > > > by
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> making a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > request
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > to a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be
> > > accepted
> > > > >
> > > > > > (more
> > > > >
> > > > > > > on
> > > > >
> > > > > > > >> > this
> > > > >
> > > > > > > >> > >>> in
> > > > >
> > > > > > > >> > >>> > > > second
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> paragraph). The
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > token
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few
> > > elements
> > > > >
> > > > > > > (owner,
> > > > >
> > > > > > > >> > >>> renewer,
> > > > >
> > > > > > > >> > >>> > > > uuid
> > > > >
> > > > > > > >> > >>> > > > > ,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> expiration,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > hmac)
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > ,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > one
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > of
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the
> finally
> > > > >
> > > > > > generated
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> > > > >
> > > > > > > >> > >>> but
> > > > >
> > > > > > > >> > >>> > > hmac
> > > > >
> > > > > > > >> > >>> > > > it
> > > > >
> > > > > > > >> > >>> > > > > > > self
> > > > >
> > > > > > > >> > >>> > > > > > > > is
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > derivable
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > if
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > you
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other
> > > > > elements
> > > > >
> > > > > > of
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the
> > > > >
> > > > > > > >> > >>> token
> > > > >
> > > > > > > >> > >>> > +
> > > > >
> > > > > > > >> > >>> > > > > secret
> > > > >
> > > > > > > >> > >>> > > > > > > key
> > > > >
> > > > > > > >> > >>> > > > > > > > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > generate
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > hmac.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper
> does
> > > not
> > > > >
> > > > > > > provide
> > > > >
> > > > > > > >> > SSL
> > > > >
> > > > > > > >> > >>> > > support
> > > > >
> > > > > > > >> > >>> > > > we
> > > > >
> > > > > > > >> > >>> > > > > > do
> > > > >
> > > > > > > >> > >>> > > > > > > > not
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> want the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > entire
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> > > > > transferred
> > > > >
> > > > > > > to
> > > > >
> > > > > > > >> > >>> zookeeper
> > > > >
> > > > > > > >> > >>> > > as
> > > > >
> > > > > > > >> > >>> > > > > that
> > > > >
> > > > > > > >> > >>> > > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> be an
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > insecure
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > wire
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead
> we
> > > > only
> > > > >
> > > > > > > store
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > all
> > > > >
> > > > > > > >> > >>> the
> > > > >
> > > > > > > >> > >>> > > other
> > > > >
> > > > > > > >> > >>> > > > > > > > elements
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> of a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers
> can
> > > read
> > > > >
> > > > > > these
> > > > >
> > > > > > > >> > >>> elements
> > > > >
> > > > > > > >> > >>> > and
> > > > >
> > > > > > > >> > >>> > > > > > because
> > > > >
> > > > > > > >> > >>> > > > > > > > > they
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> also
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > have
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > access
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they
> > > will
> > > > be
> > > > >
> > > > > > > able
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > >
> > > > > > > >> > >>> > generate
> > > > >
> > > > > > > >> > >>> > > > > hmac
> > > > >
> > > > > > > >> > >>> > > > > > on
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> their end.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the
> > alternative
> > > > >
> > > > > > > proposed
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > is
> > > > >
> > > > > > > >> > to
> > > > >
> > > > > > > >> > >>> > avoid
> > > > >
> > > > > > > >> > >>> > > > > > > zookeeper
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > altogether. A
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Client
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker
> > with
> > > > >
> > > > > > required
> > > > >
> > > > > > > >> > >>> > information
> > > > >
> > > > > > > >> > >>> > > > > > (owner,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> renwer,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > expiration)
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed
> > hmac,
> > > > >
> > > > > > uuid).
> > > > >
> > > > > > > >> > Broker
> > > > >
> > > > > > > >> > >>> > won't
> > > > >
> > > > > > > >> > >>> > > > > store
> > > > >
> > > > > > > >> > >>> > > > > > > this
> > > > >
> > > > > > > >> > >>> > > > > > > > > in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > zookeeper.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > From
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a
> client
> > > can
> > > > >
> > > > > > > contact
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > > >
> > > > > > > >> > >>> > broker
> > > > >
> > > > > > > >> > >>> > > > with
> > > > >
> > > > > > > >> > >>> > > > > > all
> > > > >
> > > > > > > >> > >>> > > > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > token
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner,
> rewner,
> > > > >
> > > > > > > expiration,
> > > > >
> > > > > > > >> > hmac,
> > > > >
> > > > > > > >> > >>> > > uuid)
> > > > >
> > > > > > > >> > >>> > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > borker
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > regenerate
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long
> as
> > it
> > > > >
> > > > > > matches
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > with
> > > > >
> > > > > > > >> > >>> hmac
> > > > >
> > > > > > > >> > >>> > > > > > presented
> > > > >
> > > > > > > >> > >>> > > > > > > by
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> client ,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > broker
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request
> to
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> > > > >
> > > > > > > >> > >>> Only
> > > > >
> > > > > > > >> > >>> > > > > problem
> > > > >
> > > > > > > >> > >>> > > > > > > with
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> this
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > approach
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > is
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > if
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
> > > > >
> > > > > > compromised
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > > >
> > > > > > > >> > >>> client
> > > > >
> > > > > > > >> > >>> > > can
> > > > >
> > > > > > > >> > >>> > > > > now
> > > > >
> > > > > > > >> > >>> > > > > > > > > generate
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > random
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > tokens
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be
> > > able
> > > > to
> > > > >
> > > > > > > >> > >>> authenticate
> > > > >
> > > > > > > >> > >>> > as
> > > > >
> > > > > > > >> > >>> > > > any
> > > > >
> > > > > > > >> > >>> > > > > > user
> > > > >
> > > > > > > >> > >>> > > > > > > > > they
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> like.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > with
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we
> > guarantee
> > > > > that
> > > > >
> > > > > > > only
> > > > >
> > > > > > > >> > >>> tokens
> > > > >
> > > > > > > >> > >>> > > > > acquired
> > > > >
> > > > > > > >> > >>> > > > > > > via
> > > > >
> > > > > > > >> > >>> > > > > > > > a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> broker
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > (using
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > some
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other
> > than
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> > > > >
> > > > > > > >> > >>> token)
> > > > >
> > > > > > > >> > >>> > > will
> > > > >
> > > > > > > >> > >>> > > > > be
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> accepted. We
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > need
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > to
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which
> > proposal
> > > > > makes
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > more
> > > > >
> > > > > > > >> > >>> sense
> > > > >
> > > > > > > >> > >>> > and
> > > > >
> > > > > > > >> > >>> > > > we
> > > > >
> > > > > > > >> > >>> > > > > > can
> > > > >
> > > > > > > >> > >>> > > > > > > go
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> over it
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > in
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you
> > forward
> > > > the
> > > > >
> > > > > > > invite
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > > >
> > > > > > > >> > >>> me?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23,
> 2016
> > > at
> > > > >
> > > > > > 10:35
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> > > > >
> > > > > > > >> > Jun
> > > > >
> > > > > > > >> > >>> > Rao <
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> jun@confluent.io>
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > wrote:
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the
> > KIP.
> > > A
> > > > > few
> > > > >
> > > > > > > >> > comments.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This
> > > potentially
> > > > > can
> > > > >
> > > > > > be
> > > > >
> > > > > > > >> > useful
> > > > >
> > > > > > > >> > >>> for
> > > > >
> > > > > > > >> > >>> > > > Kafka
> > > > >
> > > > > > > >> > >>> > > > > > > > Connect
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> and
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > rest
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker
> > agent
> > > > > will
> > > > >
> > > > > > > need
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> > > > >
> > > > > > > >> > >>> run a
> > > > >
> > > > > > > >> > >>> > > > task
> > > > >
> > > > > > > >> > >>> > > > > on
> > > > >
> > > > > > > >> > >>> > > > > > > > > behalf
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> of a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > client.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > We
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to
> > > change
> > > > > how
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> > > > >
> > > > > > > >> > >>> > services
> > > > >
> > > > > > > >> > >>> > > > use
> > > > >
> > > > > > > >> > >>> > > > > > > Kafka
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> clients
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > (producer/consumer).
> > > > >
> > > > > > Instead
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> > > > >
> > > > > > > >> > >>> > shared
> > > > >
> > > > > > > >> > >>> > > > > client
> > > > >
> > > > > > > >> > >>> > > > > > > per
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> worker,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > will
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > need
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > a
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user
> > task
> > > > > since
> > > > >
> > > > > > > the
> > > > >
> > > > > > > >> > >>> > > > authentication
> > > > >
> > > > > > > >> > >>> > > > > > > > happens
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> at the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > connection
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka
> > > > > Connect,
> > > > >
> > > > > > > the
> > > > >
> > > > > > > >> > >>> renewer
> > > > >
> > > > > > > >> > >>> > > will
> > > > >
> > > > > > > >> > >>> > > > be
> > > > >
> > > > > > > >> > >>> > > > > > the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> workers.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > So,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > we
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow
> > > multiple
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> > > > >
> > > > > > > >> > For
> > > > >
> > > > > > > >> > >>> > > Kafka
> > > > >
> > > > > > > >> > >>> > > > > rest
> > > > >
> > > > > > > >> > >>> > > > > > > > > proxy,
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> the
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > renewer
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > can
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be
> > the
> > > > >
> > > > > > creator
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> > > > >
> > > > > > > >> > the
> > > > >
> > > > > > > >> > >>> > > token.
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need
> > new
> > > > acl
> > > > > on
> > > > >
> > > > > > > who
> > > > >
> > > > > > > >> > can
> > > > >
> > > > > > > >> > >>> > > request
> > > > >
> > > > > > > >> > >>> > > > > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> tokens?
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we
> > recommend
> > > > >
> > > > > > people
> > > > >
> > > > > > > to
> > > > >
> > > > > > > >> > send
> > > > >
> > > > > > > >> > >>> > > > > delegation
> > > > >
> > > > > > > >> > >>> > > > > > > > tokens
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> in an
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > encrypted
> > > > >
> > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > > > >
> > > > > > > >> > >>>
> > > >
> > >
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
That's a good idea. Authenticating every request with delegation token will
be useful for
impersonation use-cases. But as of now, we are thinking delegation token as
just another way
to authenticate the users. We haven't think through all the use cases
related to
impersonation or using delegation token for impersonation. We want to
handle impersonation
(KAFKA-3712) as part of separate KIP.
Will that be Ok?
On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Thinking out loud here:
>
> It looks like authentication with a delegation token is going to be
> super-cheap, right? We just compare the token to a value in the broker
> cache?
>
> If I understood the KIP correctly, right now it suggests that
> authentication happens when establishing the client-broker connection (as
> normal for Kafka. But perhaps we want to consider authenticating every
> request with delegation token (if exists)?
>
> So a centralized app can create few producers, do the metadata request and
> broker discovery with its own user auth, but then use delegation tokens to
> allow performing produce/fetch requests as different users? Instead of
> having to re-connect for each impersonated user?
>
> This may over-complicate things quite a bit (basically adding extra
> information in every request), but maybe it will be useful for
> impersonation use-cases (which seem to drive much of the interest in this
> KIP)?
> Kafka Connect, NiFi and friends can probably use this to share clients
> between multiple jobs, tasks, etc.
>
> What do you think?
>
> Gwen
>
> On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <ma...@gmail.com>
> wrote:
>
> > Ashish,
> >
> > Thank you for reviewing the KIP. Please see the replies inline.
> >
> >
> > > 1. How to disable delegation token authentication?
> > >
> > > This can be achieved in various ways, however I think reusing
> delegation
> > > token secret config for this makes sense here. Avoids creating yet
> > another
> > > config and forces delegation token users to consciously set the secret.
> > If
> > > the secret is not set or set to empty string, brokers should turn off
> > > delegation token support. This will however require a new error code to
> > > indicate delegation token support is turned off on broker.
> > >
> >
> > Thanks for the suggestion. Option to turnoff delegation token
> > authentication will be useful.
> > I'll update the KIP.
> >
> >
> > >
> > > 2. ACLs on delegation token?
> > >
> > > Do we need to have ACLs defined for tokens? I do not think it buys us
> > > anything, as delegation token can be treated as impersonation of the
> > owner.
> > > Any thing the owner has permission to do, delegation tokens should be
> > > allowed to do as well. If so, we probably won't need to return
> > > authorization exception error code while creating delegation token. It
> > > however would make sense to check renew and expire requests are coming
> > from
> > > owner or renewers of the token, but that does not require explicit
> acls.
> > >
> >
> >
> > Yes, We agreed to not have new acl on who can request delegation token.
> > I'll update the KIP.
> >
> >
> > >
> > > 3. How to restrict max life time of a token?
> > >
> > > Admins might want to restrict max life time of tokens created on a
> > cluster,
> > > and this can very from cluster to cluster based on use-cases. This
> might
> > > warrant a separate broker config.
> > >
> > >
> > Currently we have "delegation.token.max.lifetime.sec" server config
> > property
> > May be we can take min(User supplied MaxTime, Server MaxTime) as max life
> > time.
> > I am open to add new config property.
> >
> > Few more comments based on recent KIP update.
> > >
> > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> > > {{ExpireTokenRequest}} with with expiryDate set to anything before
> > current
> > > date?
> > >
> >
> > makes sense. we don't need special request to cancel the token. We can
> use
> > ExpireTokenRequest.
> > I'll update the KIP.
> >
> >
> > > 2. Can we change time field names to indicate their unit is
> milliseconds,
> > > like, IssueDateMs, ExpiryDateMs, etc.?
> > >
> > >
> > Done.
> >
> >
> > > 3. Can we allow users to renew a token for a specified amount of time?
> In
> > > current version of KIP, renew request does not take time as a param,
> not
> > > sure what is expiry time set to after renewal.
> > >
> > >
> > Yes, we need to specify renew period. I'll update the KIP.
> >
> >
> > Thanks,
> > Mankumar
> >
> >
> >
> > >
> > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <ma...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > I would like to reinitiate the discussion on Delegation token support
> > for
> > > >
> > > > Kafka.
> > > >
> > > >
> > > >
> > > > Brief summary of the past discussion:
> > > >
> > > >
> > > >
> > > > 1) Broker stores delegation tokens in zookeeper. All brokers will
> > have a
> > > >
> > > > cache backed by
> > > >
> > > > zookeeper so they will all get notified whenever a new token is
> > > >
> > > > generated and they will
> > > >
> > > > update their local cache whenever token state changes.
> > > >
> > > > 2) The current proposal does not support rotation of secret
> > > >
> > > > 3) Only allow the renewal by users that authenticated using *non*
> > > >
> > > > delegation token mechanism
> > > >
> > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients
> can
> > > >
> > > > authenticate using
> > > >
> > > > SCRAM-SHA-256, providing the delegation token HMAC as password.
> > > >
> > > >
> > > >
> > > > Updated the KIP with the following:
> > > >
> > > > 1. Protocol and Config changes
> > > >
> > > > 2. format of the data stored in ZK.
> > > >
> > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > > >
> > > >
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > >
> > > > 48+Delegation+token+support+for+Kafka
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Jun, Ashish, Gwen,
> > > >
> > > >
> > > >
> > > > Pl review the updated KIP.
> > > >
> > > >
> > > >
> > > > Thanks
> > > >
> > > > Manikumar
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <as...@cloudera.com>
> > > wrote:
> > > >
> > > >
> > > >
> > > > > Harsha/ Gwen,
> > > >
> > > > >
> > > >
> > > > > How do we proceed here? I am willing to help out with here.
> > > >
> > > > >
> > > >
> > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io>
> > > > wrote:
> > > >
> > > > >
> > > >
> > > > > > Is it updated? are all concerns addressed? do you want to start a
> > > vote?
> > > >
> > > > > >
> > > >
> > > > > > Sorry for being pushy, I do appreciate that we are all volunteers
> > and
> > > >
> > > > > > finding time is difficult. This feature is important for anything
> > > that
> > > >
> > > > > > integrates with Kafka (stream processors, Flume, NiFi, etc) and I
> > > >
> > > > > > don't want to see this getting stuck because we lack coordination
> > > >
> > > > > > within the community.
> > > >
> > > > > >
> > > >
> > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > > kafka@harsha.io>
> > > >
> > > > > > wrote:
> > > >
> > > > > > > The only pending update for the KIP is to write up the protocol
> > > > changes
> > > >
> > > > > > like
> > > >
> > > > > > > we've it KIP-4. I'll update the wiki.
> > > >
> > > > > > >
> > > >
> > > > > > >
> > > >
> > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> > asingh@cloudera.com>
> > > >
> > > > > > wrote:
> > > >
> > > > > > >>
> > > >
> > > > > > >> I think we decided to not support secret rotation, I guess
> this
> > > can
> > > > be
> > > >
> > > > > > >> stated clearly on the KIP. Also, more details on how clients
> > will
> > > >
> > > > > > perform
> > > >
> > > > > > >> token distribution and how CLI will look like will be helpful.
> > > >
> > > > > > >>
> > > >
> > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> > gwen@confluent.io>
> > > >
> > > > > > wrote:
> > > >
> > > > > > >>
> > > >
> > > > > > >> > Hi Guys,
> > > >
> > > > > > >> >
> > > >
> > > > > > >> > This discussion was dead for a while. Are there still
> > > contentious
> > > >
> > > > > > >> > points? If not, why are there no votes?
> > > >
> > > > > > >> >
> > > >
> > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > >
> > > > > > >> > > Ashish,
> > > >
> > > > > > >> > >
> > > >
> > > > > > >> > > Yes, I will send out a KIP invite for next week to discuss
> > > > KIP-48
> > > >
> > > > > > and
> > > >
> > > > > > >> > other
> > > >
> > > > > > >> > > remaining KIPs.
> > > >
> > > > > > >> > >
> > > >
> > > > > > >> > > Thanks,
> > > >
> > > > > > >> > >
> > > >
> > > > > > >> > > Jun
> > > >
> > > > > > >> > >
> > > >
> > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > > >
> > > > > asingh@cloudera.com>
> > > >
> > > > > > >> > wrote:
> > > >
> > > > > > >> > >
> > > >
> > > > > > >> > >> Thanks Harsha!
> > > >
> > > > > > >> > >>
> > > >
> > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda.
> Also,
> > we
> > > > did
> > > >
> > > > > > not
> > > >
> > > > > > >> > >> actually make a call on when we should have next KIP
> call.
> > As
> > > >
> > > > > there
> > > >
> > > > > > >> > >> are
> > > >
> > > > > > >> > a
> > > >
> > > > > > >> > >> few outstanding KIPs that could not be discussed this
> week,
> > > can
> > > >
> > > > > we
> > > >
> > > > > > >> > >> have
> > > >
> > > > > > >> > a
> > > >
> > > > > > >> > >> KIP hangout call next week?
> > > >
> > > > > > >> > >>
> > > >
> > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> > > >
> > > > > > >> > >> <ka...@harsha.io>
> > > >
> > > > > > >> > >> wrote:
> > > >
> > > > > > >> > >>
> > > >
> > > > > > >> > >>> Ashish,
> > > >
> > > > > > >> > >>> Yes we are working on it. Lets discuss in the
> next
> > > KIP
> > > >
> > > > > > >> > >>> meeting.
> > > >
> > > > > > >> > >>> I'll join.
> > > >
> > > > > > >> > >>> -Harsha
> > > >
> > > > > > >> > >>>
> > > >
> > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > > >
> > > > > > asingh@cloudera.com>
> > > >
> > > > > > >> > >>> wrote:
> > > >
> > > > > > >> > >>>
> > > >
> > > > > > >> > >>> > Hello Harsha,
> > > >
> > > > > > >> > >>> >
> > > >
> > > > > > >> > >>> > Are you still working on this? Wondering if we can
> > discuss
> > > >
> > > > > this
> > > >
> > > > > > in
> > > >
> > > > > > >> > next
> > > >
> > > > > > >> > >>> KIP
> > > >
> > > > > > >> > >>> > meeting, if you can join.
> > > >
> > > > > > >> > >>> >
> > > >
> > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> > > >
> > > > > > >> > kafka@harsha.io>
> > > >
> > > > > > >> > >>> > wrote:
> > > >
> > > > > > >> > >>> >
> > > >
> > > > > > >> > >>> > > Hi Grant,
> > > >
> > > > > > >> > >>> > > We are working on it. Will add the details
> > to
> > > > KIP
> > > >
> > > > > > >> > >>> > > about
> > > >
> > > > > > >> > the
> > > >
> > > > > > >> > >>> > > request protocol.
> > > >
> > > > > > >> > >>> > >
> > > >
> > > > > > >> > >>> > > Thanks,
> > > >
> > > > > > >> > >>> > > Harsha
> > > >
> > > > > > >> > >>> > >
> > > >
> > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > > >
> > > > > > >> > >>> > > <gh...@cloudera.com>
> > > >
> > > > > > >> > >>> wrote:
> > > >
> > > > > > >> > >>> > >
> > > >
> > > > > > >> > >>> > > > Hi Parth,
> > > >
> > > > > > >> > >>> > > >
> > > >
> > > > > > >> > >>> > > > Are you still working on this? If you need any
> help
> > > > please
> > > >
> > > > > > >> > >>> > > > don't
> > > >
> > > > > > >> > >>> > hesitate
> > > >
> > > > > > >> > >>> > > > to ask.
> > > >
> > > > > > >> > >>> > > >
> > > >
> > > > > > >> > >>> > > > Thanks,
> > > >
> > > > > > >> > >>> > > > Grant
> > > >
> > > > > > >> > >>> > > >
> > > >
> > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > > >
> > > > > jun@confluent.io>
> > > >
> > > > > > >> > wrote:
> > > >
> > > > > > >> > >>> > > >
> > > >
> > > > > > >> > >>> > > > > Parth,
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > Thanks for the reply.
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > It makes sense to only allow the renewal by
> users
> > > that
> > > >
> > > > > > >> > >>> authenticated
> > > >
> > > > > > >> > >>> > > > using
> > > >
> > > > > > >> > >>> > > > > *non* delegation token mechanism. Then, should
> we
> > > make
> > > >
> > > > > the
> > > >
> > > > > > >> > >>> renewal a
> > > >
> > > > > > >> > >>> > > > list?
> > > >
> > > > > > >> > >>> > > > > For example, in the case of rest proxy, it will
> be
> > > >
> > > > > useful
> > > >
> > > > > > >> > >>> > > > > for
> > > >
> > > > > > >> > >>> every
> > > >
> > > > > > >> > >>> > > > > instance of rest proxy to be able to renew the
> > > tokens.
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > It would be clearer if we can document the
> request
> > > >
> > > > > > protocol
> > > >
> > > > > > >> > like
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > uence/display/KAFKA/KIP-
> > > >
> > > > > > >> > >>> > >
> > > >
> > > > > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> > > >
> > > > > > operations#KIP-4-
> > > >
> > > > > > >> > >>> > > Commandlineandcentralizedadministrativeoperations-
> > > >
> > > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> > 2945):(VotedandPlannedforin0.
> > > >
> > > > > > 10.1.0)
> > > >
> > > > > > >> > >>> > > > > .
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > It would also be useful to document the client
> > APIs.
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > Thanks,
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > Jun
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth
> brahmbhatt
> > <
> > > >
> > > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > > > Hi,
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > I am suggesting that we will only allow the
> > > renewal
> > > > by
> > > >
> > > > > > >> > >>> > > > > > users
> > > >
> > > > > > >> > >>> that
> > > >
> > > > > > >> > >>> > > > > > authenticated using *non* delegation token
> > > > mechanism.
> > > >
> > > > > > For
> > > >
> > > > > > >> > >>> example,
> > > >
> > > > > > >> > >>> > If
> > > >
> > > > > > >> > >>> > > > > user
> > > >
> > > > > > >> > >>> > > > > > Alice authenticated using kerberos and
> requested
> > > >
> > > > > > >> > >>> > > > > > delegation
> > > >
> > > > > > >> > >>> tokens,
> > > >
> > > > > > >> > >>> > > > only
> > > >
> > > > > > >> > >>> > > > > > user Alice authenticated via non delegation
> > token
> > > >
> > > > > > >> > >>> > > > > > mechanism
> > > >
> > > > > > >> > can
> > > >
> > > > > > >> > >>> > > renew.
> > > >
> > > > > > >> > >>> > > > > > Clients that have access to delegation tokens
> > can
> > > > not
> > > >
> > > > > > >> > >>> > > > > > issue
> > > >
> > > > > > >> > >>> > renewal
> > > >
> > > > > > >> > >>> > > > > > request for renewing their own token and this
> is
> > > >
> > > > > > primarily
> > > >
> > > > > > >> > >>> > important
> > > >
> > > > > > >> > >>> > > to
> > > >
> > > > > > >> > >>> > > > > > reduce the time window for which a compromised
> > > token
> > > >
> > > > > > will
> > > >
> > > > > > >> > >>> > > > > > be
> > > >
> > > > > > >> > >>> valid.
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > To clarify, Yes any authenticated user can
> > request
> > > >
> > > > > > >> > >>> > > > > > delegation
> > > >
> > > > > > >> > >>> > tokens
> > > >
> > > > > > >> > >>> > > > but
> > > >
> > > > > > >> > >>> > > > > > even here I would recommend to avoid creating
> a
> > > > chain
> > > >
> > > > > > >> > >>> > > > > > where a
> > > >
> > > > > > >> > >>> > client
> > > >
> > > > > > >> > >>> > > > > > authenticated via delegation token request for
> > > more
> > > >
> > > > > > >> > delegation
> > > >
> > > > > > >> > >>> > > tokens.
> > > >
> > > > > > >> > >>> > > > > > Basically anyone can request delegation token,
> > as
> > > > long
> > > >
> > > > > > as
> > > >
> > > > > > >> > they
> > > >
> > > > > > >> > >>> > > > > authenticate
> > > >
> > > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > Aren't classes listed here
> > > >
> > > > > > >> > >>> > > > > > <
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > >
> > > >
> > > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > > uence/display/KAFKA/KIP-
> > > >
> > > > > > >> > >>> > > 48+Delegation+token+support+fo
> > > >
> > > > > r+Kafka#KIP-48Delegationtokens
> > > >
> > > > > > >> > >>> upportforKaf
> > > >
> > > > > > >> > >>> > > ka-PublicInterfaces
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > sufficient?
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > Thanks
> > > >
> > > > > > >> > >>> > > > > > Parth
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > > >
> > > > > > >> > >>> > > > > > <ju...@confluent.io>
> > > >
> > > > > > >> > >>> wrote:
> > > >
> > > > > > >> > >>> > > > > >
> > > >
> > > > > > >> > >>> > > > > > > Parth,
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > Thanks for the reply. A couple of comments
> > > inline
> > > >
> > > > > > below.
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> > > > brahmbhatt <
> > > >
> > > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By
> original
> > > >
> > > > > > requester
> > > >
> > > > > > >> > >>> only? or
> > > >
> > > > > > >> > >>> > > > using
> > > >
> > > > > > >> > >>> > > > > > > > Kerberos
> > > >
> > > > > > >> > >>> > > > > > > > auth only?
> > > >
> > > > > > >> > >>> > > > > > > > My recommendation is to do this only using
> > > >
> > > > > Kerberos
> > > >
> > > > > > >> > >>> > > > > > > > auth
> > > >
> > > > > > >> > and
> > > >
> > > > > > >> > >>> > only
> > > >
> > > > > > >> > >>> > > > > threw
> > > >
> > > > > > >> > >>> > > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > renewer specified during the acquisition
> > > > request.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you
> > saying
> > > >
> > > > > that
> > > >
> > > > > > >> > >>> > > > > > > any
> > > >
> > > > > > >> > >>> client
> > > >
> > > > > > >> > >>> > > > > > > authenticated with the delegation token can
> > > renew,
> > > >
> > > > > > i.e.
> > > >
> > > > > > >> > there
> > > >
> > > > > > >> > >>> is
> > > >
> > > > > > >> > >>> > no
> > > >
> > > > > > >> > >>> > > > > > renewer
> > > >
> > > > > > >> > >>> > > > > > > needed?
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > Also, just to be clear, any authenticated
> > client
> > > >
> > > > > > (either
> > > >
> > > > > > >> > >>> through
> > > >
> > > > > > >> > >>> > > SASL
> > > >
> > > > > > >> > >>> > > > > or
> > > >
> > > > > > >> > >>> > > > > > > SSL) can request a delegation token for the
> > > >
> > > > > > >> > >>> > > > > > > authenticated
> > > >
> > > > > > >> > >>> user,
> > > >
> > > > > > >> > >>> > > > right?
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in
> > ZK?
> > > >
> > > > > > >> > >>> > > > > > > > My recommendation is still to store in ZK
> or
> > > not
> > > >
> > > > > > store
> > > >
> > > > > > >> > them
> > > >
> > > > > > >> > >>> at
> > > >
> > > > > > >> > >>> > > all.
> > > >
> > > > > > >> > >>> > > > > The
> > > >
> > > > > > >> > >>> > > > > > > > whole controller based distribution is too
> > > much
> > > >
> > > > > > >> > >>> > > > > > > > overhead
> > > >
> > > > > > >> > >>> with
> > > >
> > > > > > >> > >>> > not
> > > >
> > > > > > >> > >>> > > > > much
> > > >
> > > > > > >> > >>> > > > > > to
> > > >
> > > > > > >> > >>> > > > > > > > achieve.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> > > >
> > > > > > >> > >>> > > > > > > > Either by expiration time out or through
> an
> > > >
> > > > > explicit
> > > >
> > > > > > >> > >>> request to
> > > >
> > > > > > >> > >>> > > > > > > invalidate.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > > >
> > > > > > >> > >>> > > > > > > > SCRAM
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 5. What is the impersonation proposal (it
> > > wasn't
> > > >
> > > > > in
> > > >
> > > > > > >> > >>> > > > > > > > the
> > > >
> > > > > > >> > KIP
> > > >
> > > > > > >> > >>> but
> > > >
> > > > > > >> > >>> > > was
> > > >
> > > > > > >> > >>> > > > > > > > discussed
> > > >
> > > > > > >> > >>> > > > > > > > in this thread)?
> > > >
> > > > > > >> > >>> > > > > > > > There is no imperonation proposal. I tried
> > and
> > > >
> > > > > > >> > >>> > > > > > > > explained
> > > >
> > > > > > >> > how
> > > >
> > > > > > >> > >>> > its
> > > >
> > > > > > >> > >>> > > a
> > > >
> > > > > > >> > >>> > > > > > > > different problem and why its not really
> > > > necessary
> > > >
> > > > > > to
> > > >
> > > > > > >> > >>> discuss
> > > >
> > > > > > >> > >>> > > that
> > > >
> > > > > > >> > >>> > > > as
> > > >
> > > > > > >> > >>> > > > > > > part
> > > >
> > > > > > >> > >>> > > > > > > > of this KIP. This KIP will not support
> any
> > > >
> > > > > > >> > impersonation,
> > > >
> > > > > > >> > >>> it
> > > >
> > > > > > >> > >>> > > will
> > > >
> > > > > > >> > >>> > > > > just
> > > >
> > > > > > >> > >>> > > > > > > be
> > > >
> > > > > > >> > >>> > > > > > > > another way to authenticate.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what
> > > > actions?
> > > >
> > > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > Could we document the format of the new
> > > >
> > > > > > request/response
> > > >
> > > > > > >> > and
> > > >
> > > > > > >> > >>> > their
> > > >
> > > > > > >> > >>> > > > > > > associated Resource and Operation for ACL?
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > 7. How would the delegation token be
> > > configured
> > > > in
> > > >
> > > > > > the
> > > >
> > > > > > >> > >>> client?
> > > >
> > > > > > >> > >>> > > > > > > > Should be through config. I wasn't
> planning
> > on
> > > >
> > > > > > >> > >>> > > > > > > > supporting
> > > >
> > > > > > >> > >>> JAAS
> > > >
> > > > > > >> > >>> > > for
> > > >
> > > > > > >> > >>> > > > > > > tokens.
> > > >
> > > > > > >> > >>> > > > > > > > I don't believe hadoop does this either.
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > Thanks
> > > >
> > > > > > >> > >>> > > > > > > > Parth
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> > > >
> > > > > > >> > jun@confluent.io>
> > > >
> > > > > > >> > >>> > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > Harsha,
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > Another question.
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > 9. How would the delegation token be
> > > > configured
> > > >
> > > > > in
> > > >
> > > > > > >> > >>> > > > > > > > > the
> > > >
> > > > > > >> > >>> > client?
> > > >
> > > > > > >> > >>> > > > The
> > > >
> > > > > > >> > >>> > > > > > > > standard
> > > >
> > > > > > >> > >>> > > > > > > > > way is to do this through JAAS. However,
> > we
> > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > need
> > > >
> > > > > > >> > to
> > > >
> > > > > > >> > >>> > think
> > > >
> > > > > > >> > >>> > > > > > through
> > > >
> > > > > > >> > >>> > > > > > > if
> > > >
> > > > > > >> > >>> > > > > > > > > this is convenient in a shared
> > environment.
> > > > For
> > > >
> > > > > > >> > example,
> > > >
> > > > > > >> > >>> > when a
> > > >
> > > > > > >> > >>> > > > new
> > > >
> > > > > > >> > >>> > > > > > > task
> > > >
> > > > > > >> > >>> > > > > > > > is
> > > >
> > > > > > >> > >>> > > > > > > > > added to a Storm worker node, do we need
> > to
> > > >
> > > > > > >> > >>> > > > > > > > > dynamically
> > > >
> > > > > > >> > >>> add a
> > > >
> > > > > > >> > >>> > > new
> > > >
> > > > > > >> > >>> > > > > > > section
> > > >
> > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> > convenient
> > > if
> > > >
> > > > > we
> > > >
> > > > > > >> > >>> > > > > > > > > can
> > > >
> > > > > > >> > >>> pass in
> > > >
> > > > > > >> > >>> > > the
> > > >
> > > > > > >> > >>> > > > > > token
> > > >
> > > > > > >> > >>> > > > > > > > > through the config directly w/o going
> > > through
> > > >
> > > > > > JAAS.
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > Are you or Parth still actively working
> on
> > > > this
> > > >
> > > > > > KIP?
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > Thanks,
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > Jun
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun
> Rao <
> > > >
> > > > > > >> > >>> jun@confluent.io>
> > > >
> > > > > > >> > >>> > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > 2. It would be good to document the
> > format
> > > > of
> > > >
> > > > > > the
> > > >
> > > > > > >> > data
> > > >
> > > > > > >> > >>> > stored
> > > >
> > > > > > >> > >>> > > > in
> > > >
> > > > > > >> > >>> > > > > > ZK.
> > > >
> > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on
> > > > whether
> > > >
> > > > > > the
> > > >
> > > > > > >> > tokens
> > > >
> > > > > > >> > >>> > > should
> > > >
> > > > > > >> > >>> > > > > be
> > > >
> > > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > > config/acl/quota,
> > > >
> > > > > or
> > > >
> > > > > > >> > through
> > > >
> > > > > > >> > >>> the
> > > >
> > > > > > >> > >>> > > > > > > controller.
> > > >
> > > > > > >> > >>> > > > > > > > > > Currently, the controller is only
> > designed
> > > > for
> > > >
> > > > > > >> > >>> propagating
> > > >
> > > > > > >> > >>> > > > topic
> > > >
> > > > > > >> > >>> > > > > > > > > metadata,
> > > >
> > > > > > >> > >>> > > > > > > > > > but not other data.
> > > >
> > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the
> token
> > > >
> > > > > instead
> > > >
> > > > > > >> > >>> > > > > > > > > > of
> > > >
> > > > > > >> > >>> > > DIGEST-MD5
> > > >
> > > > > > >> > >>> > > > > > since
> > > >
> > > > > > >> > >>> > > > > > > > it's
> > > >
> > > > > > >> > >>> > > > > > > > > > deprecated?
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem
> > broken.
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > Thanks,
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > Jun
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen
> > > >
> > > > > Shapira <
> > > >
> > > > > > >> > >>> > > > > gwen@confluent.io>
> > > >
> > > > > > >> > >>> > > > > > > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> From what I can see, remaining
> > questions
> > > > are:
> > > >
> > > > > > >> > >>> > > > > > > > > >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By
> > > > original
> > > >
> > > > > > >> > requester
> > > >
> > > > > > >> > >>> > only?
> > > >
> > > > > > >> > >>> > > > or
> > > >
> > > > > > >> > >>> > > > > > > using
> > > >
> > > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > > >
> > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker
> or
> > in
> > > > ZK?
> > > >
> > > > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated /
> > expired?
> > > >
> > > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is
> used?
> > > >
> > > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal
> > (it
> > > >
> > > > > > wasn't
> > > >
> > > > > > >> > >>> > > > > > > > > >> in
> > > >
> > > > > > >> > the
> > > >
> > > > > > >> > >>> > KIP
> > > >
> > > > > > >> > >>> > > > but
> > > >
> > > > > > >> > >>> > > > > > was
> > > >
> > > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > > >
> > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for
> > what
> > > >
> > > > > > actions?
> > > >
> > > > > > >> > >>> > > > > > > > > >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> Gwen
> > > >
> > > > > > >> > >>> > > > > > > > > >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM,
> Harsha
> > <
> > > >
> > > > > > >> > >>> kafka@harsha.io>
> > > >
> > > > > > >> > >>> > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >
> > Unfortunately
> > > I
> > > >
> > > > > > >> > >>> > > > > > > > > >> > couldn't
> > > >
> > > > > > >> > >>> attend
> > > >
> > > > > > >> > >>> > > the
> > > >
> > > > > > >> > >>> > > > > KIP
> > > >
> > > > > > >> > >>> > > > > > > > > meeting
> > > >
> > > > > > >> > >>> > > > > > > > > >> > when
> > > delegation
> > > >
> > > > > > tokens
> > > >
> > > > > > >> > >>> > discussed.
> > > >
> > > > > > >> > >>> > > > > > > > Appreciate
> > > >
> > > > > > >> > >>> > > > > > > > > if
> > > >
> > > > > > >> > >>> > > > > > > > > >> > you can
> > update
> > > > the
> > > >
> > > > > > >> > thread if
> > > >
> > > > > > >> > >>> > you
> > > >
> > > > > > >> > >>> > > > have
> > > >
> > > > > > >> > >>> > > > > > any
> > > >
> > > > > > >> > >>> > > > > > > > > >> > further
> > > > questions.
> > > >
> > > > > > >> > >>> > > > > > > > > >> > Thanks,
> > > >
> > > > > > >> > >>> > > > > > > > > >> > Harsha
> > > >
> > > > > > >> > >>> > > > > > > > > >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM,
> > > Liquan
> > > >
> > > > > Pei
> > > >
> > > > > > >> > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> It seems that the links to images
> in
> > > the
> > > >
> > > > > KIP
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> are
> > > >
> > > > > > >> > >>> > broken.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> Liquan
> > > >
> > > > > > >> > >>> > > > > > > > > >> >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM,
> > parth
> > > >
> > > > > > >> > brahmbhatt <
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com>
> wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >>
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> > getDelegationTokenAs
> > > >
> > > > > mean?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > In the current proposal we only
> > > allow
> > > > a
> > > >
> > > > > > user
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > to
> > > >
> > > > > > >> > >>> get
> > > >
> > > > > > >> > >>> > > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > token
> > > >
> > > > > > >> > >>> > > > > > > > > >> for
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > the identity that it
> authenticated
> > > as
> > > >
> > > > > > using
> > > >
> > > > > > >> > >>> another
> > > >
> > > > > > >> > >>> > > > > > mechanism,
> > > >
> > > > > > >> > >>> > > > > > > > i.e.
> > > >
> > > > > > >> > >>> > > > > > > > > >> A user
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab
> > for
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > principal
> > > >
> > > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > > >
> > > > > > >> > >>> > > > > > > > > >> will get
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that user
> > > only.
> > > > In
> > > >
> > > > > > >> > future I
> > > >
> > > > > > >> > >>> > think
> > > >
> > > > > > >> > >>> > > > we
> > > >
> > > > > > >> > >>> > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > have
> > > >
> > > > > > >> > >>> > > > > > > > > >> to
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > extend support such that we
> allow
> > > some
> > > >
> > > > > set
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > of
> > > >
> > > > > > >> > >>> users (
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > > >
> > > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > > >
> > > > > > >> > >>> > > > to
> > > >
> > > > > > >> > >>> > > > > > > > acquire
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of
> > other
> > > >
> > > > > users
> > > >
> > > > > > >> > whose
> > > >
> > > > > > >> > >>> > > identity
> > > >
> > > > > > >> > >>> > > > > > they
> > > >
> > > > > > >> > >>> > > > > > > > have
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka
> > > brokers
> > > >
> > > > > > will
> > > >
> > > > > > >> > have
> > > >
> > > > > > >> > >>> ACLs
> > > >
> > > > > > >> > >>> > > to
> > > >
> > > > > > >> > >>> > > > > > > control
> > > >
> > > > > > >> > >>> > > > > > > > > >> which
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate
> > > other
> > > >
> > > > > > users
> > > >
> > > > > > >> > and
> > > >
> > > > > > >> > >>> get
> > > >
> > > > > > >> > >>> > > > tokens
> > > >
> > > > > > >> > >>> > > > > > on
> > > >
> > > > > > >> > >>> > > > > > > > > >> behalf of
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a
> > > whole
> > > >
> > > > > > >> > different
> > > >
> > > > > > >> > >>> > > problem
> > > >
> > > > > > >> > >>> > > > in
> > > >
> > > > > > >> > >>> > > > > > my
> > > >
> > > > > > >> > >>> > > > > > > > > >> opinion and
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in
> > separate
> > > >
> > > > > KIP.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of
> > > > getting
> > > >
> > > > > > and
> > > >
> > > > > > >> > >>> renewing
> > > >
> > > > > > >> > >>> > > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> tokens?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > Typically this should be very
> very
> > > > low,
> > > >
> > > > > 1
> > > >
> > > > > > >> > request
> > > >
> > > > > > >> > >>> per
> > > >
> > > > > > >> > >>> > > > > minute
> > > >
> > > > > > >> > >>> > > > > > > is a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > relatively high estimate.
> However
> > it
> > > >
> > > > > > depends
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > on
> > > >
> > > > > > >> > >>> the
> > > >
> > > > > > >> > >>> > > token
> > > >
> > > > > > >> > >>> > > > > > > > > >> expiration. I am
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > less worried about the extra
> load
> > it
> > > >
> > > > > puts
> > > >
> > > > > > on
> > > >
> > > > > > >> > >>> > controller
> > > >
> > > > > > >> > >>> > > > vs
> > > >
> > > > > > >> > >>> > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > > added
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > complexity and the value it
> > offers.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > Thanks
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > Parth
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM,
> > > > Ismael
> > > >
> > > > > > Juma
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > <
> > > >
> > > > > > >> > >>> > > > > > > ismael@juma.me.uk>
> > > >
> > > > > > >> > >>> > > > > > > > > >> wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would
> probably
> > > >
> > > > > > require a
> > > >
> > > > > > >> > >>> separate
> > > >
> > > > > > >> > >>> > > KIP
> > > >
> > > > > > >> > >>> > > > > as
> > > >
> > > > > > >> > >>> > > > > > it
> > > >
> > > > > > >> > >>> > > > > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > introduce user visible
> changes.
> > We
> > > >
> > > > > could
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > also
> > > >
> > > > > > >> > >>> > update
> > > >
> > > > > > >> > >>> > > > > KIP-48
> > > >
> > > > > > >> > >>> > > > > > > to
> > > >
> > > > > > >> > >>> > > > > > > > > >> have this
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > information, but it seems
> > cleaner
> > > to
> > > >
> > > > > do
> > > >
> > > > > > it
> > > >
> > > > > > >> > >>> > > separately.
> > > >
> > > > > > >> > >>> > > > We
> > > >
> > > > > > >> > >>> > > > > > can
> > > >
> > > > > > >> > >>> > > > > > > > > >> discuss
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > that
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > Ismael
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19
> PM,
> > > >
> > > > > Rajini
> > > >
> > > > > > >> > Sivaram
> > > >
> > > > > > >> > >>> <
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com>
> > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > Ismael,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > > >
> > > > > > >> > jira/browse/KAFKA-3751)
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
> > > >
> > > > > mechanism.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > Would
> > > >
> > > > > > >> > >>> that
> > > >
> > > > > > >> > >>> > > need
> > > >
> > > > > > >> > >>> > > > > > > another
> > > >
> > > > > > >> > >>> > > > > > > > > >> KIP? If
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this
> > mechanism,
> > > > can
> > > >
> > > > > > this
> > > >
> > > > > > >> > just
> > > >
> > > > > > >> > >>> be
> > > >
> > > > > > >> > >>> > a
> > > >
> > > > > > >> > >>> > > > JIRA
> > > >
> > > > > > >> > >>> > > > > > > that
> > > >
> > > > > > >> > >>> > > > > > > > > gets
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > reviewed
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > Thank you,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > Rajini
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46
> > PM,
> > > >
> > > > > > Ismael
> > > >
> > > > > > >> > Juma <
> > > >
> > > > > > >> > >>> > > > > > > > > ismael@juma.me.uk>
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems
> > > like
> > > > a
> > > >
> > > > > > good
> > > >
> > > > > > >> > >>> > candidate.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently
> > > mentioned
> > > >
> > > > > > this
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > as
> > > >
> > > > > > >> > a
> > > >
> > > > > > >> > >>> SASL
> > > >
> > > > > > >> > >>> > > > > > mechanism
> > > >
> > > > > > >> > >>> > > > > > > > > that
> > > >
> > > > > > >> > >>> > > > > > > > > >> might
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > be
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I
> have
> > > been
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > meaning
> > > >
> > > > > > >> > to
> > > >
> > > > > > >> > >>> > check
> > > >
> > > > > > >> > >>> > > > it
> > > >
> > > > > > >> > >>> > > > > in
> > > >
> > > > > > >> > >>> > > > > > > > more
> > > >
> > > > > > >> > >>> > > > > > > > > >> detail.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > Good
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > to know that you are
> willing
> > > to
> > > >
> > > > > > >> > contribute
> > > >
> > > > > > >> > >>> an
> > > >
> > > > > > >> > >>> > > > > > > > implementation.
> > > >
> > > > > > >> > >>> > > > > > > > > >> Maybe
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > we
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > should file a separate
> JIRA
> > > for
> > > >
> > > > > > this?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > Ismael
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at
> 2:12
> > > PM,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > Rajini
> > > >
> > > > > > >> > >>> > Sivaram <
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > rajinisivaram@googlemail.com>
> > > >
> > > > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge
> > > > Response
> > > >
> > > > > > >> > >>> > Authentication
> > > >
> > > > > > >> > >>> > > > > > > > Mechanism)
> > > >
> > > > > > >> > >>> > > > > > > > > >> is a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > better
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than
> Digest-MD5.
> > > > Java
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> > > >
> > > > > > >> > >>> come
> > > >
> > > > > > >> > >>> > > > with a
> > > >
> > > > > > >> > >>> > > > > > > > > built-in
> > > >
> > > > > > >> > >>> > > > > > > > > >> SCRAM
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or
> SaslClient,
> > > but
> > > > I
> > > >
> > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > be
> > > >
> > > > > > >> > >>> happy
> > > >
> > > > > > >> > >>> > > to
> > > >
> > > > > > >> > >>> > > > > add
> > > >
> > > > > > >> > >>> > > > > > > > > support
> > > >
> > > > > > >> > >>> > > > > > > > > >> in
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > since
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful
> > > mechanism
> > > >
> > > > > to
> > > >
> > > > > > >> > support
> > > >
> > > > > > >> > >>> > > anyway.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > https://tools.ietf.org/html/
> > > >
> > > > > > rfc7677
> > > >
> > > > > > >> > >>> > describes
> > > >
> > > > > > >> > >>> > > > the
> > > >
> > > > > > >> > >>> > > > > > > > protocol
> > > >
> > > > > > >> > >>> > > > > > > > > >> for
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at
> > 2:37
> > > > AM,
> > > >
> > > > > > Jun
> > > >
> > > > > > >> > Rao <
> > > >
> > > > > > >> > >>> > > > > > > > jun@confluent.io
> > > >
> > > > > > >> > >>> > > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the
> > > explanation.
> > > > A
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > couple
> > > >
> > > > > > >> > of
> > > >
> > > > > > >> > >>> > more
> > > >
> > > > > > >> > >>> > > > > > > questions.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> > > >
> > > > > > >> > >>> mean?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the
> typical
> > > rate
> > > >
> > > > > of
> > > >
> > > > > > >> > getting
> > > >
> > > > > > >> > >>> and
> > > >
> > > > > > >> > >>> > > > > > renewing
> > > >
> > > > > > >> > >>> > > > > > > > > >> delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > tokens?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an
> impact
> > on
> > > >
> > > > > > whether
> > > >
> > > > > > >> > they
> > > >
> > > > > > >> > >>> > > should
> > > >
> > > > > > >> > >>> > > > be
> > > >
> > > > > > >> > >>> > > > > > > > > directed
> > > >
> > > > > > >> > >>> > > > > > > > > >> to the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016
> at
> > > 1:19
> > > >
> > > > > PM,
> > > >
> > > > > > >> > parth
> > > >
> > > > > > >> > >>> > > > > brahmbhatt <
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > brahmbhatt.parth@gmail.com>
> > > >
> > > > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for
> reviewing.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a
> > Cluster
> > > >
> > > > > > action
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > >
> > > > > > >> > add
> > > >
> > > > > > >> > >>> > acls
> > > >
> > > > > > >> > >>> > > > on
> > > >
> > > > > > >> > >>> > > > > > who
> > > >
> > > > > > >> > >>> > > > > > > > can
> > > >
> > > > > > >> > >>> > > > > > > > > >> request
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see
> > the
> > > > use
> > > >
> > > > > > case
> > > >
> > > > > > >> > for
> > > >
> > > > > > >> > >>> that
> > > >
> > > > > > >> > >>> > > yet
> > > >
> > > > > > >> > >>> > > > > but
> > > >
> > > > > > >> > >>> > > > > > > > down
> > > >
> > > > > > >> > >>> > > > > > > > > >> the line
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > when
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > we
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > > >
> > > > > > >> > getDelegationTokenAs
> > > >
> > > > > > >> > >>> it
> > > >
> > > > > > >> > >>> > > will
> > > >
> > > > > > >> > >>> > > > > be
> > > >
> > > > > > >> > >>> > > > > > > > > >> necessary.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend
> > > tokens
> > > > to
> > > >
> > > > > > be
> > > >
> > > > > > >> > only
> > > >
> > > > > > >> > >>> > > > > > > used/distributed
> > > >
> > > > > > >> > >>> > > > > > > > > >> over
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > secure
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what
> > > design
> > > >
> > > > > we
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > end
> > > >
> > > > > > >> > up
> > > >
> > > > > > >> > >>> > > choosing
> > > >
> > > > > > >> > >>> > > > > > > > > >> Invalidation will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > be
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of
> > every
> > > >
> > > > > broker
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > or
> > > >
> > > > > > >> > >>> > > controller.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
> > > >
> > > > > > documented
> > > >
> > > > > > >> > >>> somewhere
> > > >
> > > > > > >> > >>> > > > that
> > > >
> > > > > > >> > >>> > > > > > > > > >> invalidation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > directly
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper
> > but
> > > >
> > > > > that
> > > >
> > > > > > is
> > > >
> > > > > > >> > not
> > > >
> > > > > > >> > >>> the
> > > >
> > > > > > >> > >>> > > > > intent.
> > > >
> > > > > > >> > >>> > > > > > > > > >> Invalidation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > either
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or
> > due
> > > to
> > > >
> > > > > > >> > >>> expiration. No
> > > >
> > > > > > >> > >>> > > > > direct
> > > >
> > > > > > >> > >>> > > > > > > > > >> zookeeper
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > interaction
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > from
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also
> stores
> > > the
> > > >
> > > > > > >> > >>> DelegationToken
> > > >
> > > > > > >> > >>> > > > > without
> > > >
> > > > > > >> > >>> > > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > > >> hmac in
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry
> > > about
> > > >
> > > > > the
> > > >
> > > > > > >> > >>> confusion.
> > > >
> > > > > > >> > >>> > > The
> > > >
> > > > > > >> > >>> > > > > sole
> > > >
> > > > > > >> > >>> > > > > > > > > >> purpose of
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > in
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
> > > >
> > > > > distribution
> > > >
> > > > > > >> > channel
> > > >
> > > > > > >> > >>> > for
> > > >
> > > > > > >> > >>> > > > > tokens
> > > >
> > > > > > >> > >>> > > > > > > > > >> between all
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > brokers
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures
> > only
> > > >
> > > > > tokens
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > that
> > > >
> > > > > > >> > >>> were
> > > >
> > > > > > >> > >>> > > > > > generated
> > > >
> > > > > > >> > >>> > > > > > > by
> > > >
> > > > > > >> > >>> > > > > > > > > >> making a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > request
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > to a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be
> > accepted
> > > >
> > > > > (more
> > > >
> > > > > > on
> > > >
> > > > > > >> > this
> > > >
> > > > > > >> > >>> in
> > > >
> > > > > > >> > >>> > > > second
> > > >
> > > > > > >> > >>> > > > > > > > > >> paragraph). The
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > token
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few
> > elements
> > > >
> > > > > > (owner,
> > > >
> > > > > > >> > >>> renewer,
> > > >
> > > > > > >> > >>> > > > uuid
> > > >
> > > > > > >> > >>> > > > > ,
> > > >
> > > > > > >> > >>> > > > > > > > > >> expiration,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > hmac)
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > ,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > one
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > of
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally
> > > >
> > > > > generated
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> > > >
> > > > > > >> > >>> but
> > > >
> > > > > > >> > >>> > > hmac
> > > >
> > > > > > >> > >>> > > > it
> > > >
> > > > > > >> > >>> > > > > > > self
> > > >
> > > > > > >> > >>> > > > > > > > is
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > derivable
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > if
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > you
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other
> > > > elements
> > > >
> > > > > of
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the
> > > >
> > > > > > >> > >>> token
> > > >
> > > > > > >> > >>> > +
> > > >
> > > > > > >> > >>> > > > > secret
> > > >
> > > > > > >> > >>> > > > > > > key
> > > >
> > > > > > >> > >>> > > > > > > > > to
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > generate
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > hmac.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does
> > not
> > > >
> > > > > > provide
> > > >
> > > > > > >> > SSL
> > > >
> > > > > > >> > >>> > > support
> > > >
> > > > > > >> > >>> > > > we
> > > >
> > > > > > >> > >>> > > > > > do
> > > >
> > > > > > >> > >>> > > > > > > > not
> > > >
> > > > > > >> > >>> > > > > > > > > >> want the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > entire
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> > > > transferred
> > > >
> > > > > > to
> > > >
> > > > > > >> > >>> zookeeper
> > > >
> > > > > > >> > >>> > > as
> > > >
> > > > > > >> > >>> > > > > that
> > > >
> > > > > > >> > >>> > > > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> be an
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > insecure
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > wire
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we
> > > only
> > > >
> > > > > > store
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > all
> > > >
> > > > > > >> > >>> the
> > > >
> > > > > > >> > >>> > > other
> > > >
> > > > > > >> > >>> > > > > > > > elements
> > > >
> > > > > > >> > >>> > > > > > > > > >> of a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can
> > read
> > > >
> > > > > these
> > > >
> > > > > > >> > >>> elements
> > > >
> > > > > > >> > >>> > and
> > > >
> > > > > > >> > >>> > > > > > because
> > > >
> > > > > > >> > >>> > > > > > > > > they
> > > >
> > > > > > >> > >>> > > > > > > > > >> also
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > have
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > access
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they
> > will
> > > be
> > > >
> > > > > > able
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > >
> > > > > > >> > >>> > generate
> > > >
> > > > > > >> > >>> > > > > hmac
> > > >
> > > > > > >> > >>> > > > > > on
> > > >
> > > > > > >> > >>> > > > > > > > > >> their end.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the
> alternative
> > > >
> > > > > > proposed
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > is
> > > >
> > > > > > >> > to
> > > >
> > > > > > >> > >>> > avoid
> > > >
> > > > > > >> > >>> > > > > > > zookeeper
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > altogether. A
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > Client
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker
> with
> > > >
> > > > > required
> > > >
> > > > > > >> > >>> > information
> > > >
> > > > > > >> > >>> > > > > > (owner,
> > > >
> > > > > > >> > >>> > > > > > > > > >> renwer,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > expiration)
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed
> hmac,
> > > >
> > > > > uuid).
> > > >
> > > > > > >> > Broker
> > > >
> > > > > > >> > >>> > won't
> > > >
> > > > > > >> > >>> > > > > store
> > > >
> > > > > > >> > >>> > > > > > > this
> > > >
> > > > > > >> > >>> > > > > > > > > in
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > zookeeper.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > From
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client
> > can
> > > >
> > > > > > contact
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > >
> > > > > > >> > >>> > broker
> > > >
> > > > > > >> > >>> > > > with
> > > >
> > > > > > >> > >>> > > > > > all
> > > >
> > > > > > >> > >>> > > > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > token
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
> > > >
> > > > > > expiration,
> > > >
> > > > > > >> > hmac,
> > > >
> > > > > > >> > >>> > > uuid)
> > > >
> > > > > > >> > >>> > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > borker
> > > >
> > > > > > >> > >>> > > > > > > > > >> will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > regenerate
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as
> it
> > > >
> > > > > matches
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > with
> > > >
> > > > > > >> > >>> hmac
> > > >
> > > > > > >> > >>> > > > > > presented
> > > >
> > > > > > >> > >>> > > > > > > by
> > > >
> > > > > > >> > >>> > > > > > > > > >> client ,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > broker
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> > > >
> > > > > > >> > >>> Only
> > > >
> > > > > > >> > >>> > > > > problem
> > > >
> > > > > > >> > >>> > > > > > > with
> > > >
> > > > > > >> > >>> > > > > > > > > >> this
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > approach
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > is
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > if
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
> > > >
> > > > > compromised
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > > >
> > > > > > >> > >>> client
> > > >
> > > > > > >> > >>> > > can
> > > >
> > > > > > >> > >>> > > > > now
> > > >
> > > > > > >> > >>> > > > > > > > > generate
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > random
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > tokens
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be
> > able
> > > to
> > > >
> > > > > > >> > >>> authenticate
> > > >
> > > > > > >> > >>> > as
> > > >
> > > > > > >> > >>> > > > any
> > > >
> > > > > > >> > >>> > > > > > user
> > > >
> > > > > > >> > >>> > > > > > > > > they
> > > >
> > > > > > >> > >>> > > > > > > > > >> like.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > with
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we
> guarantee
> > > > that
> > > >
> > > > > > only
> > > >
> > > > > > >> > >>> tokens
> > > >
> > > > > > >> > >>> > > > > acquired
> > > >
> > > > > > >> > >>> > > > > > > via
> > > >
> > > > > > >> > >>> > > > > > > > a
> > > >
> > > > > > >> > >>> > > > > > > > > >> broker
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > (using
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > some
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other
> than
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> > > >
> > > > > > >> > >>> token)
> > > >
> > > > > > >> > >>> > > will
> > > >
> > > > > > >> > >>> > > > > be
> > > >
> > > > > > >> > >>> > > > > > > > > >> accepted. We
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > need
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > to
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which
> proposal
> > > > makes
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > more
> > > >
> > > > > > >> > >>> sense
> > > >
> > > > > > >> > >>> > and
> > > >
> > > > > > >> > >>> > > > we
> > > >
> > > > > > >> > >>> > > > > > can
> > > >
> > > > > > >> > >>> > > > > > > go
> > > >
> > > > > > >> > >>> > > > > > > > > >> over it
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > in
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you
> forward
> > > the
> > > >
> > > > > > invite
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > > >
> > > > > > >> > >>> me?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016
> > at
> > > >
> > > > > 10:35
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> > > >
> > > > > > >> > Jun
> > > >
> > > > > > >> > >>> > Rao <
> > > >
> > > > > > >> > >>> > > > > > > > > >> jun@confluent.io>
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > wrote:
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the
> KIP.
> > A
> > > > few
> > > >
> > > > > > >> > comments.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This
> > potentially
> > > > can
> > > >
> > > > > be
> > > >
> > > > > > >> > useful
> > > >
> > > > > > >> > >>> for
> > > >
> > > > > > >> > >>> > > > Kafka
> > > >
> > > > > > >> > >>> > > > > > > > Connect
> > > >
> > > > > > >> > >>> > > > > > > > > >> and
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > rest
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker
> agent
> > > > will
> > > >
> > > > > > need
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> > > >
> > > > > > >> > >>> run a
> > > >
> > > > > > >> > >>> > > > task
> > > >
> > > > > > >> > >>> > > > > on
> > > >
> > > > > > >> > >>> > > > > > > > > behalf
> > > >
> > > > > > >> > >>> > > > > > > > > >> of a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > client.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > We
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to
> > change
> > > > how
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> > > >
> > > > > > >> > >>> > services
> > > >
> > > > > > >> > >>> > > > use
> > > >
> > > > > > >> > >>> > > > > > > Kafka
> > > >
> > > > > > >> > >>> > > > > > > > > >> clients
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> (producer/consumer).
> > > >
> > > > > Instead
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> > > >
> > > > > > >> > >>> > shared
> > > >
> > > > > > >> > >>> > > > > client
> > > >
> > > > > > >> > >>> > > > > > > per
> > > >
> > > > > > >> > >>> > > > > > > > > >> worker,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > we
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > will
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > need
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > a
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user
> task
> > > > since
> > > >
> > > > > > the
> > > >
> > > > > > >> > >>> > > > authentication
> > > >
> > > > > > >> > >>> > > > > > > > happens
> > > >
> > > > > > >> > >>> > > > > > > > > >> at the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > connection
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka
> > > > Connect,
> > > >
> > > > > > the
> > > >
> > > > > > >> > >>> renewer
> > > >
> > > > > > >> > >>> > > will
> > > >
> > > > > > >> > >>> > > > be
> > > >
> > > > > > >> > >>> > > > > > the
> > > >
> > > > > > >> > >>> > > > > > > > > >> workers.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > So,
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > we
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow
> > multiple
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> > > >
> > > > > > >> > For
> > > >
> > > > > > >> > >>> > > Kafka
> > > >
> > > > > > >> > >>> > > > > rest
> > > >
> > > > > > >> > >>> > > > > > > > > proxy,
> > > >
> > > > > > >> > >>> > > > > > > > > >> the
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > renewer
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > can
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be
> the
> > > >
> > > > > creator
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> > > >
> > > > > > >> > the
> > > >
> > > > > > >> > >>> > > token.
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need
> new
> > > acl
> > > > on
> > > >
> > > > > > who
> > > >
> > > > > > >> > can
> > > >
> > > > > > >> > >>> > > request
> > > >
> > > > > > >> > >>> > > > > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > > >> tokens?
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we
> recommend
> > > >
> > > > > people
> > > >
> > > > > > to
> > > >
> > > > > > >> > send
> > > >
> > > > > > >> > >>> > > > > delegation
> > > >
> > > > > > >> > >>> > > > > > > > tokens
> > > >
> > > > > > >> > >>> > > > > > > > > >> in an
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > encrypted
> > > >
> > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > > >
> > > > > > >> > >>>
> > >
> >
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Gwen Shapira <gw...@confluent.io>.
Thinking out loud here:
It looks like authentication with a delegation token is going to be
super-cheap, right? We just compare the token to a value in the broker
cache?
If I understood the KIP correctly, right now it suggests that
authentication happens when establishing the client-broker connection (as
normal for Kafka. But perhaps we want to consider authenticating every
request with delegation token (if exists)?
So a centralized app can create few producers, do the metadata request and
broker discovery with its own user auth, but then use delegation tokens to
allow performing produce/fetch requests as different users? Instead of
having to re-connect for each impersonated user?
This may over-complicate things quite a bit (basically adding extra
information in every request), but maybe it will be useful for
impersonation use-cases (which seem to drive much of the interest in this
KIP)?
Kafka Connect, NiFi and friends can probably use this to share clients
between multiple jobs, tasks, etc.
What do you think?
Gwen
On Tue, Dec 13, 2016 at 12:43 AM, Manikumar <ma...@gmail.com>
wrote:
> Ashish,
>
> Thank you for reviewing the KIP. Please see the replies inline.
>
>
> > 1. How to disable delegation token authentication?
> >
> > This can be achieved in various ways, however I think reusing delegation
> > token secret config for this makes sense here. Avoids creating yet
> another
> > config and forces delegation token users to consciously set the secret.
> If
> > the secret is not set or set to empty string, brokers should turn off
> > delegation token support. This will however require a new error code to
> > indicate delegation token support is turned off on broker.
> >
>
> Thanks for the suggestion. Option to turnoff delegation token
> authentication will be useful.
> I'll update the KIP.
>
>
> >
> > 2. ACLs on delegation token?
> >
> > Do we need to have ACLs defined for tokens? I do not think it buys us
> > anything, as delegation token can be treated as impersonation of the
> owner.
> > Any thing the owner has permission to do, delegation tokens should be
> > allowed to do as well. If so, we probably won't need to return
> > authorization exception error code while creating delegation token. It
> > however would make sense to check renew and expire requests are coming
> from
> > owner or renewers of the token, but that does not require explicit acls.
> >
>
>
> Yes, We agreed to not have new acl on who can request delegation token.
> I'll update the KIP.
>
>
> >
> > 3. How to restrict max life time of a token?
> >
> > Admins might want to restrict max life time of tokens created on a
> cluster,
> > and this can very from cluster to cluster based on use-cases. This might
> > warrant a separate broker config.
> >
> >
> Currently we have "delegation.token.max.lifetime.sec" server config
> property
> May be we can take min(User supplied MaxTime, Server MaxTime) as max life
> time.
> I am open to add new config property.
>
> Few more comments based on recent KIP update.
> >
> > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> > {{ExpireTokenRequest}} with with expiryDate set to anything before
> current
> > date?
> >
>
> makes sense. we don't need special request to cancel the token. We can use
> ExpireTokenRequest.
> I'll update the KIP.
>
>
> > 2. Can we change time field names to indicate their unit is milliseconds,
> > like, IssueDateMs, ExpiryDateMs, etc.?
> >
> >
> Done.
>
>
> > 3. Can we allow users to renew a token for a specified amount of time? In
> > current version of KIP, renew request does not take time as a param, not
> > sure what is expiry time set to after renewal.
> >
> >
> Yes, we need to specify renew period. I'll update the KIP.
>
>
> Thanks,
> Mankumar
>
>
>
> >
> > On Mon, Dec 12, 2016 at 9:08 AM Manikumar <ma...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > I would like to reinitiate the discussion on Delegation token support
> for
> > >
> > > Kafka.
> > >
> > >
> > >
> > > Brief summary of the past discussion:
> > >
> > >
> > >
> > > 1) Broker stores delegation tokens in zookeeper. All brokers will
> have a
> > >
> > > cache backed by
> > >
> > > zookeeper so they will all get notified whenever a new token is
> > >
> > > generated and they will
> > >
> > > update their local cache whenever token state changes.
> > >
> > > 2) The current proposal does not support rotation of secret
> > >
> > > 3) Only allow the renewal by users that authenticated using *non*
> > >
> > > delegation token mechanism
> > >
> > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can
> > >
> > > authenticate using
> > >
> > > SCRAM-SHA-256, providing the delegation token HMAC as password.
> > >
> > >
> > >
> > > Updated the KIP with the following:
> > >
> > > 1. Protocol and Config changes
> > >
> > > 2. format of the data stored in ZK.
> > >
> > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> > >
> > >
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >
> > > 48+Delegation+token+support+for+Kafka
> > >
> > >
> > >
> > >
> > >
> > > Jun, Ashish, Gwen,
> > >
> > >
> > >
> > > Pl review the updated KIP.
> > >
> > >
> > >
> > > Thanks
> > >
> > > Manikumar
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <as...@cloudera.com>
> > wrote:
> > >
> > >
> > >
> > > > Harsha/ Gwen,
> > >
> > > >
> > >
> > > > How do we proceed here? I am willing to help out with here.
> > >
> > > >
> > >
> > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > >
> > > >
> > >
> > > > > Is it updated? are all concerns addressed? do you want to start a
> > vote?
> > >
> > > > >
> > >
> > > > > Sorry for being pushy, I do appreciate that we are all volunteers
> and
> > >
> > > > > finding time is difficult. This feature is important for anything
> > that
> > >
> > > > > integrates with Kafka (stream processors, Flume, NiFi, etc) and I
> > >
> > > > > don't want to see this getting stuck because we lack coordination
> > >
> > > > > within the community.
> > >
> > > > >
> > >
> > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> > kafka@harsha.io>
> > >
> > > > > wrote:
> > >
> > > > > > The only pending update for the KIP is to write up the protocol
> > > changes
> > >
> > > > > like
> > >
> > > > > > we've it KIP-4. I'll update the wiki.
> > >
> > > > > >
> > >
> > > > > >
> > >
> > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <
> asingh@cloudera.com>
> > >
> > > > > wrote:
> > >
> > > > > >>
> > >
> > > > > >> I think we decided to not support secret rotation, I guess this
> > can
> > > be
> > >
> > > > > >> stated clearly on the KIP. Also, more details on how clients
> will
> > >
> > > > > perform
> > >
> > > > > >> token distribution and how CLI will look like will be helpful.
> > >
> > > > > >>
> > >
> > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <
> gwen@confluent.io>
> > >
> > > > > wrote:
> > >
> > > > > >>
> > >
> > > > > >> > Hi Guys,
> > >
> > > > > >> >
> > >
> > > > > >> > This discussion was dead for a while. Are there still
> > contentious
> > >
> > > > > >> > points? If not, why are there no votes?
> > >
> > > > > >> >
> > >
> > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io>
> > > wrote:
> > >
> > > > > >> > > Ashish,
> > >
> > > > > >> > >
> > >
> > > > > >> > > Yes, I will send out a KIP invite for next week to discuss
> > > KIP-48
> > >
> > > > > and
> > >
> > > > > >> > other
> > >
> > > > > >> > > remaining KIPs.
> > >
> > > > > >> > >
> > >
> > > > > >> > > Thanks,
> > >
> > > > > >> > >
> > >
> > > > > >> > > Jun
> > >
> > > > > >> > >
> > >
> > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> > >
> > > > asingh@cloudera.com>
> > >
> > > > > >> > wrote:
> > >
> > > > > >> > >
> > >
> > > > > >> > >> Thanks Harsha!
> > >
> > > > > >> > >>
> > >
> > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also,
> we
> > > did
> > >
> > > > > not
> > >
> > > > > >> > >> actually make a call on when we should have next KIP call.
> As
> > >
> > > > there
> > >
> > > > > >> > >> are
> > >
> > > > > >> > a
> > >
> > > > > >> > >> few outstanding KIPs that could not be discussed this week,
> > can
> > >
> > > > we
> > >
> > > > > >> > >> have
> > >
> > > > > >> > a
> > >
> > > > > >> > >> KIP hangout call next week?
> > >
> > > > > >> > >>
> > >
> > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> > >
> > > > > >> > >> <ka...@harsha.io>
> > >
> > > > > >> > >> wrote:
> > >
> > > > > >> > >>
> > >
> > > > > >> > >>> Ashish,
> > >
> > > > > >> > >>> Yes we are working on it. Lets discuss in the next
> > KIP
> > >
> > > > > >> > >>> meeting.
> > >
> > > > > >> > >>> I'll join.
> > >
> > > > > >> > >>> -Harsha
> > >
> > > > > >> > >>>
> > >
> > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > >
> > > > > asingh@cloudera.com>
> > >
> > > > > >> > >>> wrote:
> > >
> > > > > >> > >>>
> > >
> > > > > >> > >>> > Hello Harsha,
> > >
> > > > > >> > >>> >
> > >
> > > > > >> > >>> > Are you still working on this? Wondering if we can
> discuss
> > >
> > > > this
> > >
> > > > > in
> > >
> > > > > >> > next
> > >
> > > > > >> > >>> KIP
> > >
> > > > > >> > >>> > meeting, if you can join.
> > >
> > > > > >> > >>> >
> > >
> > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> > >
> > > > > >> > kafka@harsha.io>
> > >
> > > > > >> > >>> > wrote:
> > >
> > > > > >> > >>> >
> > >
> > > > > >> > >>> > > Hi Grant,
> > >
> > > > > >> > >>> > > We are working on it. Will add the details
> to
> > > KIP
> > >
> > > > > >> > >>> > > about
> > >
> > > > > >> > the
> > >
> > > > > >> > >>> > > request protocol.
> > >
> > > > > >> > >>> > >
> > >
> > > > > >> > >>> > > Thanks,
> > >
> > > > > >> > >>> > > Harsha
> > >
> > > > > >> > >>> > >
> > >
> > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > >
> > > > > >> > >>> > > <gh...@cloudera.com>
> > >
> > > > > >> > >>> wrote:
> > >
> > > > > >> > >>> > >
> > >
> > > > > >> > >>> > > > Hi Parth,
> > >
> > > > > >> > >>> > > >
> > >
> > > > > >> > >>> > > > Are you still working on this? If you need any help
> > > please
> > >
> > > > > >> > >>> > > > don't
> > >
> > > > > >> > >>> > hesitate
> > >
> > > > > >> > >>> > > > to ask.
> > >
> > > > > >> > >>> > > >
> > >
> > > > > >> > >>> > > > Thanks,
> > >
> > > > > >> > >>> > > > Grant
> > >
> > > > > >> > >>> > > >
> > >
> > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> > >
> > > > jun@confluent.io>
> > >
> > > > > >> > wrote:
> > >
> > > > > >> > >>> > > >
> > >
> > > > > >> > >>> > > > > Parth,
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > Thanks for the reply.
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > It makes sense to only allow the renewal by users
> > that
> > >
> > > > > >> > >>> authenticated
> > >
> > > > > >> > >>> > > > using
> > >
> > > > > >> > >>> > > > > *non* delegation token mechanism. Then, should we
> > make
> > >
> > > > the
> > >
> > > > > >> > >>> renewal a
> > >
> > > > > >> > >>> > > > list?
> > >
> > > > > >> > >>> > > > > For example, in the case of rest proxy, it will be
> > >
> > > > useful
> > >
> > > > > >> > >>> > > > > for
> > >
> > > > > >> > >>> every
> > >
> > > > > >> > >>> > > > > instance of rest proxy to be able to renew the
> > tokens.
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > It would be clearer if we can document the request
> > >
> > > > > protocol
> > >
> > > > > >> > like
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > uence/display/KAFKA/KIP-
> > >
> > > > > >> > >>> > >
> > >
> > > > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> > >
> > > > > operations#KIP-4-
> > >
> > > > > >> > >>> > > Commandlineandcentralizedadministrativeoperations-
> > >
> > > > > >> > >>> > > CreateTopicsRequest(KAFKA-
> 2945):(VotedandPlannedforin0.
> > >
> > > > > 10.1.0)
> > >
> > > > > >> > >>> > > > > .
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > It would also be useful to document the client
> APIs.
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > Thanks,
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > Jun
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt
> <
> > >
> > > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > > > Hi,
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > I am suggesting that we will only allow the
> > renewal
> > > by
> > >
> > > > > >> > >>> > > > > > users
> > >
> > > > > >> > >>> that
> > >
> > > > > >> > >>> > > > > > authenticated using *non* delegation token
> > > mechanism.
> > >
> > > > > For
> > >
> > > > > >> > >>> example,
> > >
> > > > > >> > >>> > If
> > >
> > > > > >> > >>> > > > > user
> > >
> > > > > >> > >>> > > > > > Alice authenticated using kerberos and requested
> > >
> > > > > >> > >>> > > > > > delegation
> > >
> > > > > >> > >>> tokens,
> > >
> > > > > >> > >>> > > > only
> > >
> > > > > >> > >>> > > > > > user Alice authenticated via non delegation
> token
> > >
> > > > > >> > >>> > > > > > mechanism
> > >
> > > > > >> > can
> > >
> > > > > >> > >>> > > renew.
> > >
> > > > > >> > >>> > > > > > Clients that have access to delegation tokens
> can
> > > not
> > >
> > > > > >> > >>> > > > > > issue
> > >
> > > > > >> > >>> > renewal
> > >
> > > > > >> > >>> > > > > > request for renewing their own token and this is
> > >
> > > > > primarily
> > >
> > > > > >> > >>> > important
> > >
> > > > > >> > >>> > > to
> > >
> > > > > >> > >>> > > > > > reduce the time window for which a compromised
> > token
> > >
> > > > > will
> > >
> > > > > >> > >>> > > > > > be
> > >
> > > > > >> > >>> valid.
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > To clarify, Yes any authenticated user can
> request
> > >
> > > > > >> > >>> > > > > > delegation
> > >
> > > > > >> > >>> > tokens
> > >
> > > > > >> > >>> > > > but
> > >
> > > > > >> > >>> > > > > > even here I would recommend to avoid creating a
> > > chain
> > >
> > > > > >> > >>> > > > > > where a
> > >
> > > > > >> > >>> > client
> > >
> > > > > >> > >>> > > > > > authenticated via delegation token request for
> > more
> > >
> > > > > >> > delegation
> > >
> > > > > >> > >>> > > tokens.
> > >
> > > > > >> > >>> > > > > > Basically anyone can request delegation token,
> as
> > > long
> > >
> > > > > as
> > >
> > > > > >> > they
> > >
> > > > > >> > >>> > > > > authenticate
> > >
> > > > > >> > >>> > > > > > via a non delegation token mechanism.
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > Aren't classes listed here
> > >
> > > > > >> > >>> > > > > > <
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > >
> > >
> > > > > >> > >>> > > > https://cwiki.apache.org/confl
> > uence/display/KAFKA/KIP-
> > >
> > > > > >> > >>> > > 48+Delegation+token+support+fo
> > >
> > > > r+Kafka#KIP-48Delegationtokens
> > >
> > > > > >> > >>> upportforKaf
> > >
> > > > > >> > >>> > > ka-PublicInterfaces
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > sufficient?
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > Thanks
> > >
> > > > > >> > >>> > > > > > Parth
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > >
> > > > > >> > >>> > > > > > <ju...@confluent.io>
> > >
> > > > > >> > >>> wrote:
> > >
> > > > > >> > >>> > > > > >
> > >
> > > > > >> > >>> > > > > > > Parth,
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > Thanks for the reply. A couple of comments
> > inline
> > >
> > > > > below.
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> > > brahmbhatt <
> > >
> > > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original
> > >
> > > > > requester
> > >
> > > > > >> > >>> only? or
> > >
> > > > > >> > >>> > > > using
> > >
> > > > > >> > >>> > > > > > > > Kerberos
> > >
> > > > > >> > >>> > > > > > > > auth only?
> > >
> > > > > >> > >>> > > > > > > > My recommendation is to do this only using
> > >
> > > > Kerberos
> > >
> > > > > >> > >>> > > > > > > > auth
> > >
> > > > > >> > and
> > >
> > > > > >> > >>> > only
> > >
> > > > > >> > >>> > > > > threw
> > >
> > > > > >> > >>> > > > > > > the
> > >
> > > > > >> > >>> > > > > > > > renewer specified during the acquisition
> > > request.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you
> saying
> > >
> > > > that
> > >
> > > > > >> > >>> > > > > > > any
> > >
> > > > > >> > >>> client
> > >
> > > > > >> > >>> > > > > > > authenticated with the delegation token can
> > renew,
> > >
> > > > > i.e.
> > >
> > > > > >> > there
> > >
> > > > > >> > >>> is
> > >
> > > > > >> > >>> > no
> > >
> > > > > >> > >>> > > > > > renewer
> > >
> > > > > >> > >>> > > > > > > needed?
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > Also, just to be clear, any authenticated
> client
> > >
> > > > > (either
> > >
> > > > > >> > >>> through
> > >
> > > > > >> > >>> > > SASL
> > >
> > > > > >> > >>> > > > > or
> > >
> > > > > >> > >>> > > > > > > SSL) can request a delegation token for the
> > >
> > > > > >> > >>> > > > > > > authenticated
> > >
> > > > > >> > >>> user,
> > >
> > > > > >> > >>> > > > right?
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in
> ZK?
> > >
> > > > > >> > >>> > > > > > > > My recommendation is still to store in ZK or
> > not
> > >
> > > > > store
> > >
> > > > > >> > them
> > >
> > > > > >> > >>> at
> > >
> > > > > >> > >>> > > all.
> > >
> > > > > >> > >>> > > > > The
> > >
> > > > > >> > >>> > > > > > > > whole controller based distribution is too
> > much
> > >
> > > > > >> > >>> > > > > > > > overhead
> > >
> > > > > >> > >>> with
> > >
> > > > > >> > >>> > not
> > >
> > > > > >> > >>> > > > > much
> > >
> > > > > >> > >>> > > > > > to
> > >
> > > > > >> > >>> > > > > > > > achieve.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> > >
> > > > > >> > >>> > > > > > > > Either by expiration time out or through an
> > >
> > > > explicit
> > >
> > > > > >> > >>> request to
> > >
> > > > > >> > >>> > > > > > > invalidate.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > >
> > > > > >> > >>> > > > > > > > SCRAM
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 5. What is the impersonation proposal (it
> > wasn't
> > >
> > > > in
> > >
> > > > > >> > >>> > > > > > > > the
> > >
> > > > > >> > KIP
> > >
> > > > > >> > >>> but
> > >
> > > > > >> > >>> > > was
> > >
> > > > > >> > >>> > > > > > > > discussed
> > >
> > > > > >> > >>> > > > > > > > in this thread)?
> > >
> > > > > >> > >>> > > > > > > > There is no imperonation proposal. I tried
> and
> > >
> > > > > >> > >>> > > > > > > > explained
> > >
> > > > > >> > how
> > >
> > > > > >> > >>> > its
> > >
> > > > > >> > >>> > > a
> > >
> > > > > >> > >>> > > > > > > > different problem and why its not really
> > > necessary
> > >
> > > > > to
> > >
> > > > > >> > >>> discuss
> > >
> > > > > >> > >>> > > that
> > >
> > > > > >> > >>> > > > as
> > >
> > > > > >> > >>> > > > > > > part
> > >
> > > > > >> > >>> > > > > > > > of this KIP. This KIP will not support any
> > >
> > > > > >> > impersonation,
> > >
> > > > > >> > >>> it
> > >
> > > > > >> > >>> > > will
> > >
> > > > > >> > >>> > > > > just
> > >
> > > > > >> > >>> > > > > > > be
> > >
> > > > > >> > >>> > > > > > > > another way to authenticate.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what
> > > actions?
> > >
> > > > > >> > >>> > > > > > > > We do not need new ACLs.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > Could we document the format of the new
> > >
> > > > > request/response
> > >
> > > > > >> > and
> > >
> > > > > >> > >>> > their
> > >
> > > > > >> > >>> > > > > > > associated Resource and Operation for ACL?
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > 7. How would the delegation token be
> > configured
> > > in
> > >
> > > > > the
> > >
> > > > > >> > >>> client?
> > >
> > > > > >> > >>> > > > > > > > Should be through config. I wasn't planning
> on
> > >
> > > > > >> > >>> > > > > > > > supporting
> > >
> > > > > >> > >>> JAAS
> > >
> > > > > >> > >>> > > for
> > >
> > > > > >> > >>> > > > > > > tokens.
> > >
> > > > > >> > >>> > > > > > > > I don't believe hadoop does this either.
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > Thanks
> > >
> > > > > >> > >>> > > > > > > > Parth
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> > >
> > > > > >> > jun@confluent.io>
> > >
> > > > > >> > >>> > > wrote:
> > >
> > > > > >> > >>> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > Harsha,
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > Another question.
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > 9. How would the delegation token be
> > > configured
> > >
> > > > in
> > >
> > > > > >> > >>> > > > > > > > > the
> > >
> > > > > >> > >>> > client?
> > >
> > > > > >> > >>> > > > The
> > >
> > > > > >> > >>> > > > > > > > standard
> > >
> > > > > >> > >>> > > > > > > > > way is to do this through JAAS. However,
> we
> > > will
> > >
> > > > > >> > >>> > > > > > > > > need
> > >
> > > > > >> > to
> > >
> > > > > >> > >>> > think
> > >
> > > > > >> > >>> > > > > > through
> > >
> > > > > >> > >>> > > > > > > if
> > >
> > > > > >> > >>> > > > > > > > > this is convenient in a shared
> environment.
> > > For
> > >
> > > > > >> > example,
> > >
> > > > > >> > >>> > when a
> > >
> > > > > >> > >>> > > > new
> > >
> > > > > >> > >>> > > > > > > task
> > >
> > > > > >> > >>> > > > > > > > is
> > >
> > > > > >> > >>> > > > > > > > > added to a Storm worker node, do we need
> to
> > >
> > > > > >> > >>> > > > > > > > > dynamically
> > >
> > > > > >> > >>> add a
> > >
> > > > > >> > >>> > > new
> > >
> > > > > >> > >>> > > > > > > section
> > >
> > > > > >> > >>> > > > > > > > > in the JAAS file? It may be more
> convenient
> > if
> > >
> > > > we
> > >
> > > > > >> > >>> > > > > > > > > can
> > >
> > > > > >> > >>> pass in
> > >
> > > > > >> > >>> > > the
> > >
> > > > > >> > >>> > > > > > token
> > >
> > > > > >> > >>> > > > > > > > > through the config directly w/o going
> > through
> > >
> > > > > JAAS.
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > Are you or Parth still actively working on
> > > this
> > >
> > > > > KIP?
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > Thanks,
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > Jun
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> > >
> > > > > >> > >>> jun@confluent.io>
> > >
> > > > > >> > >>> > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > Just to add on that list.
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > 2. It would be good to document the
> format
> > > of
> > >
> > > > > the
> > >
> > > > > >> > data
> > >
> > > > > >> > >>> > stored
> > >
> > > > > >> > >>> > > > in
> > >
> > > > > >> > >>> > > > > > ZK.
> > >
> > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on
> > > whether
> > >
> > > > > the
> > >
> > > > > >> > tokens
> > >
> > > > > >> > >>> > > should
> > >
> > > > > >> > >>> > > > > be
> > >
> > > > > >> > >>> > > > > > > > > > propagated through ZK like
> > config/acl/quota,
> > >
> > > > or
> > >
> > > > > >> > through
> > >
> > > > > >> > >>> the
> > >
> > > > > >> > >>> > > > > > > controller.
> > >
> > > > > >> > >>> > > > > > > > > > Currently, the controller is only
> designed
> > > for
> > >
> > > > > >> > >>> propagating
> > >
> > > > > >> > >>> > > > topic
> > >
> > > > > >> > >>> > > > > > > > > metadata,
> > >
> > > > > >> > >>> > > > > > > > > > but not other data.
> > >
> > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token
> > >
> > > > instead
> > >
> > > > > >> > >>> > > > > > > > > > of
> > >
> > > > > >> > >>> > > DIGEST-MD5
> > >
> > > > > >> > >>> > > > > > since
> > >
> > > > > >> > >>> > > > > > > > it's
> > >
> > > > > >> > >>> > > > > > > > > > deprecated?
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem
> broken.
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > Thanks,
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > Jun
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen
> > >
> > > > Shapira <
> > >
> > > > > >> > >>> > > > > gwen@confluent.io>
> > >
> > > > > >> > >>> > > > > > > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> From what I can see, remaining
> questions
> > > are:
> > >
> > > > > >> > >>> > > > > > > > > >>
> > >
> > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By
> > > original
> > >
> > > > > >> > requester
> > >
> > > > > >> > >>> > only?
> > >
> > > > > >> > >>> > > > or
> > >
> > > > > >> > >>> > > > > > > using
> > >
> > > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> > >
> > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or
> in
> > > ZK?
> > >
> > > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated /
> expired?
> > >
> > > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> > >
> > > > > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal
> (it
> > >
> > > > > wasn't
> > >
> > > > > >> > >>> > > > > > > > > >> in
> > >
> > > > > >> > the
> > >
> > > > > >> > >>> > KIP
> > >
> > > > > >> > >>> > > > but
> > >
> > > > > >> > >>> > > > > > was
> > >
> > > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> > >
> > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for
> what
> > >
> > > > > actions?
> > >
> > > > > >> > >>> > > > > > > > > >>
> > >
> > > > > >> > >>> > > > > > > > > >> Gwen
> > >
> > > > > >> > >>> > > > > > > > > >>
> > >
> > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha
> <
> > >
> > > > > >> > >>> kafka@harsha.io>
> > >
> > > > > >> > >>> > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > >
> > > > > >> > >>> > > > > > > > > >> >
> Unfortunately
> > I
> > >
> > > > > >> > >>> > > > > > > > > >> > couldn't
> > >
> > > > > >> > >>> attend
> > >
> > > > > >> > >>> > > the
> > >
> > > > > >> > >>> > > > > KIP
> > >
> > > > > >> > >>> > > > > > > > > meeting
> > >
> > > > > >> > >>> > > > > > > > > >> > when
> > delegation
> > >
> > > > > tokens
> > >
> > > > > >> > >>> > discussed.
> > >
> > > > > >> > >>> > > > > > > > Appreciate
> > >
> > > > > >> > >>> > > > > > > > > if
> > >
> > > > > >> > >>> > > > > > > > > >> > you can
> update
> > > the
> > >
> > > > > >> > thread if
> > >
> > > > > >> > >>> > you
> > >
> > > > > >> > >>> > > > have
> > >
> > > > > >> > >>> > > > > > any
> > >
> > > > > >> > >>> > > > > > > > > >> > further
> > > questions.
> > >
> > > > > >> > >>> > > > > > > > > >> > Thanks,
> > >
> > > > > >> > >>> > > > > > > > > >> > Harsha
> > >
> > > > > >> > >>> > > > > > > > > >> >
> > >
> > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM,
> > Liquan
> > >
> > > > Pei
> > >
> > > > > >> > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> It seems that the links to images in
> > the
> > >
> > > > KIP
> > >
> > > > > >> > >>> > > > > > > > > >> >> are
> > >
> > > > > >> > >>> > broken.
> > >
> > > > > >> > >>> > > > > > > > > >> >>
> > >
> > > > > >> > >>> > > > > > > > > >> >> Liquan
> > >
> > > > > >> > >>> > > > > > > > > >> >>
> > >
> > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM,
> parth
> > >
> > > > > >> > brahmbhatt <
> > >
> > > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >>
> > >
> > > > > >> > >>> > > > > > > > > >> >> > 110. What does
> getDelegationTokenAs
> > >
> > > > mean?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > In the current proposal we only
> > allow
> > > a
> > >
> > > > > user
> > >
> > > > > >> > >>> > > > > > > > > >> >> > to
> > >
> > > > > >> > >>> get
> > >
> > > > > >> > >>> > > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > > token
> > >
> > > > > >> > >>> > > > > > > > > >> for
> > >
> > > > > >> > >>> > > > > > > > > >> >> > the identity that it authenticated
> > as
> > >
> > > > > using
> > >
> > > > > >> > >>> another
> > >
> > > > > >> > >>> > > > > > mechanism,
> > >
> > > > > >> > >>> > > > > > > > i.e.
> > >
> > > > > >> > >>> > > > > > > > > >> A user
> > >
> > > > > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab
> for
> > >
> > > > > >> > >>> > > > > > > > > >> >> > principal
> > >
> > > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> > >
> > > > > >> > >>> > > > > > > > > >> will get
> > >
> > > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that user
> > only.
> > > In
> > >
> > > > > >> > future I
> > >
> > > > > >> > >>> > think
> > >
> > > > > >> > >>> > > > we
> > >
> > > > > >> > >>> > > > > > will
> > >
> > > > > >> > >>> > > > > > > > > have
> > >
> > > > > >> > >>> > > > > > > > > >> to
> > >
> > > > > >> > >>> > > > > > > > > >> >> > extend support such that we allow
> > some
> > >
> > > > set
> > >
> > > > > >> > >>> > > > > > > > > >> >> > of
> > >
> > > > > >> > >>> users (
> > >
> > > > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > >
> > > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > >
> > > > > >> > >>> > > > to
> > >
> > > > > >> > >>> > > > > > > > acquire
> > >
> > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of
> other
> > >
> > > > users
> > >
> > > > > >> > whose
> > >
> > > > > >> > >>> > > identity
> > >
> > > > > >> > >>> > > > > > they
> > >
> > > > > >> > >>> > > > > > > > have
> > >
> > > > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka
> > brokers
> > >
> > > > > will
> > >
> > > > > >> > have
> > >
> > > > > >> > >>> ACLs
> > >
> > > > > >> > >>> > > to
> > >
> > > > > >> > >>> > > > > > > control
> > >
> > > > > >> > >>> > > > > > > > > >> which
> > >
> > > > > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate
> > other
> > >
> > > > > users
> > >
> > > > > >> > and
> > >
> > > > > >> > >>> get
> > >
> > > > > >> > >>> > > > tokens
> > >
> > > > > >> > >>> > > > > > on
> > >
> > > > > >> > >>> > > > > > > > > >> behalf of
> > >
> > > > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a
> > whole
> > >
> > > > > >> > different
> > >
> > > > > >> > >>> > > problem
> > >
> > > > > >> > >>> > > > in
> > >
> > > > > >> > >>> > > > > > my
> > >
> > > > > >> > >>> > > > > > > > > >> opinion and
> > >
> > > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in
> separate
> > >
> > > > KIP.
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of
> > > getting
> > >
> > > > > and
> > >
> > > > > >> > >>> renewing
> > >
> > > > > >> > >>> > > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > > >> tokens?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > Typically this should be very very
> > > low,
> > >
> > > > 1
> > >
> > > > > >> > request
> > >
> > > > > >> > >>> per
> > >
> > > > > >> > >>> > > > > minute
> > >
> > > > > >> > >>> > > > > > > is a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > relatively high estimate. However
> it
> > >
> > > > > depends
> > >
> > > > > >> > >>> > > > > > > > > >> >> > on
> > >
> > > > > >> > >>> the
> > >
> > > > > >> > >>> > > token
> > >
> > > > > >> > >>> > > > > > > > > >> expiration. I am
> > >
> > > > > >> > >>> > > > > > > > > >> >> > less worried about the extra load
> it
> > >
> > > > puts
> > >
> > > > > on
> > >
> > > > > >> > >>> > controller
> > >
> > > > > >> > >>> > > > vs
> > >
> > > > > >> > >>> > > > > > the
> > >
> > > > > >> > >>> > > > > > > > > added
> > >
> > > > > >> > >>> > > > > > > > > >> >> > complexity and the value it
> offers.
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > Thanks
> > >
> > > > > >> > >>> > > > > > > > > >> >> > Parth
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM,
> > > Ismael
> > >
> > > > > Juma
> > >
> > > > > >> > >>> > > > > > > > > >> >> > <
> > >
> > > > > >> > >>> > > > > > > ismael@juma.me.uk>
> > >
> > > > > >> > >>> > > > > > > > > >> wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably
> > >
> > > > > require a
> > >
> > > > > >> > >>> separate
> > >
> > > > > >> > >>> > > KIP
> > >
> > > > > >> > >>> > > > > as
> > >
> > > > > >> > >>> > > > > > it
> > >
> > > > > >> > >>> > > > > > > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > introduce user visible changes.
> We
> > >
> > > > could
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > also
> > >
> > > > > >> > >>> > update
> > >
> > > > > >> > >>> > > > > KIP-48
> > >
> > > > > >> > >>> > > > > > > to
> > >
> > > > > >> > >>> > > > > > > > > >> have this
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > information, but it seems
> cleaner
> > to
> > >
> > > > do
> > >
> > > > > it
> > >
> > > > > >> > >>> > > separately.
> > >
> > > > > >> > >>> > > > We
> > >
> > > > > >> > >>> > > > > > can
> > >
> > > > > >> > >>> > > > > > > > > >> discuss
> > >
> > > > > >> > >>> > > > > > > > > >> >> > that
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > Ismael
> > >
> > > > > >> > >>> > > > > > > > > >> >> > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM,
> > >
> > > > Rajini
> > >
> > > > > >> > Sivaram
> > >
> > > > > >> > >>> <
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com>
> > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > Ismael,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > >
> > > > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > >
> > > > > >> > jira/browse/KAFKA-3751)
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
> > >
> > > > mechanism.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > Would
> > >
> > > > > >> > >>> that
> > >
> > > > > >> > >>> > > need
> > >
> > > > > >> > >>> > > > > > > another
> > >
> > > > > >> > >>> > > > > > > > > >> KIP? If
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this
> mechanism,
> > > can
> > >
> > > > > this
> > >
> > > > > >> > just
> > >
> > > > > >> > >>> be
> > >
> > > > > >> > >>> > a
> > >
> > > > > >> > >>> > > > JIRA
> > >
> > > > > >> > >>> > > > > > > that
> > >
> > > > > >> > >>> > > > > > > > > gets
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > reviewed
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > Thank you,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > Rajini
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46
> PM,
> > >
> > > > > Ismael
> > >
> > > > > >> > Juma <
> > >
> > > > > >> > >>> > > > > > > > > ismael@juma.me.uk>
> > >
> > > > > >> > >>> > > > > > > > > >> >> > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems
> > like
> > > a
> > >
> > > > > good
> > >
> > > > > >> > >>> > candidate.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently
> > mentioned
> > >
> > > > > this
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > as
> > >
> > > > > >> > a
> > >
> > > > > >> > >>> SASL
> > >
> > > > > >> > >>> > > > > > mechanism
> > >
> > > > > >> > >>> > > > > > > > > that
> > >
> > > > > >> > >>> > > > > > > > > >> might
> > >
> > > > > >> > >>> > > > > > > > > >> >> > be
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have
> > been
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > meaning
> > >
> > > > > >> > to
> > >
> > > > > >> > >>> > check
> > >
> > > > > >> > >>> > > > it
> > >
> > > > > >> > >>> > > > > in
> > >
> > > > > >> > >>> > > > > > > > more
> > >
> > > > > >> > >>> > > > > > > > > >> detail.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > Good
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > to know that you are willing
> > to
> > >
> > > > > >> > contribute
> > >
> > > > > >> > >>> an
> > >
> > > > > >> > >>> > > > > > > > implementation.
> > >
> > > > > >> > >>> > > > > > > > > >> Maybe
> > >
> > > > > >> > >>> > > > > > > > > >> >> > we
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA
> > for
> > >
> > > > > this?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > Ismael
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12
> > PM,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > Rajini
> > >
> > > > > >> > >>> > Sivaram <
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > >
> rajinisivaram@googlemail.com>
> > >
> > > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge
> > > Response
> > >
> > > > > >> > >>> > Authentication
> > >
> > > > > >> > >>> > > > > > > > Mechanism)
> > >
> > > > > >> > >>> > > > > > > > > >> is a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > better
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5.
> > > Java
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> > >
> > > > > >> > >>> come
> > >
> > > > > >> > >>> > > > with a
> > >
> > > > > >> > >>> > > > > > > > > built-in
> > >
> > > > > >> > >>> > > > > > > > > >> SCRAM
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient,
> > but
> > > I
> > >
> > > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > be
> > >
> > > > > >> > >>> happy
> > >
> > > > > >> > >>> > > to
> > >
> > > > > >> > >>> > > > > add
> > >
> > > > > >> > >>> > > > > > > > > support
> > >
> > > > > >> > >>> > > > > > > > > >> in
> > >
> > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > since
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful
> > mechanism
> > >
> > > > to
> > >
> > > > > >> > support
> > >
> > > > > >> > >>> > > anyway.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > https://tools.ietf.org/html/
> > >
> > > > > rfc7677
> > >
> > > > > >> > >>> > describes
> > >
> > > > > >> > >>> > > > the
> > >
> > > > > >> > >>> > > > > > > > protocol
> > >
> > > > > >> > >>> > > > > > > > > >> for
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at
> 2:37
> > > AM,
> > >
> > > > > Jun
> > >
> > > > > >> > Rao <
> > >
> > > > > >> > >>> > > > > > > > jun@confluent.io
> > >
> > > > > >> > >>> > > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the
> > explanation.
> > > A
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > couple
> > >
> > > > > >> > of
> > >
> > > > > >> > >>> > more
> > >
> > > > > >> > >>> > > > > > > questions.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> > >
> > > > > >> > >>> mean?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical
> > rate
> > >
> > > > of
> > >
> > > > > >> > getting
> > >
> > > > > >> > >>> and
> > >
> > > > > >> > >>> > > > > > renewing
> > >
> > > > > >> > >>> > > > > > > > > >> delegation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > tokens?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact
> on
> > >
> > > > > whether
> > >
> > > > > >> > they
> > >
> > > > > >> > >>> > > should
> > >
> > > > > >> > >>> > > > be
> > >
> > > > > >> > >>> > > > > > > > > directed
> > >
> > > > > >> > >>> > > > > > > > > >> to the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at
> > 1:19
> > >
> > > > PM,
> > >
> > > > > >> > parth
> > >
> > > > > >> > >>> > > > > brahmbhatt <
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > brahmbhatt.parth@gmail.com>
> > >
> > > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a
> Cluster
> > >
> > > > > action
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >
> > > > > >> > add
> > >
> > > > > >> > >>> > acls
> > >
> > > > > >> > >>> > > > on
> > >
> > > > > >> > >>> > > > > > who
> > >
> > > > > >> > >>> > > > > > > > can
> > >
> > > > > >> > >>> > > > > > > > > >> request
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see
> the
> > > use
> > >
> > > > > case
> > >
> > > > > >> > for
> > >
> > > > > >> > >>> that
> > >
> > > > > >> > >>> > > yet
> > >
> > > > > >> > >>> > > > > but
> > >
> > > > > >> > >>> > > > > > > > down
> > >
> > > > > >> > >>> > > > > > > > > >> the line
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > when
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > we
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > >
> > > > > >> > getDelegationTokenAs
> > >
> > > > > >> > >>> it
> > >
> > > > > >> > >>> > > will
> > >
> > > > > >> > >>> > > > > be
> > >
> > > > > >> > >>> > > > > > > > > >> necessary.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend
> > tokens
> > > to
> > >
> > > > > be
> > >
> > > > > >> > only
> > >
> > > > > >> > >>> > > > > > > used/distributed
> > >
> > > > > >> > >>> > > > > > > > > >> over
> > >
> > > > > >> > >>> > > > > > > > > >> >> > secure
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what
> > design
> > >
> > > > we
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > end
> > >
> > > > > >> > up
> > >
> > > > > >> > >>> > > choosing
> > >
> > > > > >> > >>> > > > > > > > > >> Invalidation will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > be
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of
> every
> > >
> > > > broker
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > or
> > >
> > > > > >> > >>> > > controller.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
> > >
> > > > > documented
> > >
> > > > > >> > >>> somewhere
> > >
> > > > > >> > >>> > > > that
> > >
> > > > > >> > >>> > > > > > > > > >> invalidation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > directly
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper
> but
> > >
> > > > that
> > >
> > > > > is
> > >
> > > > > >> > not
> > >
> > > > > >> > >>> the
> > >
> > > > > >> > >>> > > > > intent.
> > >
> > > > > >> > >>> > > > > > > > > >> Invalidation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > either
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or
> due
> > to
> > >
> > > > > >> > >>> expiration. No
> > >
> > > > > >> > >>> > > > > direct
> > >
> > > > > >> > >>> > > > > > > > > >> zookeeper
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > interaction
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > from
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores
> > the
> > >
> > > > > >> > >>> DelegationToken
> > >
> > > > > >> > >>> > > > > without
> > >
> > > > > >> > >>> > > > > > > the
> > >
> > > > > >> > >>> > > > > > > > > >> hmac in
> > >
> > > > > >> > >>> > > > > > > > > >> >> > the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry
> > about
> > >
> > > > the
> > >
> > > > > >> > >>> confusion.
> > >
> > > > > >> > >>> > > The
> > >
> > > > > >> > >>> > > > > sole
> > >
> > > > > >> > >>> > > > > > > > > >> purpose of
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > in
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
> > >
> > > > distribution
> > >
> > > > > >> > channel
> > >
> > > > > >> > >>> > for
> > >
> > > > > >> > >>> > > > > tokens
> > >
> > > > > >> > >>> > > > > > > > > >> between all
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > brokers
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > and a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures
> only
> > >
> > > > tokens
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > that
> > >
> > > > > >> > >>> were
> > >
> > > > > >> > >>> > > > > > generated
> > >
> > > > > >> > >>> > > > > > > by
> > >
> > > > > >> > >>> > > > > > > > > >> making a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > request
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > to a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be
> accepted
> > >
> > > > (more
> > >
> > > > > on
> > >
> > > > > >> > this
> > >
> > > > > >> > >>> in
> > >
> > > > > >> > >>> > > > second
> > >
> > > > > >> > >>> > > > > > > > > >> paragraph). The
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > token
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few
> elements
> > >
> > > > > (owner,
> > >
> > > > > >> > >>> renewer,
> > >
> > > > > >> > >>> > > > uuid
> > >
> > > > > >> > >>> > > > > ,
> > >
> > > > > >> > >>> > > > > > > > > >> expiration,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > hmac)
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > ,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > one
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > of
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally
> > >
> > > > generated
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> > >
> > > > > >> > >>> but
> > >
> > > > > >> > >>> > > hmac
> > >
> > > > > >> > >>> > > > it
> > >
> > > > > >> > >>> > > > > > > self
> > >
> > > > > >> > >>> > > > > > > > is
> > >
> > > > > >> > >>> > > > > > > > > >> >> > derivable
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > if
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > you
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other
> > > elements
> > >
> > > > of
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the
> > >
> > > > > >> > >>> token
> > >
> > > > > >> > >>> > +
> > >
> > > > > >> > >>> > > > > secret
> > >
> > > > > >> > >>> > > > > > > key
> > >
> > > > > >> > >>> > > > > > > > > to
> > >
> > > > > >> > >>> > > > > > > > > >> >> > generate
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > hmac.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does
> not
> > >
> > > > > provide
> > >
> > > > > >> > SSL
> > >
> > > > > >> > >>> > > support
> > >
> > > > > >> > >>> > > > we
> > >
> > > > > >> > >>> > > > > > do
> > >
> > > > > >> > >>> > > > > > > > not
> > >
> > > > > >> > >>> > > > > > > > > >> want the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > entire
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> > > transferred
> > >
> > > > > to
> > >
> > > > > >> > >>> zookeeper
> > >
> > > > > >> > >>> > > as
> > >
> > > > > >> > >>> > > > > that
> > >
> > > > > >> > >>> > > > > > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> be an
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > insecure
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > wire
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we
> > only
> > >
> > > > > store
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > all
> > >
> > > > > >> > >>> the
> > >
> > > > > >> > >>> > > other
> > >
> > > > > >> > >>> > > > > > > > elements
> > >
> > > > > >> > >>> > > > > > > > > >> of a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can
> read
> > >
> > > > these
> > >
> > > > > >> > >>> elements
> > >
> > > > > >> > >>> > and
> > >
> > > > > >> > >>> > > > > > because
> > >
> > > > > >> > >>> > > > > > > > > they
> > >
> > > > > >> > >>> > > > > > > > > >> also
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > have
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > access
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they
> will
> > be
> > >
> > > > > able
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >
> > > > > >> > >>> > generate
> > >
> > > > > >> > >>> > > > > hmac
> > >
> > > > > >> > >>> > > > > > on
> > >
> > > > > >> > >>> > > > > > > > > >> their end.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative
> > >
> > > > > proposed
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > is
> > >
> > > > > >> > to
> > >
> > > > > >> > >>> > avoid
> > >
> > > > > >> > >>> > > > > > > zookeeper
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > altogether. A
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > Client
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with
> > >
> > > > required
> > >
> > > > > >> > >>> > information
> > >
> > > > > >> > >>> > > > > > (owner,
> > >
> > > > > >> > >>> > > > > > > > > >> renwer,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > expiration)
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac,
> > >
> > > > uuid).
> > >
> > > > > >> > Broker
> > >
> > > > > >> > >>> > won't
> > >
> > > > > >> > >>> > > > > store
> > >
> > > > > >> > >>> > > > > > > this
> > >
> > > > > >> > >>> > > > > > > > > in
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > zookeeper.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > From
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client
> can
> > >
> > > > > contact
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > >
> > > > > >> > >>> > broker
> > >
> > > > > >> > >>> > > > with
> > >
> > > > > >> > >>> > > > > > all
> > >
> > > > > >> > >>> > > > > > > > the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > delegation
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > token
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
> > >
> > > > > expiration,
> > >
> > > > > >> > hmac,
> > >
> > > > > >> > >>> > > uuid)
> > >
> > > > > >> > >>> > > > > the
> > >
> > > > > >> > >>> > > > > > > > borker
> > >
> > > > > >> > >>> > > > > > > > > >> will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > regenerate
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it
> > >
> > > > matches
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > with
> > >
> > > > > >> > >>> hmac
> > >
> > > > > >> > >>> > > > > > presented
> > >
> > > > > >> > >>> > > > > > > by
> > >
> > > > > >> > >>> > > > > > > > > >> client ,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > broker
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> > >
> > > > > >> > >>> Only
> > >
> > > > > >> > >>> > > > > problem
> > >
> > > > > >> > >>> > > > > > > with
> > >
> > > > > >> > >>> > > > > > > > > >> this
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > approach
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > is
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > if
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
> > >
> > > > compromised
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > >
> > > > > >> > >>> client
> > >
> > > > > >> > >>> > > can
> > >
> > > > > >> > >>> > > > > now
> > >
> > > > > >> > >>> > > > > > > > > generate
> > >
> > > > > >> > >>> > > > > > > > > >> >> > random
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > tokens
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be
> able
> > to
> > >
> > > > > >> > >>> authenticate
> > >
> > > > > >> > >>> > as
> > >
> > > > > >> > >>> > > > any
> > >
> > > > > >> > >>> > > > > > user
> > >
> > > > > >> > >>> > > > > > > > > they
> > >
> > > > > >> > >>> > > > > > > > > >> like.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > with
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee
> > > that
> > >
> > > > > only
> > >
> > > > > >> > >>> tokens
> > >
> > > > > >> > >>> > > > > acquired
> > >
> > > > > >> > >>> > > > > > > via
> > >
> > > > > >> > >>> > > > > > > > a
> > >
> > > > > >> > >>> > > > > > > > > >> broker
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > (using
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > some
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> > >
> > > > > >> > >>> token)
> > >
> > > > > >> > >>> > > will
> > >
> > > > > >> > >>> > > > > be
> > >
> > > > > >> > >>> > > > > > > > > >> accepted. We
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > need
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > to
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal
> > > makes
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > more
> > >
> > > > > >> > >>> sense
> > >
> > > > > >> > >>> > and
> > >
> > > > > >> > >>> > > > we
> > >
> > > > > >> > >>> > > > > > can
> > >
> > > > > >> > >>> > > > > > > go
> > >
> > > > > >> > >>> > > > > > > > > >> over it
> > >
> > > > > >> > >>> > > > > > > > > >> >> > in
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward
> > the
> > >
> > > > > invite
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >
> > > > > >> > >>> me?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016
> at
> > >
> > > > 10:35
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> > >
> > > > > >> > Jun
> > >
> > > > > >> > >>> > Rao <
> > >
> > > > > >> > >>> > > > > > > > > >> jun@confluent.io>
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > wrote:
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP.
> A
> > > few
> > >
> > > > > >> > comments.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This
> potentially
> > > can
> > >
> > > > be
> > >
> > > > > >> > useful
> > >
> > > > > >> > >>> for
> > >
> > > > > >> > >>> > > > Kafka
> > >
> > > > > >> > >>> > > > > > > > Connect
> > >
> > > > > >> > >>> > > > > > > > > >> and
> > >
> > > > > >> > >>> > > > > > > > > >> >> > Kafka
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > rest
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent
> > > will
> > >
> > > > > need
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> > >
> > > > > >> > >>> run a
> > >
> > > > > >> > >>> > > > task
> > >
> > > > > >> > >>> > > > > on
> > >
> > > > > >> > >>> > > > > > > > > behalf
> > >
> > > > > >> > >>> > > > > > > > > >> of a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > client.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > We
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to
> change
> > > how
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> > >
> > > > > >> > >>> > services
> > >
> > > > > >> > >>> > > > use
> > >
> > > > > >> > >>> > > > > > > Kafka
> > >
> > > > > >> > >>> > > > > > > > > >> clients
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer).
> > >
> > > > Instead
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> > >
> > > > > >> > >>> > shared
> > >
> > > > > >> > >>> > > > > client
> > >
> > > > > >> > >>> > > > > > > per
> > >
> > > > > >> > >>> > > > > > > > > >> worker,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > we
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > will
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > need
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > a
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task
> > > since
> > >
> > > > > the
> > >
> > > > > >> > >>> > > > authentication
> > >
> > > > > >> > >>> > > > > > > > happens
> > >
> > > > > >> > >>> > > > > > > > > >> at the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > connection
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka
> > > Connect,
> > >
> > > > > the
> > >
> > > > > >> > >>> renewer
> > >
> > > > > >> > >>> > > will
> > >
> > > > > >> > >>> > > > be
> > >
> > > > > >> > >>> > > > > > the
> > >
> > > > > >> > >>> > > > > > > > > >> workers.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > So,
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > we
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow
> multiple
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> > >
> > > > > >> > For
> > >
> > > > > >> > >>> > > Kafka
> > >
> > > > > >> > >>> > > > > rest
> > >
> > > > > >> > >>> > > > > > > > > proxy,
> > >
> > > > > >> > >>> > > > > > > > > >> the
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > renewer
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > can
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the
> > >
> > > > creator
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> > >
> > > > > >> > the
> > >
> > > > > >> > >>> > > token.
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new
> > acl
> > > on
> > >
> > > > > who
> > >
> > > > > >> > can
> > >
> > > > > >> > >>> > > request
> > >
> > > > > >> > >>> > > > > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > > >> tokens?
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend
> > >
> > > > people
> > >
> > > > > to
> > >
> > > > > >> > send
> > >
> > > > > >> > >>> > > > > delegation
> > >
> > > > > >> > >>> > > > > > > > tokens
> > >
> > > > > >> > >>> > > > > > > > > >> in an
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > encrypted
> > >
> > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > >
> > > > > >> > >>>
> >
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Ashish,
Thank you for reviewing the KIP. Please see the replies inline.
> 1. How to disable delegation token authentication?
>
> This can be achieved in various ways, however I think reusing delegation
> token secret config for this makes sense here. Avoids creating yet another
> config and forces delegation token users to consciously set the secret. If
> the secret is not set or set to empty string, brokers should turn off
> delegation token support. This will however require a new error code to
> indicate delegation token support is turned off on broker.
>
Thanks for the suggestion. Option to turnoff delegation token
authentication will be useful.
I'll update the KIP.
>
> 2. ACLs on delegation token?
>
> Do we need to have ACLs defined for tokens? I do not think it buys us
> anything, as delegation token can be treated as impersonation of the owner.
> Any thing the owner has permission to do, delegation tokens should be
> allowed to do as well. If so, we probably won't need to return
> authorization exception error code while creating delegation token. It
> however would make sense to check renew and expire requests are coming from
> owner or renewers of the token, but that does not require explicit acls.
>
Yes, We agreed to not have new acl on who can request delegation token.
I'll update the KIP.
>
> 3. How to restrict max life time of a token?
>
> Admins might want to restrict max life time of tokens created on a cluster,
> and this can very from cluster to cluster based on use-cases. This might
> warrant a separate broker config.
>
>
Currently we have "delegation.token.max.lifetime.sec" server config
property
May be we can take min(User supplied MaxTime, Server MaxTime) as max life
time.
I am open to add new config property.
Few more comments based on recent KIP update.
>
> 1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
> {{ExpireTokenRequest}} with with expiryDate set to anything before current
> date?
>
makes sense. we don't need special request to cancel the token. We can use
ExpireTokenRequest.
I'll update the KIP.
> 2. Can we change time field names to indicate their unit is milliseconds,
> like, IssueDateMs, ExpiryDateMs, etc.?
>
>
Done.
> 3. Can we allow users to renew a token for a specified amount of time? In
> current version of KIP, renew request does not take time as a param, not
> sure what is expiry time set to after renewal.
>
>
Yes, we need to specify renew period. I'll update the KIP.
Thanks,
Mankumar
>
> On Mon, Dec 12, 2016 at 9:08 AM Manikumar <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> >
> >
> > I would like to reinitiate the discussion on Delegation token support for
> >
> > Kafka.
> >
> >
> >
> > Brief summary of the past discussion:
> >
> >
> >
> > 1) Broker stores delegation tokens in zookeeper. All brokers will have a
> >
> > cache backed by
> >
> > zookeeper so they will all get notified whenever a new token is
> >
> > generated and they will
> >
> > update their local cache whenever token state changes.
> >
> > 2) The current proposal does not support rotation of secret
> >
> > 3) Only allow the renewal by users that authenticated using *non*
> >
> > delegation token mechanism
> >
> > 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can
> >
> > authenticate using
> >
> > SCRAM-SHA-256, providing the delegation token HMAC as password.
> >
> >
> >
> > Updated the KIP with the following:
> >
> > 1. Protocol and Config changes
> >
> > 2. format of the data stored in ZK.
> >
> > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
> >
> >
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >
> > 48+Delegation+token+support+for+Kafka
> >
> >
> >
> >
> >
> > Jun, Ashish, Gwen,
> >
> >
> >
> > Pl review the updated KIP.
> >
> >
> >
> > Thanks
> >
> > Manikumar
> >
> >
> >
> >
> >
> > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <as...@cloudera.com>
> wrote:
> >
> >
> >
> > > Harsha/ Gwen,
> >
> > >
> >
> > > How do we proceed here? I am willing to help out with here.
> >
> > >
> >
> > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> >
> > >
> >
> > > > Is it updated? are all concerns addressed? do you want to start a
> vote?
> >
> > > >
> >
> > > > Sorry for being pushy, I do appreciate that we are all volunteers and
> >
> > > > finding time is difficult. This feature is important for anything
> that
> >
> > > > integrates with Kafka (stream processors, Flume, NiFi, etc) and I
> >
> > > > don't want to see this getting stuck because we lack coordination
> >
> > > > within the community.
> >
> > > >
> >
> > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <
> kafka@harsha.io>
> >
> > > > wrote:
> >
> > > > > The only pending update for the KIP is to write up the protocol
> > changes
> >
> > > > like
> >
> > > > > we've it KIP-4. I'll update the wiki.
> >
> > > > >
> >
> > > > >
> >
> > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com>
> >
> > > > wrote:
> >
> > > > >>
> >
> > > > >> I think we decided to not support secret rotation, I guess this
> can
> > be
> >
> > > > >> stated clearly on the KIP. Also, more details on how clients will
> >
> > > > perform
> >
> > > > >> token distribution and how CLI will look like will be helpful.
> >
> > > > >>
> >
> > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io>
> >
> > > > wrote:
> >
> > > > >>
> >
> > > > >> > Hi Guys,
> >
> > > > >> >
> >
> > > > >> > This discussion was dead for a while. Are there still
> contentious
> >
> > > > >> > points? If not, why are there no votes?
> >
> > > > >> >
> >
> > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> >
> > > > >> > > Ashish,
> >
> > > > >> > >
> >
> > > > >> > > Yes, I will send out a KIP invite for next week to discuss
> > KIP-48
> >
> > > > and
> >
> > > > >> > other
> >
> > > > >> > > remaining KIPs.
> >
> > > > >> > >
> >
> > > > >> > > Thanks,
> >
> > > > >> > >
> >
> > > > >> > > Jun
> >
> > > > >> > >
> >
> > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> >
> > > asingh@cloudera.com>
> >
> > > > >> > wrote:
> >
> > > > >> > >
> >
> > > > >> > >> Thanks Harsha!
> >
> > > > >> > >>
> >
> > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we
> > did
> >
> > > > not
> >
> > > > >> > >> actually make a call on when we should have next KIP call. As
> >
> > > there
> >
> > > > >> > >> are
> >
> > > > >> > a
> >
> > > > >> > >> few outstanding KIPs that could not be discussed this week,
> can
> >
> > > we
> >
> > > > >> > >> have
> >
> > > > >> > a
> >
> > > > >> > >> KIP hangout call next week?
> >
> > > > >> > >>
> >
> > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> >
> > > > >> > >> <ka...@harsha.io>
> >
> > > > >> > >> wrote:
> >
> > > > >> > >>
> >
> > > > >> > >>> Ashish,
> >
> > > > >> > >>> Yes we are working on it. Lets discuss in the next
> KIP
> >
> > > > >> > >>> meeting.
> >
> > > > >> > >>> I'll join.
> >
> > > > >> > >>> -Harsha
> >
> > > > >> > >>>
> >
> > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> >
> > > > asingh@cloudera.com>
> >
> > > > >> > >>> wrote:
> >
> > > > >> > >>>
> >
> > > > >> > >>> > Hello Harsha,
> >
> > > > >> > >>> >
> >
> > > > >> > >>> > Are you still working on this? Wondering if we can discuss
> >
> > > this
> >
> > > > in
> >
> > > > >> > next
> >
> > > > >> > >>> KIP
> >
> > > > >> > >>> > meeting, if you can join.
> >
> > > > >> > >>> >
> >
> > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> >
> > > > >> > kafka@harsha.io>
> >
> > > > >> > >>> > wrote:
> >
> > > > >> > >>> >
> >
> > > > >> > >>> > > Hi Grant,
> >
> > > > >> > >>> > > We are working on it. Will add the details to
> > KIP
> >
> > > > >> > >>> > > about
> >
> > > > >> > the
> >
> > > > >> > >>> > > request protocol.
> >
> > > > >> > >>> > >
> >
> > > > >> > >>> > > Thanks,
> >
> > > > >> > >>> > > Harsha
> >
> > > > >> > >>> > >
> >
> > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> >
> > > > >> > >>> > > <gh...@cloudera.com>
> >
> > > > >> > >>> wrote:
> >
> > > > >> > >>> > >
> >
> > > > >> > >>> > > > Hi Parth,
> >
> > > > >> > >>> > > >
> >
> > > > >> > >>> > > > Are you still working on this? If you need any help
> > please
> >
> > > > >> > >>> > > > don't
> >
> > > > >> > >>> > hesitate
> >
> > > > >> > >>> > > > to ask.
> >
> > > > >> > >>> > > >
> >
> > > > >> > >>> > > > Thanks,
> >
> > > > >> > >>> > > > Grant
> >
> > > > >> > >>> > > >
> >
> > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> >
> > > jun@confluent.io>
> >
> > > > >> > wrote:
> >
> > > > >> > >>> > > >
> >
> > > > >> > >>> > > > > Parth,
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > Thanks for the reply.
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > It makes sense to only allow the renewal by users
> that
> >
> > > > >> > >>> authenticated
> >
> > > > >> > >>> > > > using
> >
> > > > >> > >>> > > > > *non* delegation token mechanism. Then, should we
> make
> >
> > > the
> >
> > > > >> > >>> renewal a
> >
> > > > >> > >>> > > > list?
> >
> > > > >> > >>> > > > > For example, in the case of rest proxy, it will be
> >
> > > useful
> >
> > > > >> > >>> > > > > for
> >
> > > > >> > >>> every
> >
> > > > >> > >>> > > > > instance of rest proxy to be able to renew the
> tokens.
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > It would be clearer if we can document the request
> >
> > > > protocol
> >
> > > > >> > like
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > https://cwiki.apache.org/confl
> uence/display/KAFKA/KIP-
> >
> > > > >> > >>> > >
> >
> > > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> >
> > > > operations#KIP-4-
> >
> > > > >> > >>> > > Commandlineandcentralizedadministrativeoperations-
> >
> > > > >> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.
> >
> > > > 10.1.0)
> >
> > > > >> > >>> > > > > .
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > It would also be useful to document the client APIs.
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > Thanks,
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > Jun
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> >
> > > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > > > Hi,
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > I am suggesting that we will only allow the
> renewal
> > by
> >
> > > > >> > >>> > > > > > users
> >
> > > > >> > >>> that
> >
> > > > >> > >>> > > > > > authenticated using *non* delegation token
> > mechanism.
> >
> > > > For
> >
> > > > >> > >>> example,
> >
> > > > >> > >>> > If
> >
> > > > >> > >>> > > > > user
> >
> > > > >> > >>> > > > > > Alice authenticated using kerberos and requested
> >
> > > > >> > >>> > > > > > delegation
> >
> > > > >> > >>> tokens,
> >
> > > > >> > >>> > > > only
> >
> > > > >> > >>> > > > > > user Alice authenticated via non delegation token
> >
> > > > >> > >>> > > > > > mechanism
> >
> > > > >> > can
> >
> > > > >> > >>> > > renew.
> >
> > > > >> > >>> > > > > > Clients that have access to delegation tokens can
> > not
> >
> > > > >> > >>> > > > > > issue
> >
> > > > >> > >>> > renewal
> >
> > > > >> > >>> > > > > > request for renewing their own token and this is
> >
> > > > primarily
> >
> > > > >> > >>> > important
> >
> > > > >> > >>> > > to
> >
> > > > >> > >>> > > > > > reduce the time window for which a compromised
> token
> >
> > > > will
> >
> > > > >> > >>> > > > > > be
> >
> > > > >> > >>> valid.
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > To clarify, Yes any authenticated user can request
> >
> > > > >> > >>> > > > > > delegation
> >
> > > > >> > >>> > tokens
> >
> > > > >> > >>> > > > but
> >
> > > > >> > >>> > > > > > even here I would recommend to avoid creating a
> > chain
> >
> > > > >> > >>> > > > > > where a
> >
> > > > >> > >>> > client
> >
> > > > >> > >>> > > > > > authenticated via delegation token request for
> more
> >
> > > > >> > delegation
> >
> > > > >> > >>> > > tokens.
> >
> > > > >> > >>> > > > > > Basically anyone can request delegation token, as
> > long
> >
> > > > as
> >
> > > > >> > they
> >
> > > > >> > >>> > > > > authenticate
> >
> > > > >> > >>> > > > > > via a non delegation token mechanism.
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > Aren't classes listed here
> >
> > > > >> > >>> > > > > > <
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > >
> >
> > > > >> > >>> > > > https://cwiki.apache.org/confl
> uence/display/KAFKA/KIP-
> >
> > > > >> > >>> > > 48+Delegation+token+support+fo
> >
> > > r+Kafka#KIP-48Delegationtokens
> >
> > > > >> > >>> upportforKaf
> >
> > > > >> > >>> > > ka-PublicInterfaces
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > sufficient?
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > Thanks
> >
> > > > >> > >>> > > > > > Parth
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> >
> > > > >> > >>> > > > > > <ju...@confluent.io>
> >
> > > > >> > >>> wrote:
> >
> > > > >> > >>> > > > > >
> >
> > > > >> > >>> > > > > > > Parth,
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > Thanks for the reply. A couple of comments
> inline
> >
> > > > below.
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> > brahmbhatt <
> >
> > > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original
> >
> > > > requester
> >
> > > > >> > >>> only? or
> >
> > > > >> > >>> > > > using
> >
> > > > >> > >>> > > > > > > > Kerberos
> >
> > > > >> > >>> > > > > > > > auth only?
> >
> > > > >> > >>> > > > > > > > My recommendation is to do this only using
> >
> > > Kerberos
> >
> > > > >> > >>> > > > > > > > auth
> >
> > > > >> > and
> >
> > > > >> > >>> > only
> >
> > > > >> > >>> > > > > threw
> >
> > > > >> > >>> > > > > > > the
> >
> > > > >> > >>> > > > > > > > renewer specified during the acquisition
> > request.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying
> >
> > > that
> >
> > > > >> > >>> > > > > > > any
> >
> > > > >> > >>> client
> >
> > > > >> > >>> > > > > > > authenticated with the delegation token can
> renew,
> >
> > > > i.e.
> >
> > > > >> > there
> >
> > > > >> > >>> is
> >
> > > > >> > >>> > no
> >
> > > > >> > >>> > > > > > renewer
> >
> > > > >> > >>> > > > > > > needed?
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > Also, just to be clear, any authenticated client
> >
> > > > (either
> >
> > > > >> > >>> through
> >
> > > > >> > >>> > > SASL
> >
> > > > >> > >>> > > > > or
> >
> > > > >> > >>> > > > > > > SSL) can request a delegation token for the
> >
> > > > >> > >>> > > > > > > authenticated
> >
> > > > >> > >>> user,
> >
> > > > >> > >>> > > > right?
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> >
> > > > >> > >>> > > > > > > > My recommendation is still to store in ZK or
> not
> >
> > > > store
> >
> > > > >> > them
> >
> > > > >> > >>> at
> >
> > > > >> > >>> > > all.
> >
> > > > >> > >>> > > > > The
> >
> > > > >> > >>> > > > > > > > whole controller based distribution is too
> much
> >
> > > > >> > >>> > > > > > > > overhead
> >
> > > > >> > >>> with
> >
> > > > >> > >>> > not
> >
> > > > >> > >>> > > > > much
> >
> > > > >> > >>> > > > > > to
> >
> > > > >> > >>> > > > > > > > achieve.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> >
> > > > >> > >>> > > > > > > > Either by expiration time out or through an
> >
> > > explicit
> >
> > > > >> > >>> request to
> >
> > > > >> > >>> > > > > > > invalidate.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> >
> > > > >> > >>> > > > > > > > SCRAM
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > 5. What is the impersonation proposal (it
> wasn't
> >
> > > in
> >
> > > > >> > >>> > > > > > > > the
> >
> > > > >> > KIP
> >
> > > > >> > >>> but
> >
> > > > >> > >>> > > was
> >
> > > > >> > >>> > > > > > > > discussed
> >
> > > > >> > >>> > > > > > > > in this thread)?
> >
> > > > >> > >>> > > > > > > > There is no imperonation proposal. I tried and
> >
> > > > >> > >>> > > > > > > > explained
> >
> > > > >> > how
> >
> > > > >> > >>> > its
> >
> > > > >> > >>> > > a
> >
> > > > >> > >>> > > > > > > > different problem and why its not really
> > necessary
> >
> > > > to
> >
> > > > >> > >>> discuss
> >
> > > > >> > >>> > > that
> >
> > > > >> > >>> > > > as
> >
> > > > >> > >>> > > > > > > part
> >
> > > > >> > >>> > > > > > > > of this KIP. This KIP will not support any
> >
> > > > >> > impersonation,
> >
> > > > >> > >>> it
> >
> > > > >> > >>> > > will
> >
> > > > >> > >>> > > > > just
> >
> > > > >> > >>> > > > > > > be
> >
> > > > >> > >>> > > > > > > > another way to authenticate.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what
> > actions?
> >
> > > > >> > >>> > > > > > > > We do not need new ACLs.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > Could we document the format of the new
> >
> > > > request/response
> >
> > > > >> > and
> >
> > > > >> > >>> > their
> >
> > > > >> > >>> > > > > > > associated Resource and Operation for ACL?
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > >
> >
> > > > >> > >>> > > > > > > > 7. How would the delegation token be
> configured
> > in
> >
> > > > the
> >
> > > > >> > >>> client?
> >
> > > > >> > >>> > > > > > > > Should be through config. I wasn't planning on
> >
> > > > >> > >>> > > > > > > > supporting
> >
> > > > >> > >>> JAAS
> >
> > > > >> > >>> > > for
> >
> > > > >> > >>> > > > > > > tokens.
> >
> > > > >> > >>> > > > > > > > I don't believe hadoop does this either.
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > Thanks
> >
> > > > >> > >>> > > > > > > > Parth
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> >
> > > > >> > jun@confluent.io>
> >
> > > > >> > >>> > > wrote:
> >
> > > > >> > >>> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > Harsha,
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > Another question.
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > 9. How would the delegation token be
> > configured
> >
> > > in
> >
> > > > >> > >>> > > > > > > > > the
> >
> > > > >> > >>> > client?
> >
> > > > >> > >>> > > > The
> >
> > > > >> > >>> > > > > > > > standard
> >
> > > > >> > >>> > > > > > > > > way is to do this through JAAS. However, we
> > will
> >
> > > > >> > >>> > > > > > > > > need
> >
> > > > >> > to
> >
> > > > >> > >>> > think
> >
> > > > >> > >>> > > > > > through
> >
> > > > >> > >>> > > > > > > if
> >
> > > > >> > >>> > > > > > > > > this is convenient in a shared environment.
> > For
> >
> > > > >> > example,
> >
> > > > >> > >>> > when a
> >
> > > > >> > >>> > > > new
> >
> > > > >> > >>> > > > > > > task
> >
> > > > >> > >>> > > > > > > > is
> >
> > > > >> > >>> > > > > > > > > added to a Storm worker node, do we need to
> >
> > > > >> > >>> > > > > > > > > dynamically
> >
> > > > >> > >>> add a
> >
> > > > >> > >>> > > new
> >
> > > > >> > >>> > > > > > > section
> >
> > > > >> > >>> > > > > > > > > in the JAAS file? It may be more convenient
> if
> >
> > > we
> >
> > > > >> > >>> > > > > > > > > can
> >
> > > > >> > >>> pass in
> >
> > > > >> > >>> > > the
> >
> > > > >> > >>> > > > > > token
> >
> > > > >> > >>> > > > > > > > > through the config directly w/o going
> through
> >
> > > > JAAS.
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > Are you or Parth still actively working on
> > this
> >
> > > > KIP?
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > Thanks,
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > Jun
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> >
> > > > >> > >>> jun@confluent.io>
> >
> > > > >> > >>> > > > wrote:
> >
> > > > >> > >>> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > Just to add on that list.
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > 2. It would be good to document the format
> > of
> >
> > > > the
> >
> > > > >> > data
> >
> > > > >> > >>> > stored
> >
> > > > >> > >>> > > > in
> >
> > > > >> > >>> > > > > > ZK.
> >
> > > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on
> > whether
> >
> > > > the
> >
> > > > >> > tokens
> >
> > > > >> > >>> > > should
> >
> > > > >> > >>> > > > > be
> >
> > > > >> > >>> > > > > > > > > > propagated through ZK like
> config/acl/quota,
> >
> > > or
> >
> > > > >> > through
> >
> > > > >> > >>> the
> >
> > > > >> > >>> > > > > > > controller.
> >
> > > > >> > >>> > > > > > > > > > Currently, the controller is only designed
> > for
> >
> > > > >> > >>> propagating
> >
> > > > >> > >>> > > > topic
> >
> > > > >> > >>> > > > > > > > > metadata,
> >
> > > > >> > >>> > > > > > > > > > but not other data.
> >
> > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token
> >
> > > instead
> >
> > > > >> > >>> > > > > > > > > > of
> >
> > > > >> > >>> > > DIGEST-MD5
> >
> > > > >> > >>> > > > > > since
> >
> > > > >> > >>> > > > > > > > it's
> >
> > > > >> > >>> > > > > > > > > > deprecated?
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > Thanks,
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > Jun
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen
> >
> > > Shapira <
> >
> > > > >> > >>> > > > > gwen@confluent.io>
> >
> > > > >> > >>> > > > > > > > > wrote:
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> From what I can see, remaining questions
> > are:
> >
> > > > >> > >>> > > > > > > > > >>
> >
> > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By
> > original
> >
> > > > >> > requester
> >
> > > > >> > >>> > only?
> >
> > > > >> > >>> > > > or
> >
> > > > >> > >>> > > > > > > using
> >
> > > > >> > >>> > > > > > > > > >> Kerberos auth only?
> >
> > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in
> > ZK?
> >
> > > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> >
> > > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> >
> > > > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it
> >
> > > > wasn't
> >
> > > > >> > >>> > > > > > > > > >> in
> >
> > > > >> > the
> >
> > > > >> > >>> > KIP
> >
> > > > >> > >>> > > > but
> >
> > > > >> > >>> > > > > > was
> >
> > > > >> > >>> > > > > > > > > >> discussed in this thread)?
> >
> > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what
> >
> > > > actions?
> >
> > > > >> > >>> > > > > > > > > >>
> >
> > > > >> > >>> > > > > > > > > >> Gwen
> >
> > > > >> > >>> > > > > > > > > >>
> >
> > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> >
> > > > >> > >>> kafka@harsha.io>
> >
> > > > >> > >>> > > > wrote:
> >
> > > > >> > >>> > > > > > > > > >> > Jun & Ismael,
> >
> > > > >> > >>> > > > > > > > > >> > Unfortunately
> I
> >
> > > > >> > >>> > > > > > > > > >> > couldn't
> >
> > > > >> > >>> attend
> >
> > > > >> > >>> > > the
> >
> > > > >> > >>> > > > > KIP
> >
> > > > >> > >>> > > > > > > > > meeting
> >
> > > > >> > >>> > > > > > > > > >> > when
> delegation
> >
> > > > tokens
> >
> > > > >> > >>> > discussed.
> >
> > > > >> > >>> > > > > > > > Appreciate
> >
> > > > >> > >>> > > > > > > > > if
> >
> > > > >> > >>> > > > > > > > > >> > you can update
> > the
> >
> > > > >> > thread if
> >
> > > > >> > >>> > you
> >
> > > > >> > >>> > > > have
> >
> > > > >> > >>> > > > > > any
> >
> > > > >> > >>> > > > > > > > > >> > further
> > questions.
> >
> > > > >> > >>> > > > > > > > > >> > Thanks,
> >
> > > > >> > >>> > > > > > > > > >> > Harsha
> >
> > > > >> > >>> > > > > > > > > >> >
> >
> > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM,
> Liquan
> >
> > > Pei
> >
> > > > >> > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> It seems that the links to images in
> the
> >
> > > KIP
> >
> > > > >> > >>> > > > > > > > > >> >> are
> >
> > > > >> > >>> > broken.
> >
> > > > >> > >>> > > > > > > > > >> >>
> >
> > > > >> > >>> > > > > > > > > >> >> Liquan
> >
> > > > >> > >>> > > > > > > > > >> >>
> >
> > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> >
> > > > >> > brahmbhatt <
> >
> > > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> >
> > > > >> > >>> > > > > > > > > >> >>
> >
> > > > >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs
> >
> > > mean?
> >
> > > > >> > >>> > > > > > > > > >> >> > In the current proposal we only
> allow
> > a
> >
> > > > user
> >
> > > > >> > >>> > > > > > > > > >> >> > to
> >
> > > > >> > >>> get
> >
> > > > >> > >>> > > > > > delegation
> >
> > > > >> > >>> > > > > > > > > token
> >
> > > > >> > >>> > > > > > > > > >> for
> >
> > > > >> > >>> > > > > > > > > >> >> > the identity that it authenticated
> as
> >
> > > > using
> >
> > > > >> > >>> another
> >
> > > > >> > >>> > > > > > mechanism,
> >
> > > > >> > >>> > > > > > > > i.e.
> >
> > > > >> > >>> > > > > > > > > >> A user
> >
> > > > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
> >
> > > > >> > >>> > > > > > > > > >> >> > principal
> >
> > > > >> > >>> > > > > > > user1@EXAMPLE.COM
> >
> > > > >> > >>> > > > > > > > > >> will get
> >
> > > > >> > >>> > > > > > > > > >> >> > delegation tokens for that user
> only.
> > In
> >
> > > > >> > future I
> >
> > > > >> > >>> > think
> >
> > > > >> > >>> > > > we
> >
> > > > >> > >>> > > > > > will
> >
> > > > >> > >>> > > > > > > > > have
> >
> > > > >> > >>> > > > > > > > > >> to
> >
> > > > >> > >>> > > > > > > > > >> >> > extend support such that we allow
> some
> >
> > > set
> >
> > > > >> > >>> > > > > > > > > >> >> > of
> >
> > > > >> > >>> users (
> >
> > > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> >
> > > > >> > >>> > storm-nimbus@EXAMPLE.COM)
> >
> > > > >> > >>> > > > to
> >
> > > > >> > >>> > > > > > > > acquire
> >
> > > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other
> >
> > > users
> >
> > > > >> > whose
> >
> > > > >> > >>> > > identity
> >
> > > > >> > >>> > > > > > they
> >
> > > > >> > >>> > > > > > > > have
> >
> > > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka
> brokers
> >
> > > > will
> >
> > > > >> > have
> >
> > > > >> > >>> ACLs
> >
> > > > >> > >>> > > to
> >
> > > > >> > >>> > > > > > > control
> >
> > > > >> > >>> > > > > > > > > >> which
> >
> > > > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate
> other
> >
> > > > users
> >
> > > > >> > and
> >
> > > > >> > >>> get
> >
> > > > >> > >>> > > > tokens
> >
> > > > >> > >>> > > > > > on
> >
> > > > >> > >>> > > > > > > > > >> behalf of
> >
> > > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a
> whole
> >
> > > > >> > different
> >
> > > > >> > >>> > > problem
> >
> > > > >> > >>> > > > in
> >
> > > > >> > >>> > > > > > my
> >
> > > > >> > >>> > > > > > > > > >> opinion and
> >
> > > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in separate
> >
> > > KIP.
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of
> > getting
> >
> > > > and
> >
> > > > >> > >>> renewing
> >
> > > > >> > >>> > > > > > delegation
> >
> > > > >> > >>> > > > > > > > > >> tokens?
> >
> > > > >> > >>> > > > > > > > > >> >> > Typically this should be very very
> > low,
> >
> > > 1
> >
> > > > >> > request
> >
> > > > >> > >>> per
> >
> > > > >> > >>> > > > > minute
> >
> > > > >> > >>> > > > > > > is a
> >
> > > > >> > >>> > > > > > > > > >> >> > relatively high estimate. However it
> >
> > > > depends
> >
> > > > >> > >>> > > > > > > > > >> >> > on
> >
> > > > >> > >>> the
> >
> > > > >> > >>> > > token
> >
> > > > >> > >>> > > > > > > > > >> expiration. I am
> >
> > > > >> > >>> > > > > > > > > >> >> > less worried about the extra load it
> >
> > > puts
> >
> > > > on
> >
> > > > >> > >>> > controller
> >
> > > > >> > >>> > > > vs
> >
> > > > >> > >>> > > > > > the
> >
> > > > >> > >>> > > > > > > > > added
> >
> > > > >> > >>> > > > > > > > > >> >> > complexity and the value it offers.
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> > Thanks
> >
> > > > >> > >>> > > > > > > > > >> >> > Parth
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM,
> > Ismael
> >
> > > > Juma
> >
> > > > >> > >>> > > > > > > > > >> >> > <
> >
> > > > >> > >>> > > > > > > ismael@juma.me.uk>
> >
> > > > >> > >>> > > > > > > > > >> wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> >
> >
> > > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably
> >
> > > > require a
> >
> > > > >> > >>> separate
> >
> > > > >> > >>> > > KIP
> >
> > > > >> > >>> > > > > as
> >
> > > > >> > >>> > > > > > it
> >
> > > > >> > >>> > > > > > > > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > introduce user visible changes. We
> >
> > > could
> >
> > > > >> > >>> > > > > > > > > >> >> > > also
> >
> > > > >> > >>> > update
> >
> > > > >> > >>> > > > > KIP-48
> >
> > > > >> > >>> > > > > > > to
> >
> > > > >> > >>> > > > > > > > > >> have this
> >
> > > > >> > >>> > > > > > > > > >> >> > > information, but it seems cleaner
> to
> >
> > > do
> >
> > > > it
> >
> > > > >> > >>> > > separately.
> >
> > > > >> > >>> > > > We
> >
> > > > >> > >>> > > > > > can
> >
> > > > >> > >>> > > > > > > > > >> discuss
> >
> > > > >> > >>> > > > > > > > > >> >> > that
> >
> > > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> >
> > > > >> > >>> > > > > > > > > >> >> > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > Ismael
> >
> > > > >> > >>> > > > > > > > > >> >> > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM,
> >
> > > Rajini
> >
> > > > >> > Sivaram
> >
> > > > >> > >>> <
> >
> > > > >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com>
> > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > Ismael,
> >
> > > > >> > >>> > > > > > > > > >> >> > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> >
> > > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> >
> > > > >> > jira/browse/KAFKA-3751)
> >
> > > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
> >
> > > mechanism.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > Would
> >
> > > > >> > >>> that
> >
> > > > >> > >>> > > need
> >
> > > > >> > >>> > > > > > > another
> >
> > > > >> > >>> > > > > > > > > >> KIP? If
> >
> > > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism,
> > can
> >
> > > > this
> >
> > > > >> > just
> >
> > > > >> > >>> be
> >
> > > > >> > >>> > a
> >
> > > > >> > >>> > > > JIRA
> >
> > > > >> > >>> > > > > > > that
> >
> > > > >> > >>> > > > > > > > > gets
> >
> > > > >> > >>> > > > > > > > > >> >> > > reviewed
> >
> > > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> >
> > > > >> > >>> > > > > > > > > >> >> > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > Thank you,
> >
> > > > >> > >>> > > > > > > > > >> >> > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > Rajini
> >
> > > > >> > >>> > > > > > > > > >> >> > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM,
> >
> > > > Ismael
> >
> > > > >> > Juma <
> >
> > > > >> > >>> > > > > > > > > ismael@juma.me.uk>
> >
> > > > >> > >>> > > > > > > > > >> >> > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems
> like
> > a
> >
> > > > good
> >
> > > > >> > >>> > candidate.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently
> mentioned
> >
> > > > this
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > as
> >
> > > > >> > a
> >
> > > > >> > >>> SASL
> >
> > > > >> > >>> > > > > > mechanism
> >
> > > > >> > >>> > > > > > > > > that
> >
> > > > >> > >>> > > > > > > > > >> might
> >
> > > > >> > >>> > > > > > > > > >> >> > be
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have
> been
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > meaning
> >
> > > > >> > to
> >
> > > > >> > >>> > check
> >
> > > > >> > >>> > > > it
> >
> > > > >> > >>> > > > > in
> >
> > > > >> > >>> > > > > > > > more
> >
> > > > >> > >>> > > > > > > > > >> detail.
> >
> > > > >> > >>> > > > > > > > > >> >> > > Good
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > to know that you are willing
> to
> >
> > > > >> > contribute
> >
> > > > >> > >>> an
> >
> > > > >> > >>> > > > > > > > implementation.
> >
> > > > >> > >>> > > > > > > > > >> Maybe
> >
> > > > >> > >>> > > > > > > > > >> >> > we
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA
> for
> >
> > > > this?
> >
> > > > >> > >>> > > > > > > > > >> >> > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > Ismael
> >
> > > > >> > >>> > > > > > > > > >> >> > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12
> PM,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > Rajini
> >
> > > > >> > >>> > Sivaram <
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com>
> >
> > > > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge
> > Response
> >
> > > > >> > >>> > Authentication
> >
> > > > >> > >>> > > > > > > > Mechanism)
> >
> > > > >> > >>> > > > > > > > > >> is a
> >
> > > > >> > >>> > > > > > > > > >> >> > > better
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5.
> > Java
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> >
> > > > >> > >>> come
> >
> > > > >> > >>> > > > with a
> >
> > > > >> > >>> > > > > > > > > built-in
> >
> > > > >> > >>> > > > > > > > > >> SCRAM
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient,
> but
> > I
> >
> > > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > be
> >
> > > > >> > >>> happy
> >
> > > > >> > >>> > > to
> >
> > > > >> > >>> > > > > add
> >
> > > > >> > >>> > > > > > > > > support
> >
> > > > >> > >>> > > > > > > > > >> in
> >
> > > > >> > >>> > > > > > > > > >> >> > Kafka
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > since
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful
> mechanism
> >
> > > to
> >
> > > > >> > support
> >
> > > > >> > >>> > > anyway.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > >
> https://tools.ietf.org/html/
> >
> > > > rfc7677
> >
> > > > >> > >>> > describes
> >
> > > > >> > >>> > > > the
> >
> > > > >> > >>> > > > > > > > protocol
> >
> > > > >> > >>> > > > > > > > > >> for
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37
> > AM,
> >
> > > > Jun
> >
> > > > >> > Rao <
> >
> > > > >> > >>> > > > > > > > jun@confluent.io
> >
> > > > >> > >>> > > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the
> explanation.
> > A
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > couple
> >
> > > > >> > of
> >
> > > > >> > >>> > more
> >
> > > > >> > >>> > > > > > > questions.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> >
> > > > >> > >>> mean?
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical
> rate
> >
> > > of
> >
> > > > >> > getting
> >
> > > > >> > >>> and
> >
> > > > >> > >>> > > > > > renewing
> >
> > > > >> > >>> > > > > > > > > >> delegation
> >
> > > > >> > >>> > > > > > > > > >> >> > > > tokens?
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on
> >
> > > > whether
> >
> > > > >> > they
> >
> > > > >> > >>> > > should
> >
> > > > >> > >>> > > > be
> >
> > > > >> > >>> > > > > > > > > directed
> >
> > > > >> > >>> > > > > > > > > >> to the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at
> 1:19
> >
> > > PM,
> >
> > > > >> > parth
> >
> > > > >> > >>> > > > > brahmbhatt <
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> brahmbhatt.parth@gmail.com>
> >
> > > > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster
> >
> > > > action
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> >
> > > > >> > add
> >
> > > > >> > >>> > acls
> >
> > > > >> > >>> > > > on
> >
> > > > >> > >>> > > > > > who
> >
> > > > >> > >>> > > > > > > > can
> >
> > > > >> > >>> > > > > > > > > >> request
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > delegation
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the
> > use
> >
> > > > case
> >
> > > > >> > for
> >
> > > > >> > >>> that
> >
> > > > >> > >>> > > yet
> >
> > > > >> > >>> > > > > but
> >
> > > > >> > >>> > > > > > > > down
> >
> > > > >> > >>> > > > > > > > > >> the line
> >
> > > > >> > >>> > > > > > > > > >> >> > > > when
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > we
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> >
> > > > >> > getDelegationTokenAs
> >
> > > > >> > >>> it
> >
> > > > >> > >>> > > will
> >
> > > > >> > >>> > > > > be
> >
> > > > >> > >>> > > > > > > > > >> necessary.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend
> tokens
> > to
> >
> > > > be
> >
> > > > >> > only
> >
> > > > >> > >>> > > > > > > used/distributed
> >
> > > > >> > >>> > > > > > > > > >> over
> >
> > > > >> > >>> > > > > > > > > >> >> > secure
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what
> design
> >
> > > we
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > end
> >
> > > > >> > up
> >
> > > > >> > >>> > > choosing
> >
> > > > >> > >>> > > > > > > > > >> Invalidation will
> >
> > > > >> > >>> > > > > > > > > >> >> > > be
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every
> >
> > > broker
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > or
> >
> > > > >> > >>> > > controller.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
> >
> > > > documented
> >
> > > > >> > >>> somewhere
> >
> > > > >> > >>> > > > that
> >
> > > > >> > >>> > > > > > > > > >> invalidation
> >
> > > > >> > >>> > > > > > > > > >> >> > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > directly
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but
> >
> > > that
> >
> > > > is
> >
> > > > >> > not
> >
> > > > >> > >>> the
> >
> > > > >> > >>> > > > > intent.
> >
> > > > >> > >>> > > > > > > > > >> Invalidation
> >
> > > > >> > >>> > > > > > > > > >> >> > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > either
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or due
> to
> >
> > > > >> > >>> expiration. No
> >
> > > > >> > >>> > > > > direct
> >
> > > > >> > >>> > > > > > > > > >> zookeeper
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > interaction
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > from
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores
> the
> >
> > > > >> > >>> DelegationToken
> >
> > > > >> > >>> > > > > without
> >
> > > > >> > >>> > > > > > > the
> >
> > > > >> > >>> > > > > > > > > >> hmac in
> >
> > > > >> > >>> > > > > > > > > >> >> > the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry
> about
> >
> > > the
> >
> > > > >> > >>> confusion.
> >
> > > > >> > >>> > > The
> >
> > > > >> > >>> > > > > sole
> >
> > > > >> > >>> > > > > > > > > >> purpose of
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > in
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
> >
> > > distribution
> >
> > > > >> > channel
> >
> > > > >> > >>> > for
> >
> > > > >> > >>> > > > > tokens
> >
> > > > >> > >>> > > > > > > > > >> between all
> >
> > > > >> > >>> > > > > > > > > >> >> > > > brokers
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > and a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only
> >
> > > tokens
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > that
> >
> > > > >> > >>> were
> >
> > > > >> > >>> > > > > > generated
> >
> > > > >> > >>> > > > > > > by
> >
> > > > >> > >>> > > > > > > > > >> making a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > request
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > to a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted
> >
> > > (more
> >
> > > > on
> >
> > > > >> > this
> >
> > > > >> > >>> in
> >
> > > > >> > >>> > > > second
> >
> > > > >> > >>> > > > > > > > > >> paragraph). The
> >
> > > > >> > >>> > > > > > > > > >> >> > > > token
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements
> >
> > > > (owner,
> >
> > > > >> > >>> renewer,
> >
> > > > >> > >>> > > > uuid
> >
> > > > >> > >>> > > > > ,
> >
> > > > >> > >>> > > > > > > > > >> expiration,
> >
> > > > >> > >>> > > > > > > > > >> >> > > hmac)
> >
> > > > >> > >>> > > > > > > > > >> >> > > > ,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > one
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > of
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally
> >
> > > generated
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> >
> > > > >> > >>> but
> >
> > > > >> > >>> > > hmac
> >
> > > > >> > >>> > > > it
> >
> > > > >> > >>> > > > > > > self
> >
> > > > >> > >>> > > > > > > > is
> >
> > > > >> > >>> > > > > > > > > >> >> > derivable
> >
> > > > >> > >>> > > > > > > > > >> >> > > > if
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > you
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other
> > elements
> >
> > > of
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > the
> >
> > > > >> > >>> token
> >
> > > > >> > >>> > +
> >
> > > > >> > >>> > > > > secret
> >
> > > > >> > >>> > > > > > > key
> >
> > > > >> > >>> > > > > > > > > to
> >
> > > > >> > >>> > > > > > > > > >> >> > generate
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > hmac.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not
> >
> > > > provide
> >
> > > > >> > SSL
> >
> > > > >> > >>> > > support
> >
> > > > >> > >>> > > > we
> >
> > > > >> > >>> > > > > > do
> >
> > > > >> > >>> > > > > > > > not
> >
> > > > >> > >>> > > > > > > > > >> want the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > entire
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> > transferred
> >
> > > > to
> >
> > > > >> > >>> zookeeper
> >
> > > > >> > >>> > > as
> >
> > > > >> > >>> > > > > that
> >
> > > > >> > >>> > > > > > > > will
> >
> > > > >> > >>> > > > > > > > > >> be an
> >
> > > > >> > >>> > > > > > > > > >> >> > > > insecure
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > wire
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we
> only
> >
> > > > store
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > all
> >
> > > > >> > >>> the
> >
> > > > >> > >>> > > other
> >
> > > > >> > >>> > > > > > > > elements
> >
> > > > >> > >>> > > > > > > > > >> of a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > delegation
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read
> >
> > > these
> >
> > > > >> > >>> elements
> >
> > > > >> > >>> > and
> >
> > > > >> > >>> > > > > > because
> >
> > > > >> > >>> > > > > > > > > they
> >
> > > > >> > >>> > > > > > > > > >> also
> >
> > > > >> > >>> > > > > > > > > >> >> > > have
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > access
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will
> be
> >
> > > > able
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> >
> > > > >> > >>> > generate
> >
> > > > >> > >>> > > > > hmac
> >
> > > > >> > >>> > > > > > on
> >
> > > > >> > >>> > > > > > > > > >> their end.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative
> >
> > > > proposed
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > is
> >
> > > > >> > to
> >
> > > > >> > >>> > avoid
> >
> > > > >> > >>> > > > > > > zookeeper
> >
> > > > >> > >>> > > > > > > > > >> >> > > altogether. A
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > Client
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with
> >
> > > required
> >
> > > > >> > >>> > information
> >
> > > > >> > >>> > > > > > (owner,
> >
> > > > >> > >>> > > > > > > > > >> renwer,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > expiration)
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac,
> >
> > > uuid).
> >
> > > > >> > Broker
> >
> > > > >> > >>> > won't
> >
> > > > >> > >>> > > > > store
> >
> > > > >> > >>> > > > > > > this
> >
> > > > >> > >>> > > > > > > > > in
> >
> > > > >> > >>> > > > > > > > > >> >> > > zookeeper.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > From
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client can
> >
> > > > contact
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> >
> > > > >> > >>> > broker
> >
> > > > >> > >>> > > > with
> >
> > > > >> > >>> > > > > > all
> >
> > > > >> > >>> > > > > > > > the
> >
> > > > >> > >>> > > > > > > > > >> >> > > delegation
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > token
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
> >
> > > > expiration,
> >
> > > > >> > hmac,
> >
> > > > >> > >>> > > uuid)
> >
> > > > >> > >>> > > > > the
> >
> > > > >> > >>> > > > > > > > borker
> >
> > > > >> > >>> > > > > > > > > >> will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > regenerate
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it
> >
> > > matches
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > with
> >
> > > > >> > >>> hmac
> >
> > > > >> > >>> > > > > > presented
> >
> > > > >> > >>> > > > > > > by
> >
> > > > >> > >>> > > > > > > > > >> client ,
> >
> > > > >> > >>> > > > > > > > > >> >> > > > broker
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> >
> > > > >> > >>> Only
> >
> > > > >> > >>> > > > > problem
> >
> > > > >> > >>> > > > > > > with
> >
> > > > >> > >>> > > > > > > > > >> this
> >
> > > > >> > >>> > > > > > > > > >> >> > > approach
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > is
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > if
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
> >
> > > compromised
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > any
> >
> > > > >> > >>> client
> >
> > > > >> > >>> > > can
> >
> > > > >> > >>> > > > > now
> >
> > > > >> > >>> > > > > > > > > generate
> >
> > > > >> > >>> > > > > > > > > >> >> > random
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > tokens
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > and
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be able
> to
> >
> > > > >> > >>> authenticate
> >
> > > > >> > >>> > as
> >
> > > > >> > >>> > > > any
> >
> > > > >> > >>> > > > > > user
> >
> > > > >> > >>> > > > > > > > > they
> >
> > > > >> > >>> > > > > > > > > >> like.
> >
> > > > >> > >>> > > > > > > > > >> >> > > with
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee
> > that
> >
> > > > only
> >
> > > > >> > >>> tokens
> >
> > > > >> > >>> > > > > acquired
> >
> > > > >> > >>> > > > > > > via
> >
> > > > >> > >>> > > > > > > > a
> >
> > > > >> > >>> > > > > > > > > >> broker
> >
> > > > >> > >>> > > > > > > > > >> >> > > > (using
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > some
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> >
> > > > >> > >>> token)
> >
> > > > >> > >>> > > will
> >
> > > > >> > >>> > > > > be
> >
> > > > >> > >>> > > > > > > > > >> accepted. We
> >
> > > > >> > >>> > > > > > > > > >> >> > > need
> >
> > > > >> > >>> > > > > > > > > >> >> > > > to
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal
> > makes
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > more
> >
> > > > >> > >>> sense
> >
> > > > >> > >>> > and
> >
> > > > >> > >>> > > > we
> >
> > > > >> > >>> > > > > > can
> >
> > > > >> > >>> > > > > > > go
> >
> > > > >> > >>> > > > > > > > > >> over it
> >
> > > > >> > >>> > > > > > > > > >> >> > in
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward
> the
> >
> > > > invite
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > to
> >
> > > > >> > >>> me?
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at
> >
> > > 10:35
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> >
> > > > >> > Jun
> >
> > > > >> > >>> > Rao <
> >
> > > > >> > >>> > > > > > > > > >> jun@confluent.io>
> >
> > > > >> > >>> > > > > > > > > >> >> > > > wrote:
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A
> > few
> >
> > > > >> > comments.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially
> > can
> >
> > > be
> >
> > > > >> > useful
> >
> > > > >> > >>> for
> >
> > > > >> > >>> > > > Kafka
> >
> > > > >> > >>> > > > > > > > Connect
> >
> > > > >> > >>> > > > > > > > > >> and
> >
> > > > >> > >>> > > > > > > > > >> >> > Kafka
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > rest
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent
> > will
> >
> > > > need
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> >
> > > > >> > >>> run a
> >
> > > > >> > >>> > > > task
> >
> > > > >> > >>> > > > > on
> >
> > > > >> > >>> > > > > > > > > behalf
> >
> > > > >> > >>> > > > > > > > > >> of a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > client.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > We
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change
> > how
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> >
> > > > >> > >>> > services
> >
> > > > >> > >>> > > > use
> >
> > > > >> > >>> > > > > > > Kafka
> >
> > > > >> > >>> > > > > > > > > >> clients
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer).
> >
> > > Instead
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> >
> > > > >> > >>> > shared
> >
> > > > >> > >>> > > > > client
> >
> > > > >> > >>> > > > > > > per
> >
> > > > >> > >>> > > > > > > > > >> worker,
> >
> > > > >> > >>> > > > > > > > > >> >> > we
> >
> > > > >> > >>> > > > > > > > > >> >> > > > will
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > need
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > a
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task
> > since
> >
> > > > the
> >
> > > > >> > >>> > > > authentication
> >
> > > > >> > >>> > > > > > > > happens
> >
> > > > >> > >>> > > > > > > > > >> at the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > connection
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka
> > Connect,
> >
> > > > the
> >
> > > > >> > >>> renewer
> >
> > > > >> > >>> > > will
> >
> > > > >> > >>> > > > be
> >
> > > > >> > >>> > > > > > the
> >
> > > > >> > >>> > > > > > > > > >> workers.
> >
> > > > >> > >>> > > > > > > > > >> >> > So,
> >
> > > > >> > >>> > > > > > > > > >> >> > > we
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> >
> > > > >> > For
> >
> > > > >> > >>> > > Kafka
> >
> > > > >> > >>> > > > > rest
> >
> > > > >> > >>> > > > > > > > > proxy,
> >
> > > > >> > >>> > > > > > > > > >> the
> >
> > > > >> > >>> > > > > > > > > >> >> > > > renewer
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > can
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the
> >
> > > creator
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> >
> > > > >> > the
> >
> > > > >> > >>> > > token.
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new
> acl
> > on
> >
> > > > who
> >
> > > > >> > can
> >
> > > > >> > >>> > > request
> >
> > > > >> > >>> > > > > > > > delegation
> >
> > > > >> > >>> > > > > > > > > >> tokens?
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > >
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend
> >
> > > people
> >
> > > > to
> >
> > > > >> > send
> >
> > > > >> > >>> > > > > delegation
> >
> > > > >> > >>> > > > > > > > tokens
> >
> > > > >> > >>> > > > > > > > > >> in an
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > encrypted
> >
> > > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> >
> > > > >> > >>>
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ashish Singh <as...@cloudera.com>.
Manikumar,
Thanks for the update. As its been a while since any progress was made
here, I started working on POC for this. Will be happy to share the
progress I have made so far, let's discuss that on parent JIRA. However,
more importantly while working on POC, I had following questions that I
think are worth considering.
1. How to disable delegation token authentication?
Not everyone would want to authenticate via delegation token. If someone is
not interested in this, then this unnecessarily increases threat vector for
such users. Having a way to turn it off will be appreciated by these users.
Rather I think by default this should be kept off.
This can be achieved in various ways, however I think reusing delegation
token secret config for this makes sense here. Avoids creating yet another
config and forces delegation token users to consciously set the secret. If
the secret is not set or set to empty string, brokers should turn off
delegation token support. This will however require a new error code to
indicate delegation token support is turned off on broker.
2. ACLs on delegation token?
Do we need to have ACLs defined for tokens? I do not think it buys us
anything, as delegation token can be treated as impersonation of the owner.
Any thing the owner has permission to do, delegation tokens should be
allowed to do as well. If so, we probably won't need to return
authorization exception error code while creating delegation token. It
however would make sense to check renew and expire requests are coming from
owner or renewers of the token, but that does not require explicit acls.
3. How to restrict max life time of a token?
Admins might want to restrict max life time of tokens created on a cluster,
and this can very from cluster to cluster based on use-cases. This might
warrant a separate broker config.
Few more comments based on recent KIP update.
1. Do we need a separate {{InvalidateTokenRequest}}? Can't we use
{{ExpireTokenRequest}} with with expiryDate set to anything before current
date?
2. Can we change time field names to indicate their unit is milliseconds,
like, IssueDateMs, ExpiryDateMs, etc.?
3. Can we allow users to renew a token for a specified amount of time? In
current version of KIP, renew request does not take time as a param, not
sure what is expiry time set to after renewal.
On Mon, Dec 12, 2016 at 9:08 AM Manikumar <ma...@gmail.com> wrote:
> Hi,
>
>
>
> I would like to reinitiate the discussion on Delegation token support for
>
> Kafka.
>
>
>
> Brief summary of the past discussion:
>
>
>
> 1) Broker stores delegation tokens in zookeeper. All brokers will have a
>
> cache backed by
>
> zookeeper so they will all get notified whenever a new token is
>
> generated and they will
>
> update their local cache whenever token state changes.
>
> 2) The current proposal does not support rotation of secret
>
> 3) Only allow the renewal by users that authenticated using *non*
>
> delegation token mechanism
>
> 4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can
>
> authenticate using
>
> SCRAM-SHA-256, providing the delegation token HMAC as password.
>
>
>
> Updated the KIP with the following:
>
> 1. Protocol and Config changes
>
> 2. format of the data stored in ZK.
>
> 3. Changes to Java Clients/Usage of SASL SCRAM mechanism
>
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>
> 48+Delegation+token+support+for+Kafka
>
>
>
>
>
> Jun, Ashish, Gwen,
>
>
>
> Pl review the updated KIP.
>
>
>
> Thanks
>
> Manikumar
>
>
>
>
>
> On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <as...@cloudera.com> wrote:
>
>
>
> > Harsha/ Gwen,
>
> >
>
> > How do we proceed here? I am willing to help out with here.
>
> >
>
> > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
>
> >
>
> > > Is it updated? are all concerns addressed? do you want to start a vote?
>
> > >
>
> > > Sorry for being pushy, I do appreciate that we are all volunteers and
>
> > > finding time is difficult. This feature is important for anything that
>
> > > integrates with Kafka (stream processors, Flume, NiFi, etc) and I
>
> > > don't want to see this getting stuck because we lack coordination
>
> > > within the community.
>
> > >
>
> > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <ka...@harsha.io>
>
> > > wrote:
>
> > > > The only pending update for the KIP is to write up the protocol
> changes
>
> > > like
>
> > > > we've it KIP-4. I'll update the wiki.
>
> > > >
>
> > > >
>
> > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com>
>
> > > wrote:
>
> > > >>
>
> > > >> I think we decided to not support secret rotation, I guess this can
> be
>
> > > >> stated clearly on the KIP. Also, more details on how clients will
>
> > > perform
>
> > > >> token distribution and how CLI will look like will be helpful.
>
> > > >>
>
> > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io>
>
> > > wrote:
>
> > > >>
>
> > > >> > Hi Guys,
>
> > > >> >
>
> > > >> > This discussion was dead for a while. Are there still contentious
>
> > > >> > points? If not, why are there no votes?
>
> > > >> >
>
> > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io>
> wrote:
>
> > > >> > > Ashish,
>
> > > >> > >
>
> > > >> > > Yes, I will send out a KIP invite for next week to discuss
> KIP-48
>
> > > and
>
> > > >> > other
>
> > > >> > > remaining KIPs.
>
> > > >> > >
>
> > > >> > > Thanks,
>
> > > >> > >
>
> > > >> > > Jun
>
> > > >> > >
>
> > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
>
> > asingh@cloudera.com>
>
> > > >> > wrote:
>
> > > >> > >
>
> > > >> > >> Thanks Harsha!
>
> > > >> > >>
>
> > > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we
> did
>
> > > not
>
> > > >> > >> actually make a call on when we should have next KIP call. As
>
> > there
>
> > > >> > >> are
>
> > > >> > a
>
> > > >> > >> few outstanding KIPs that could not be discussed this week, can
>
> > we
>
> > > >> > >> have
>
> > > >> > a
>
> > > >> > >> KIP hangout call next week?
>
> > > >> > >>
>
> > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
>
> > > >> > >> <ka...@harsha.io>
>
> > > >> > >> wrote:
>
> > > >> > >>
>
> > > >> > >>> Ashish,
>
> > > >> > >>> Yes we are working on it. Lets discuss in the next KIP
>
> > > >> > >>> meeting.
>
> > > >> > >>> I'll join.
>
> > > >> > >>> -Harsha
>
> > > >> > >>>
>
> > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
>
> > > asingh@cloudera.com>
>
> > > >> > >>> wrote:
>
> > > >> > >>>
>
> > > >> > >>> > Hello Harsha,
>
> > > >> > >>> >
>
> > > >> > >>> > Are you still working on this? Wondering if we can discuss
>
> > this
>
> > > in
>
> > > >> > next
>
> > > >> > >>> KIP
>
> > > >> > >>> > meeting, if you can join.
>
> > > >> > >>> >
>
> > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
>
> > > >> > kafka@harsha.io>
>
> > > >> > >>> > wrote:
>
> > > >> > >>> >
>
> > > >> > >>> > > Hi Grant,
>
> > > >> > >>> > > We are working on it. Will add the details to
> KIP
>
> > > >> > >>> > > about
>
> > > >> > the
>
> > > >> > >>> > > request protocol.
>
> > > >> > >>> > >
>
> > > >> > >>> > > Thanks,
>
> > > >> > >>> > > Harsha
>
> > > >> > >>> > >
>
> > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
>
> > > >> > >>> > > <gh...@cloudera.com>
>
> > > >> > >>> wrote:
>
> > > >> > >>> > >
>
> > > >> > >>> > > > Hi Parth,
>
> > > >> > >>> > > >
>
> > > >> > >>> > > > Are you still working on this? If you need any help
> please
>
> > > >> > >>> > > > don't
>
> > > >> > >>> > hesitate
>
> > > >> > >>> > > > to ask.
>
> > > >> > >>> > > >
>
> > > >> > >>> > > > Thanks,
>
> > > >> > >>> > > > Grant
>
> > > >> > >>> > > >
>
> > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
>
> > jun@confluent.io>
>
> > > >> > wrote:
>
> > > >> > >>> > > >
>
> > > >> > >>> > > > > Parth,
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > Thanks for the reply.
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > It makes sense to only allow the renewal by users that
>
> > > >> > >>> authenticated
>
> > > >> > >>> > > > using
>
> > > >> > >>> > > > > *non* delegation token mechanism. Then, should we make
>
> > the
>
> > > >> > >>> renewal a
>
> > > >> > >>> > > > list?
>
> > > >> > >>> > > > > For example, in the case of rest proxy, it will be
>
> > useful
>
> > > >> > >>> > > > > for
>
> > > >> > >>> every
>
> > > >> > >>> > > > > instance of rest proxy to be able to renew the tokens.
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > It would be clearer if we can document the request
>
> > > protocol
>
> > > >> > like
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>
> > > >> > >>> > >
>
> > > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
>
> > > operations#KIP-4-
>
> > > >> > >>> > > Commandlineandcentralizedadministrativeoperations-
>
> > > >> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.
>
> > > 10.1.0)
>
> > > >> > >>> > > > > .
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > It would also be useful to document the client APIs.
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > Thanks,
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > Jun
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
>
> > > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > > > Hi,
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > I am suggesting that we will only allow the renewal
> by
>
> > > >> > >>> > > > > > users
>
> > > >> > >>> that
>
> > > >> > >>> > > > > > authenticated using *non* delegation token
> mechanism.
>
> > > For
>
> > > >> > >>> example,
>
> > > >> > >>> > If
>
> > > >> > >>> > > > > user
>
> > > >> > >>> > > > > > Alice authenticated using kerberos and requested
>
> > > >> > >>> > > > > > delegation
>
> > > >> > >>> tokens,
>
> > > >> > >>> > > > only
>
> > > >> > >>> > > > > > user Alice authenticated via non delegation token
>
> > > >> > >>> > > > > > mechanism
>
> > > >> > can
>
> > > >> > >>> > > renew.
>
> > > >> > >>> > > > > > Clients that have access to delegation tokens can
> not
>
> > > >> > >>> > > > > > issue
>
> > > >> > >>> > renewal
>
> > > >> > >>> > > > > > request for renewing their own token and this is
>
> > > primarily
>
> > > >> > >>> > important
>
> > > >> > >>> > > to
>
> > > >> > >>> > > > > > reduce the time window for which a compromised token
>
> > > will
>
> > > >> > >>> > > > > > be
>
> > > >> > >>> valid.
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > To clarify, Yes any authenticated user can request
>
> > > >> > >>> > > > > > delegation
>
> > > >> > >>> > tokens
>
> > > >> > >>> > > > but
>
> > > >> > >>> > > > > > even here I would recommend to avoid creating a
> chain
>
> > > >> > >>> > > > > > where a
>
> > > >> > >>> > client
>
> > > >> > >>> > > > > > authenticated via delegation token request for more
>
> > > >> > delegation
>
> > > >> > >>> > > tokens.
>
> > > >> > >>> > > > > > Basically anyone can request delegation token, as
> long
>
> > > as
>
> > > >> > they
>
> > > >> > >>> > > > > authenticate
>
> > > >> > >>> > > > > > via a non delegation token mechanism.
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > Aren't classes listed here
>
> > > >> > >>> > > > > > <
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > >
>
> > > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>
> > > >> > >>> > > 48+Delegation+token+support+fo
>
> > r+Kafka#KIP-48Delegationtokens
>
> > > >> > >>> upportforKaf
>
> > > >> > >>> > > ka-PublicInterfaces
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > sufficient?
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > Thanks
>
> > > >> > >>> > > > > > Parth
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
>
> > > >> > >>> > > > > > <ju...@confluent.io>
>
> > > >> > >>> wrote:
>
> > > >> > >>> > > > > >
>
> > > >> > >>> > > > > > > Parth,
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > Thanks for the reply. A couple of comments inline
>
> > > below.
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth
> brahmbhatt <
>
> > > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original
>
> > > requester
>
> > > >> > >>> only? or
>
> > > >> > >>> > > > using
>
> > > >> > >>> > > > > > > > Kerberos
>
> > > >> > >>> > > > > > > > auth only?
>
> > > >> > >>> > > > > > > > My recommendation is to do this only using
>
> > Kerberos
>
> > > >> > >>> > > > > > > > auth
>
> > > >> > and
>
> > > >> > >>> > only
>
> > > >> > >>> > > > > threw
>
> > > >> > >>> > > > > > > the
>
> > > >> > >>> > > > > > > > renewer specified during the acquisition
> request.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying
>
> > that
>
> > > >> > >>> > > > > > > any
>
> > > >> > >>> client
>
> > > >> > >>> > > > > > > authenticated with the delegation token can renew,
>
> > > i.e.
>
> > > >> > there
>
> > > >> > >>> is
>
> > > >> > >>> > no
>
> > > >> > >>> > > > > > renewer
>
> > > >> > >>> > > > > > > needed?
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > Also, just to be clear, any authenticated client
>
> > > (either
>
> > > >> > >>> through
>
> > > >> > >>> > > SASL
>
> > > >> > >>> > > > > or
>
> > > >> > >>> > > > > > > SSL) can request a delegation token for the
>
> > > >> > >>> > > > > > > authenticated
>
> > > >> > >>> user,
>
> > > >> > >>> > > > right?
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
>
> > > >> > >>> > > > > > > > My recommendation is still to store in ZK or not
>
> > > store
>
> > > >> > them
>
> > > >> > >>> at
>
> > > >> > >>> > > all.
>
> > > >> > >>> > > > > The
>
> > > >> > >>> > > > > > > > whole controller based distribution is too much
>
> > > >> > >>> > > > > > > > overhead
>
> > > >> > >>> with
>
> > > >> > >>> > not
>
> > > >> > >>> > > > > much
>
> > > >> > >>> > > > > > to
>
> > > >> > >>> > > > > > > > achieve.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
>
> > > >> > >>> > > > > > > > Either by expiration time out or through an
>
> > explicit
>
> > > >> > >>> request to
>
> > > >> > >>> > > > > > > invalidate.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
>
> > > >> > >>> > > > > > > > SCRAM
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't
>
> > in
>
> > > >> > >>> > > > > > > > the
>
> > > >> > KIP
>
> > > >> > >>> but
>
> > > >> > >>> > > was
>
> > > >> > >>> > > > > > > > discussed
>
> > > >> > >>> > > > > > > > in this thread)?
>
> > > >> > >>> > > > > > > > There is no imperonation proposal. I tried and
>
> > > >> > >>> > > > > > > > explained
>
> > > >> > how
>
> > > >> > >>> > its
>
> > > >> > >>> > > a
>
> > > >> > >>> > > > > > > > different problem and why its not really
> necessary
>
> > > to
>
> > > >> > >>> discuss
>
> > > >> > >>> > > that
>
> > > >> > >>> > > > as
>
> > > >> > >>> > > > > > > part
>
> > > >> > >>> > > > > > > > of this KIP. This KIP will not support any
>
> > > >> > impersonation,
>
> > > >> > >>> it
>
> > > >> > >>> > > will
>
> > > >> > >>> > > > > just
>
> > > >> > >>> > > > > > > be
>
> > > >> > >>> > > > > > > > another way to authenticate.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what
> actions?
>
> > > >> > >>> > > > > > > > We do not need new ACLs.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > Could we document the format of the new
>
> > > request/response
>
> > > >> > and
>
> > > >> > >>> > their
>
> > > >> > >>> > > > > > > associated Resource and Operation for ACL?
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > >
>
> > > >> > >>> > > > > > > > 7. How would the delegation token be configured
> in
>
> > > the
>
> > > >> > >>> client?
>
> > > >> > >>> > > > > > > > Should be through config. I wasn't planning on
>
> > > >> > >>> > > > > > > > supporting
>
> > > >> > >>> JAAS
>
> > > >> > >>> > > for
>
> > > >> > >>> > > > > > > tokens.
>
> > > >> > >>> > > > > > > > I don't believe hadoop does this either.
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > Thanks
>
> > > >> > >>> > > > > > > > Parth
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
>
> > > >> > jun@confluent.io>
>
> > > >> > >>> > > wrote:
>
> > > >> > >>> > > > > > > >
>
> > > >> > >>> > > > > > > > > Harsha,
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > Another question.
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > 9. How would the delegation token be
> configured
>
> > in
>
> > > >> > >>> > > > > > > > > the
>
> > > >> > >>> > client?
>
> > > >> > >>> > > > The
>
> > > >> > >>> > > > > > > > standard
>
> > > >> > >>> > > > > > > > > way is to do this through JAAS. However, we
> will
>
> > > >> > >>> > > > > > > > > need
>
> > > >> > to
>
> > > >> > >>> > think
>
> > > >> > >>> > > > > > through
>
> > > >> > >>> > > > > > > if
>
> > > >> > >>> > > > > > > > > this is convenient in a shared environment.
> For
>
> > > >> > example,
>
> > > >> > >>> > when a
>
> > > >> > >>> > > > new
>
> > > >> > >>> > > > > > > task
>
> > > >> > >>> > > > > > > > is
>
> > > >> > >>> > > > > > > > > added to a Storm worker node, do we need to
>
> > > >> > >>> > > > > > > > > dynamically
>
> > > >> > >>> add a
>
> > > >> > >>> > > new
>
> > > >> > >>> > > > > > > section
>
> > > >> > >>> > > > > > > > > in the JAAS file? It may be more convenient if
>
> > we
>
> > > >> > >>> > > > > > > > > can
>
> > > >> > >>> pass in
>
> > > >> > >>> > > the
>
> > > >> > >>> > > > > > token
>
> > > >> > >>> > > > > > > > > through the config directly w/o going through
>
> > > JAAS.
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > Are you or Parth still actively working on
> this
>
> > > KIP?
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > Thanks,
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > Jun
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
>
> > > >> > >>> jun@confluent.io>
>
> > > >> > >>> > > > wrote:
>
> > > >> > >>> > > > > > > > >
>
> > > >> > >>> > > > > > > > > > Just to add on that list.
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > > 2. It would be good to document the format
> of
>
> > > the
>
> > > >> > data
>
> > > >> > >>> > stored
>
> > > >> > >>> > > > in
>
> > > >> > >>> > > > > > ZK.
>
> > > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on
> whether
>
> > > the
>
> > > >> > tokens
>
> > > >> > >>> > > should
>
> > > >> > >>> > > > > be
>
> > > >> > >>> > > > > > > > > > propagated through ZK like config/acl/quota,
>
> > or
>
> > > >> > through
>
> > > >> > >>> the
>
> > > >> > >>> > > > > > > controller.
>
> > > >> > >>> > > > > > > > > > Currently, the controller is only designed
> for
>
> > > >> > >>> propagating
>
> > > >> > >>> > > > topic
>
> > > >> > >>> > > > > > > > > metadata,
>
> > > >> > >>> > > > > > > > > > but not other data.
>
> > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token
>
> > instead
>
> > > >> > >>> > > > > > > > > > of
>
> > > >> > >>> > > DIGEST-MD5
>
> > > >> > >>> > > > > > since
>
> > > >> > >>> > > > > > > > it's
>
> > > >> > >>> > > > > > > > > > deprecated?
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > > Thanks,
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > > Jun
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen
>
> > Shapira <
>
> > > >> > >>> > > > > gwen@confluent.io>
>
> > > >> > >>> > > > > > > > > wrote:
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > >> From what I can see, remaining questions
> are:
>
> > > >> > >>> > > > > > > > > >>
>
> > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By
> original
>
> > > >> > requester
>
> > > >> > >>> > only?
>
> > > >> > >>> > > > or
>
> > > >> > >>> > > > > > > using
>
> > > >> > >>> > > > > > > > > >> Kerberos auth only?
>
> > > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in
> ZK?
>
> > > >> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
>
> > > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
>
> > > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it
>
> > > wasn't
>
> > > >> > >>> > > > > > > > > >> in
>
> > > >> > the
>
> > > >> > >>> > KIP
>
> > > >> > >>> > > > but
>
> > > >> > >>> > > > > > was
>
> > > >> > >>> > > > > > > > > >> discussed in this thread)?
>
> > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what
>
> > > actions?
>
> > > >> > >>> > > > > > > > > >>
>
> > > >> > >>> > > > > > > > > >> Gwen
>
> > > >> > >>> > > > > > > > > >>
>
> > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
>
> > > >> > >>> kafka@harsha.io>
>
> > > >> > >>> > > > wrote:
>
> > > >> > >>> > > > > > > > > >> > Jun & Ismael,
>
> > > >> > >>> > > > > > > > > >> > Unfortunately I
>
> > > >> > >>> > > > > > > > > >> > couldn't
>
> > > >> > >>> attend
>
> > > >> > >>> > > the
>
> > > >> > >>> > > > > KIP
>
> > > >> > >>> > > > > > > > > meeting
>
> > > >> > >>> > > > > > > > > >> > when delegation
>
> > > tokens
>
> > > >> > >>> > discussed.
>
> > > >> > >>> > > > > > > > Appreciate
>
> > > >> > >>> > > > > > > > > if
>
> > > >> > >>> > > > > > > > > >> > you can update
> the
>
> > > >> > thread if
>
> > > >> > >>> > you
>
> > > >> > >>> > > > have
>
> > > >> > >>> > > > > > any
>
> > > >> > >>> > > > > > > > > >> > further
> questions.
>
> > > >> > >>> > > > > > > > > >> > Thanks,
>
> > > >> > >>> > > > > > > > > >> > Harsha
>
> > > >> > >>> > > > > > > > > >> >
>
> > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan
>
> > Pei
>
> > > >> > wrote:
>
> > > >> > >>> > > > > > > > > >> >> It seems that the links to images in the
>
> > KIP
>
> > > >> > >>> > > > > > > > > >> >> are
>
> > > >> > >>> > broken.
>
> > > >> > >>> > > > > > > > > >> >>
>
> > > >> > >>> > > > > > > > > >> >> Liquan
>
> > > >> > >>> > > > > > > > > >> >>
>
> > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
>
> > > >> > brahmbhatt <
>
> > > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
>
> > > >> > >>> > > > > > > > > >> >>
>
> > > >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs
>
> > mean?
>
> > > >> > >>> > > > > > > > > >> >> > In the current proposal we only allow
> a
>
> > > user
>
> > > >> > >>> > > > > > > > > >> >> > to
>
> > > >> > >>> get
>
> > > >> > >>> > > > > > delegation
>
> > > >> > >>> > > > > > > > > token
>
> > > >> > >>> > > > > > > > > >> for
>
> > > >> > >>> > > > > > > > > >> >> > the identity that it authenticated as
>
> > > using
>
> > > >> > >>> another
>
> > > >> > >>> > > > > > mechanism,
>
> > > >> > >>> > > > > > > > i.e.
>
> > > >> > >>> > > > > > > > > >> A user
>
> > > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
>
> > > >> > >>> > > > > > > > > >> >> > principal
>
> > > >> > >>> > > > > > > user1@EXAMPLE.COM
>
> > > >> > >>> > > > > > > > > >> will get
>
> > > >> > >>> > > > > > > > > >> >> > delegation tokens for that user only.
> In
>
> > > >> > future I
>
> > > >> > >>> > think
>
> > > >> > >>> > > > we
>
> > > >> > >>> > > > > > will
>
> > > >> > >>> > > > > > > > > have
>
> > > >> > >>> > > > > > > > > >> to
>
> > > >> > >>> > > > > > > > > >> >> > extend support such that we allow some
>
> > set
>
> > > >> > >>> > > > > > > > > >> >> > of
>
> > > >> > >>> users (
>
> > > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
>
> > > >> > >>> > storm-nimbus@EXAMPLE.COM)
>
> > > >> > >>> > > > to
>
> > > >> > >>> > > > > > > > acquire
>
> > > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other
>
> > users
>
> > > >> > whose
>
> > > >> > >>> > > identity
>
> > > >> > >>> > > > > > they
>
> > > >> > >>> > > > > > > > have
>
> > > >> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers
>
> > > will
>
> > > >> > have
>
> > > >> > >>> ACLs
>
> > > >> > >>> > > to
>
> > > >> > >>> > > > > > > control
>
> > > >> > >>> > > > > > > > > >> which
>
> > > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate other
>
> > > users
>
> > > >> > and
>
> > > >> > >>> get
>
> > > >> > >>> > > > tokens
>
> > > >> > >>> > > > > > on
>
> > > >> > >>> > > > > > > > > >> behalf of
>
> > > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
>
> > > >> > different
>
> > > >> > >>> > > problem
>
> > > >> > >>> > > > in
>
> > > >> > >>> > > > > > my
>
> > > >> > >>> > > > > > > > > >> opinion and
>
> > > >> > >>> > > > > > > > > >> >> > I think we can tackle it in separate
>
> > KIP.
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of
> getting
>
> > > and
>
> > > >> > >>> renewing
>
> > > >> > >>> > > > > > delegation
>
> > > >> > >>> > > > > > > > > >> tokens?
>
> > > >> > >>> > > > > > > > > >> >> > Typically this should be very very
> low,
>
> > 1
>
> > > >> > request
>
> > > >> > >>> per
>
> > > >> > >>> > > > > minute
>
> > > >> > >>> > > > > > > is a
>
> > > >> > >>> > > > > > > > > >> >> > relatively high estimate. However it
>
> > > depends
>
> > > >> > >>> > > > > > > > > >> >> > on
>
> > > >> > >>> the
>
> > > >> > >>> > > token
>
> > > >> > >>> > > > > > > > > >> expiration. I am
>
> > > >> > >>> > > > > > > > > >> >> > less worried about the extra load it
>
> > puts
>
> > > on
>
> > > >> > >>> > controller
>
> > > >> > >>> > > > vs
>
> > > >> > >>> > > > > > the
>
> > > >> > >>> > > > > > > > > added
>
> > > >> > >>> > > > > > > > > >> >> > complexity and the value it offers.
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> > Thanks
>
> > > >> > >>> > > > > > > > > >> >> > Parth
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM,
> Ismael
>
> > > Juma
>
> > > >> > >>> > > > > > > > > >> >> > <
>
> > > >> > >>> > > > > > > ismael@juma.me.uk>
>
> > > >> > >>> > > > > > > > > >> wrote:
>
> > > >> > >>> > > > > > > > > >> >> >
>
> > > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably
>
> > > require a
>
> > > >> > >>> separate
>
> > > >> > >>> > > KIP
>
> > > >> > >>> > > > > as
>
> > > >> > >>> > > > > > it
>
> > > >> > >>> > > > > > > > > will
>
> > > >> > >>> > > > > > > > > >> >> > > introduce user visible changes. We
>
> > could
>
> > > >> > >>> > > > > > > > > >> >> > > also
>
> > > >> > >>> > update
>
> > > >> > >>> > > > > KIP-48
>
> > > >> > >>> > > > > > > to
>
> > > >> > >>> > > > > > > > > >> have this
>
> > > >> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to
>
> > do
>
> > > it
>
> > > >> > >>> > > separately.
>
> > > >> > >>> > > > We
>
> > > >> > >>> > > > > > can
>
> > > >> > >>> > > > > > > > > >> discuss
>
> > > >> > >>> > > > > > > > > >> >> > that
>
> > > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
>
> > > >> > >>> > > > > > > > > >> >> > >
>
> > > >> > >>> > > > > > > > > >> >> > > Ismael
>
> > > >> > >>> > > > > > > > > >> >> > >
>
> > > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM,
>
> > Rajini
>
> > > >> > Sivaram
>
> > > >> > >>> <
>
> > > >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com>
> wrote:
>
> > > >> > >>> > > > > > > > > >> >> > >
>
> > > >> > >>> > > > > > > > > >> >> > > > Ismael,
>
> > > >> > >>> > > > > > > > > >> >> > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
>
> > > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
>
> > > >> > jira/browse/KAFKA-3751)
>
> > > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
>
> > mechanism.
>
> > > >> > >>> > > > > > > > > >> >> > > > Would
>
> > > >> > >>> that
>
> > > >> > >>> > > need
>
> > > >> > >>> > > > > > > another
>
> > > >> > >>> > > > > > > > > >> KIP? If
>
> > > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism,
> can
>
> > > this
>
> > > >> > just
>
> > > >> > >>> be
>
> > > >> > >>> > a
>
> > > >> > >>> > > > JIRA
>
> > > >> > >>> > > > > > > that
>
> > > >> > >>> > > > > > > > > gets
>
> > > >> > >>> > > > > > > > > >> >> > > reviewed
>
> > > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
>
> > > >> > >>> > > > > > > > > >> >> > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > Thank you,
>
> > > >> > >>> > > > > > > > > >> >> > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > Rajini
>
> > > >> > >>> > > > > > > > > >> >> > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM,
>
> > > Ismael
>
> > > >> > Juma <
>
> > > >> > >>> > > > > > > > > ismael@juma.me.uk>
>
> > > >> > >>> > > > > > > > > >> >> > wrote:
>
> > > >> > >>> > > > > > > > > >> >> > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like
> a
>
> > > good
>
> > > >> > >>> > candidate.
>
> > > >> > >>> > > > > > > > > >> >> > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned
>
> > > this
>
> > > >> > >>> > > > > > > > > >> >> > > > > as
>
> > > >> > a
>
> > > >> > >>> SASL
>
> > > >> > >>> > > > > > mechanism
>
> > > >> > >>> > > > > > > > > that
>
> > > >> > >>> > > > > > > > > >> might
>
> > > >> > >>> > > > > > > > > >> >> > be
>
> > > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been
>
> > > >> > >>> > > > > > > > > >> >> > > > > meaning
>
> > > >> > to
>
> > > >> > >>> > check
>
> > > >> > >>> > > > it
>
> > > >> > >>> > > > > in
>
> > > >> > >>> > > > > > > > more
>
> > > >> > >>> > > > > > > > > >> detail.
>
> > > >> > >>> > > > > > > > > >> >> > > Good
>
> > > >> > >>> > > > > > > > > >> >> > > > > to know that you are willing to
>
> > > >> > contribute
>
> > > >> > >>> an
>
> > > >> > >>> > > > > > > > implementation.
>
> > > >> > >>> > > > > > > > > >> Maybe
>
> > > >> > >>> > > > > > > > > >> >> > we
>
> > > >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for
>
> > > this?
>
> > > >> > >>> > > > > > > > > >> >> > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > Ismael
>
> > > >> > >>> > > > > > > > > >> >> > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM,
>
> > > >> > >>> > > > > > > > > >> >> > > > > Rajini
>
> > > >> > >>> > Sivaram <
>
> > > >> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com>
>
> > > wrote:
>
> > > >> > >>> > > > > > > > > >> >> > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge
> Response
>
> > > >> > >>> > Authentication
>
> > > >> > >>> > > > > > > > Mechanism)
>
> > > >> > >>> > > > > > > > > >> is a
>
> > > >> > >>> > > > > > > > > >> >> > > better
>
> > > >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5.
> Java
>
> > > >> > >>> > > > > > > > > >> >> > > > > > doesn't
>
> > > >> > >>> come
>
> > > >> > >>> > > > with a
>
> > > >> > >>> > > > > > > > > built-in
>
> > > >> > >>> > > > > > > > > >> SCRAM
>
> > > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but
> I
>
> > > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > be
>
> > > >> > >>> happy
>
> > > >> > >>> > > to
>
> > > >> > >>> > > > > add
>
> > > >> > >>> > > > > > > > > support
>
> > > >> > >>> > > > > > > > > >> in
>
> > > >> > >>> > > > > > > > > >> >> > Kafka
>
> > > >> > >>> > > > > > > > > >> >> > > > > since
>
> > > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism
>
> > to
>
> > > >> > support
>
> > > >> > >>> > > anyway.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/
>
> > > rfc7677
>
> > > >> > >>> > describes
>
> > > >> > >>> > > > the
>
> > > >> > >>> > > > > > > > protocol
>
> > > >> > >>> > > > > > > > > >> for
>
> > > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
>
> > > >> > >>> > > > > > > > > >> >> > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37
> AM,
>
> > > Jun
>
> > > >> > Rao <
>
> > > >> > >>> > > > > > > > jun@confluent.io
>
> > > >> > >>> > > > > > > > > >
>
> > > >> > >>> > > > > > > > > >> wrote:
>
> > > >> > >>> > > > > > > > > >> >> > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation.
> A
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > couple
>
> > > >> > of
>
> > > >> > >>> > more
>
> > > >> > >>> > > > > > > questions.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
>
> > > >> > >>> mean?
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate
>
> > of
>
> > > >> > getting
>
> > > >> > >>> and
>
> > > >> > >>> > > > > > renewing
>
> > > >> > >>> > > > > > > > > >> delegation
>
> > > >> > >>> > > > > > > > > >> >> > > > tokens?
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on
>
> > > whether
>
> > > >> > they
>
> > > >> > >>> > > should
>
> > > >> > >>> > > > be
>
> > > >> > >>> > > > > > > > > directed
>
> > > >> > >>> > > > > > > > > >> to the
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > controller.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > Jun
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19
>
> > PM,
>
> > > >> > parth
>
> > > >> > >>> > > > > brahmbhatt <
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com>
>
> > > wrote:
>
> > > >> > >>> > > > > > > > > >> >> > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster
>
> > > action
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > to
>
> > > >> > add
>
> > > >> > >>> > acls
>
> > > >> > >>> > > > on
>
> > > >> > >>> > > > > > who
>
> > > >> > >>> > > > > > > > can
>
> > > >> > >>> > > > > > > > > >> request
>
> > > >> > >>> > > > > > > > > >> >> > > > > > delegation
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the
> use
>
> > > case
>
> > > >> > for
>
> > > >> > >>> that
>
> > > >> > >>> > > yet
>
> > > >> > >>> > > > > but
>
> > > >> > >>> > > > > > > > down
>
> > > >> > >>> > > > > > > > > >> the line
>
> > > >> > >>> > > > > > > > > >> >> > > > when
>
> > > >> > >>> > > > > > > > > >> >> > > > > we
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
>
> > > >> > getDelegationTokenAs
>
> > > >> > >>> it
>
> > > >> > >>> > > will
>
> > > >> > >>> > > > > be
>
> > > >> > >>> > > > > > > > > >> necessary.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens
> to
>
> > > be
>
> > > >> > only
>
> > > >> > >>> > > > > > > used/distributed
>
> > > >> > >>> > > > > > > > > >> over
>
> > > >> > >>> > > > > > > > > >> >> > secure
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > channels.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design
>
> > we
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > end
>
> > > >> > up
>
> > > >> > >>> > > choosing
>
> > > >> > >>> > > > > > > > > >> Invalidation will
>
> > > >> > >>> > > > > > > > > >> >> > > be
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every
>
> > broker
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > or
>
> > > >> > >>> > > controller.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
>
> > > documented
>
> > > >> > >>> somewhere
>
> > > >> > >>> > > > that
>
> > > >> > >>> > > > > > > > > >> invalidation
>
> > > >> > >>> > > > > > > > > >> >> > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > directly
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but
>
> > that
>
> > > is
>
> > > >> > not
>
> > > >> > >>> the
>
> > > >> > >>> > > > > intent.
>
> > > >> > >>> > > > > > > > > >> Invalidation
>
> > > >> > >>> > > > > > > > > >> >> > > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > either
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to
>
> > > >> > >>> expiration. No
>
> > > >> > >>> > > > > direct
>
> > > >> > >>> > > > > > > > > >> zookeeper
>
> > > >> > >>> > > > > > > > > >> >> > > > > interaction
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > from
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
>
> > > >> > >>> DelegationToken
>
> > > >> > >>> > > > > without
>
> > > >> > >>> > > > > > > the
>
> > > >> > >>> > > > > > > > > >> hmac in
>
> > > >> > >>> > > > > > > > > >> >> > the
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about
>
> > the
>
> > > >> > >>> confusion.
>
> > > >> > >>> > > The
>
> > > >> > >>> > > > > sole
>
> > > >> > >>> > > > > > > > > >> purpose of
>
> > > >> > >>> > > > > > > > > >> >> > > > > zookeeper
>
> > > >> > >>> > > > > > > > > >> >> > > > > > in
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
>
> > distribution
>
> > > >> > channel
>
> > > >> > >>> > for
>
> > > >> > >>> > > > > tokens
>
> > > >> > >>> > > > > > > > > >> between all
>
> > > >> > >>> > > > > > > > > >> >> > > > brokers
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > and a
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only
>
> > tokens
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > that
>
> > > >> > >>> were
>
> > > >> > >>> > > > > > generated
>
> > > >> > >>> > > > > > > by
>
> > > >> > >>> > > > > > > > > >> making a
>
> > > >> > >>> > > > > > > > > >> >> > > > > request
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > to a
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted
>
> > (more
>
> > > on
>
> > > >> > this
>
> > > >> > >>> in
>
> > > >> > >>> > > > second
>
> > > >> > >>> > > > > > > > > >> paragraph). The
>
> > > >> > >>> > > > > > > > > >> >> > > > token
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements
>
> > > (owner,
>
> > > >> > >>> renewer,
>
> > > >> > >>> > > > uuid
>
> > > >> > >>> > > > > ,
>
> > > >> > >>> > > > > > > > > >> expiration,
>
> > > >> > >>> > > > > > > > > >> >> > > hmac)
>
> > > >> > >>> > > > > > > > > >> >> > > > ,
>
> > > >> > >>> > > > > > > > > >> >> > > > > > one
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > of
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally
>
> > generated
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
>
> > > >> > >>> but
>
> > > >> > >>> > > hmac
>
> > > >> > >>> > > > it
>
> > > >> > >>> > > > > > > self
>
> > > >> > >>> > > > > > > > is
>
> > > >> > >>> > > > > > > > > >> >> > derivable
>
> > > >> > >>> > > > > > > > > >> >> > > > if
>
> > > >> > >>> > > > > > > > > >> >> > > > > > you
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other
> elements
>
> > of
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > the
>
> > > >> > >>> token
>
> > > >> > >>> > +
>
> > > >> > >>> > > > > secret
>
> > > >> > >>> > > > > > > key
>
> > > >> > >>> > > > > > > > > to
>
> > > >> > >>> > > > > > > > > >> >> > generate
>
> > > >> > >>> > > > > > > > > >> >> > > > > hmac.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not
>
> > > provide
>
> > > >> > SSL
>
> > > >> > >>> > > support
>
> > > >> > >>> > > > we
>
> > > >> > >>> > > > > > do
>
> > > >> > >>> > > > > > > > not
>
> > > >> > >>> > > > > > > > > >> want the
>
> > > >> > >>> > > > > > > > > >> >> > > > > entire
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire
> transferred
>
> > > to
>
> > > >> > >>> zookeeper
>
> > > >> > >>> > > as
>
> > > >> > >>> > > > > that
>
> > > >> > >>> > > > > > > > will
>
> > > >> > >>> > > > > > > > > >> be an
>
> > > >> > >>> > > > > > > > > >> >> > > > insecure
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > wire
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only
>
> > > store
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > all
>
> > > >> > >>> the
>
> > > >> > >>> > > other
>
> > > >> > >>> > > > > > > > elements
>
> > > >> > >>> > > > > > > > > >> of a
>
> > > >> > >>> > > > > > > > > >> >> > > > > delegation
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read
>
> > these
>
> > > >> > >>> elements
>
> > > >> > >>> > and
>
> > > >> > >>> > > > > > because
>
> > > >> > >>> > > > > > > > > they
>
> > > >> > >>> > > > > > > > > >> also
>
> > > >> > >>> > > > > > > > > >> >> > > have
>
> > > >> > >>> > > > > > > > > >> >> > > > > > access
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be
>
> > > able
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > to
>
> > > >> > >>> > generate
>
> > > >> > >>> > > > > hmac
>
> > > >> > >>> > > > > > on
>
> > > >> > >>> > > > > > > > > >> their end.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative
>
> > > proposed
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > is
>
> > > >> > to
>
> > > >> > >>> > avoid
>
> > > >> > >>> > > > > > > zookeeper
>
> > > >> > >>> > > > > > > > > >> >> > > altogether. A
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > Client
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with
>
> > required
>
> > > >> > >>> > information
>
> > > >> > >>> > > > > > (owner,
>
> > > >> > >>> > > > > > > > > >> renwer,
>
> > > >> > >>> > > > > > > > > >> >> > > > > expiration)
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > and
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac,
>
> > uuid).
>
> > > >> > Broker
>
> > > >> > >>> > won't
>
> > > >> > >>> > > > > store
>
> > > >> > >>> > > > > > > this
>
> > > >> > >>> > > > > > > > > in
>
> > > >> > >>> > > > > > > > > >> >> > > zookeeper.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > From
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client can
>
> > > contact
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > any
>
> > > >> > >>> > broker
>
> > > >> > >>> > > > with
>
> > > >> > >>> > > > > > all
>
> > > >> > >>> > > > > > > > the
>
> > > >> > >>> > > > > > > > > >> >> > > delegation
>
> > > >> > >>> > > > > > > > > >> >> > > > > > token
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
>
> > > expiration,
>
> > > >> > hmac,
>
> > > >> > >>> > > uuid)
>
> > > >> > >>> > > > > the
>
> > > >> > >>> > > > > > > > borker
>
> > > >> > >>> > > > > > > > > >> will
>
> > > >> > >>> > > > > > > > > >> >> > > > > regenerate
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > the
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it
>
> > matches
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > with
>
> > > >> > >>> hmac
>
> > > >> > >>> > > > > > presented
>
> > > >> > >>> > > > > > > by
>
> > > >> > >>> > > > > > > > > >> client ,
>
> > > >> > >>> > > > > > > > > >> >> > > > broker
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
>
> > > >> > >>> Only
>
> > > >> > >>> > > > > problem
>
> > > >> > >>> > > > > > > with
>
> > > >> > >>> > > > > > > > > >> this
>
> > > >> > >>> > > > > > > > > >> >> > > approach
>
> > > >> > >>> > > > > > > > > >> >> > > > > is
>
> > > >> > >>> > > > > > > > > >> >> > > > > > if
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
>
> > compromised
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > any
>
> > > >> > >>> client
>
> > > >> > >>> > > can
>
> > > >> > >>> > > > > now
>
> > > >> > >>> > > > > > > > > generate
>
> > > >> > >>> > > > > > > > > >> >> > random
>
> > > >> > >>> > > > > > > > > >> >> > > > > tokens
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > and
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to
>
> > > >> > >>> authenticate
>
> > > >> > >>> > as
>
> > > >> > >>> > > > any
>
> > > >> > >>> > > > > > user
>
> > > >> > >>> > > > > > > > > they
>
> > > >> > >>> > > > > > > > > >> like.
>
> > > >> > >>> > > > > > > > > >> >> > > with
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee
> that
>
> > > only
>
> > > >> > >>> tokens
>
> > > >> > >>> > > > > acquired
>
> > > >> > >>> > > > > > > via
>
> > > >> > >>> > > > > > > > a
>
> > > >> > >>> > > > > > > > > >> broker
>
> > > >> > >>> > > > > > > > > >> >> > > > (using
>
> > > >> > >>> > > > > > > > > >> >> > > > > > some
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
>
> > > >> > >>> token)
>
> > > >> > >>> > > will
>
> > > >> > >>> > > > > be
>
> > > >> > >>> > > > > > > > > >> accepted. We
>
> > > >> > >>> > > > > > > > > >> >> > > need
>
> > > >> > >>> > > > > > > > > >> >> > > > to
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal
> makes
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > more
>
> > > >> > >>> sense
>
> > > >> > >>> > and
>
> > > >> > >>> > > > we
>
> > > >> > >>> > > > > > can
>
> > > >> > >>> > > > > > > go
>
> > > >> > >>> > > > > > > > > >> over it
>
> > > >> > >>> > > > > > > > > >> >> > in
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the
>
> > > invite
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > to
>
> > > >> > >>> me?
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at
>
> > 10:35
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
>
> > > >> > Jun
>
> > > >> > >>> > Rao <
>
> > > >> > >>> > > > > > > > > >> jun@confluent.io>
>
> > > >> > >>> > > > > > > > > >> >> > > > wrote:
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A
> few
>
> > > >> > comments.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially
> can
>
> > be
>
> > > >> > useful
>
> > > >> > >>> for
>
> > > >> > >>> > > > Kafka
>
> > > >> > >>> > > > > > > > Connect
>
> > > >> > >>> > > > > > > > > >> and
>
> > > >> > >>> > > > > > > > > >> >> > Kafka
>
> > > >> > >>> > > > > > > > > >> >> > > > > rest
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent
> will
>
> > > need
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > to
>
> > > >> > >>> run a
>
> > > >> > >>> > > > task
>
> > > >> > >>> > > > > on
>
> > > >> > >>> > > > > > > > > behalf
>
> > > >> > >>> > > > > > > > > >> of a
>
> > > >> > >>> > > > > > > > > >> >> > > > client.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > We
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change
> how
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > those
>
> > > >> > >>> > services
>
> > > >> > >>> > > > use
>
> > > >> > >>> > > > > > > Kafka
>
> > > >> > >>> > > > > > > > > >> clients
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer).
>
> > Instead
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
>
> > > >> > >>> > shared
>
> > > >> > >>> > > > > client
>
> > > >> > >>> > > > > > > per
>
> > > >> > >>> > > > > > > > > >> worker,
>
> > > >> > >>> > > > > > > > > >> >> > we
>
> > > >> > >>> > > > > > > > > >> >> > > > will
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > need
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > a
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task
> since
>
> > > the
>
> > > >> > >>> > > > authentication
>
> > > >> > >>> > > > > > > > happens
>
> > > >> > >>> > > > > > > > > >> at the
>
> > > >> > >>> > > > > > > > > >> >> > > > > > connection
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka
> Connect,
>
> > > the
>
> > > >> > >>> renewer
>
> > > >> > >>> > > will
>
> > > >> > >>> > > > be
>
> > > >> > >>> > > > > > the
>
> > > >> > >>> > > > > > > > > >> workers.
>
> > > >> > >>> > > > > > > > > >> >> > So,
>
> > > >> > >>> > > > > > > > > >> >> > > we
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > probably
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
>
> > > >> > For
>
> > > >> > >>> > > Kafka
>
> > > >> > >>> > > > > rest
>
> > > >> > >>> > > > > > > > > proxy,
>
> > > >> > >>> > > > > > > > > >> the
>
> > > >> > >>> > > > > > > > > >> >> > > > renewer
>
> > > >> > >>> > > > > > > > > >> >> > > > > > can
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the
>
> > creator
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > of
>
> > > >> > the
>
> > > >> > >>> > > token.
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl
> on
>
> > > who
>
> > > >> > can
>
> > > >> > >>> > > request
>
> > > >> > >>> > > > > > > > delegation
>
> > > >> > >>> > > > > > > > > >> tokens?
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > >
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend
>
> > people
>
> > > to
>
> > > >> > send
>
> > > >> > >>> > > > > delegation
>
> > > >> > >>> > > > > > > > tokens
>
> > > >> > >>> > > > > > > > > >> in an
>
> > > >> > >>> > > > > > > > > >> >> > > > > encrypted
>
> > > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
>
> > > >> > >>>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Manikumar <ma...@gmail.com>.
Hi,
I would like to reinitiate the discussion on Delegation token support for
Kafka.
Brief summary of the past discussion:
1) Broker stores delegation tokens in zookeeper. All brokers will have a
cache backed by
zookeeper so they will all get notified whenever a new token is
generated and they will
update their local cache whenever token state changes.
2) The current proposal does not support rotation of secret
3) Only allow the renewal by users that authenticated using *non*
delegation token mechanism
4) KIP-84 proposes to support SASL SCRAM mechanisms. Kafka clients can
authenticate using
SCRAM-SHA-256, providing the delegation token HMAC as password.
Updated the KIP with the following:
1. Protocol and Config changes
2. format of the data stored in ZK.
3. Changes to Java Clients/Usage of SASL SCRAM mechanism
https://cwiki.apache.org/confluence/display/KAFKA/KIP-
48+Delegation+token+support+for+Kafka
Jun, Ashish, Gwen,
Pl review the updated KIP.
Thanks
Manikumar
On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh <as...@cloudera.com> wrote:
> Harsha/ Gwen,
>
> How do we proceed here? I am willing to help out with here.
>
> On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Is it updated? are all concerns addressed? do you want to start a vote?
> >
> > Sorry for being pushy, I do appreciate that we are all volunteers and
> > finding time is difficult. This feature is important for anything that
> > integrates with Kafka (stream processors, Flume, NiFi, etc) and I
> > don't want to see this getting stuck because we lack coordination
> > within the community.
> >
> > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> > > The only pending update for the KIP is to write up the protocol changes
> > like
> > > we've it KIP-4. I'll update the wiki.
> > >
> > >
> > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com>
> > wrote:
> > >>
> > >> I think we decided to not support secret rotation, I guess this can be
> > >> stated clearly on the KIP. Also, more details on how clients will
> > perform
> > >> token distribution and how CLI will look like will be helpful.
> > >>
> > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > >>
> > >> > Hi Guys,
> > >> >
> > >> > This discussion was dead for a while. Are there still contentious
> > >> > points? If not, why are there no votes?
> > >> >
> > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> > >> > > Ashish,
> > >> > >
> > >> > > Yes, I will send out a KIP invite for next week to discuss KIP-48
> > and
> > >> > other
> > >> > > remaining KIPs.
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <
> asingh@cloudera.com>
> > >> > wrote:
> > >> > >
> > >> > >> Thanks Harsha!
> > >> > >>
> > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did
> > not
> > >> > >> actually make a call on when we should have next KIP call. As
> there
> > >> > >> are
> > >> > a
> > >> > >> few outstanding KIPs that could not be discussed this week, can
> we
> > >> > >> have
> > >> > a
> > >> > >> KIP hangout call next week?
> > >> > >>
> > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> > >> > >> <ka...@harsha.io>
> > >> > >> wrote:
> > >> > >>
> > >> > >>> Ashish,
> > >> > >>> Yes we are working on it. Lets discuss in the next KIP
> > >> > >>> meeting.
> > >> > >>> I'll join.
> > >> > >>> -Harsha
> > >> > >>>
> > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> > asingh@cloudera.com>
> > >> > >>> wrote:
> > >> > >>>
> > >> > >>> > Hello Harsha,
> > >> > >>> >
> > >> > >>> > Are you still working on this? Wondering if we can discuss
> this
> > in
> > >> > next
> > >> > >>> KIP
> > >> > >>> > meeting, if you can join.
> > >> > >>> >
> > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> > >> > kafka@harsha.io>
> > >> > >>> > wrote:
> > >> > >>> >
> > >> > >>> > > Hi Grant,
> > >> > >>> > > We are working on it. Will add the details to KIP
> > >> > >>> > > about
> > >> > the
> > >> > >>> > > request protocol.
> > >> > >>> > >
> > >> > >>> > > Thanks,
> > >> > >>> > > Harsha
> > >> > >>> > >
> > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> > >> > >>> > > <gh...@cloudera.com>
> > >> > >>> wrote:
> > >> > >>> > >
> > >> > >>> > > > Hi Parth,
> > >> > >>> > > >
> > >> > >>> > > > Are you still working on this? If you need any help please
> > >> > >>> > > > don't
> > >> > >>> > hesitate
> > >> > >>> > > > to ask.
> > >> > >>> > > >
> > >> > >>> > > > Thanks,
> > >> > >>> > > > Grant
> > >> > >>> > > >
> > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <
> jun@confluent.io>
> > >> > wrote:
> > >> > >>> > > >
> > >> > >>> > > > > Parth,
> > >> > >>> > > > >
> > >> > >>> > > > > Thanks for the reply.
> > >> > >>> > > > >
> > >> > >>> > > > > It makes sense to only allow the renewal by users that
> > >> > >>> authenticated
> > >> > >>> > > > using
> > >> > >>> > > > > *non* delegation token mechanism. Then, should we make
> the
> > >> > >>> renewal a
> > >> > >>> > > > list?
> > >> > >>> > > > > For example, in the case of rest proxy, it will be
> useful
> > >> > >>> > > > > for
> > >> > >>> every
> > >> > >>> > > > > instance of rest proxy to be able to renew the tokens.
> > >> > >>> > > > >
> > >> > >>> > > > > It would be clearer if we can document the request
> > protocol
> > >> > like
> > >> > >>> > > > >
> > >> > >>> > > > >
> > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >> > >>> > >
> > >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> > operations#KIP-4-
> > >> > >>> > > Commandlineandcentralizedadministrativeoperations-
> > >> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.
> > 10.1.0)
> > >> > >>> > > > > .
> > >> > >>> > > > >
> > >> > >>> > > > > It would also be useful to document the client APIs.
> > >> > >>> > > > >
> > >> > >>> > > > > Thanks,
> > >> > >>> > > > >
> > >> > >>> > > > > Jun
> > >> > >>> > > > >
> > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > >> > >>> > > > >
> > >> > >>> > > > > > Hi,
> > >> > >>> > > > > >
> > >> > >>> > > > > > I am suggesting that we will only allow the renewal by
> > >> > >>> > > > > > users
> > >> > >>> that
> > >> > >>> > > > > > authenticated using *non* delegation token mechanism.
> > For
> > >> > >>> example,
> > >> > >>> > If
> > >> > >>> > > > > user
> > >> > >>> > > > > > Alice authenticated using kerberos and requested
> > >> > >>> > > > > > delegation
> > >> > >>> tokens,
> > >> > >>> > > > only
> > >> > >>> > > > > > user Alice authenticated via non delegation token
> > >> > >>> > > > > > mechanism
> > >> > can
> > >> > >>> > > renew.
> > >> > >>> > > > > > Clients that have access to delegation tokens can not
> > >> > >>> > > > > > issue
> > >> > >>> > renewal
> > >> > >>> > > > > > request for renewing their own token and this is
> > primarily
> > >> > >>> > important
> > >> > >>> > > to
> > >> > >>> > > > > > reduce the time window for which a compromised token
> > will
> > >> > >>> > > > > > be
> > >> > >>> valid.
> > >> > >>> > > > > >
> > >> > >>> > > > > > To clarify, Yes any authenticated user can request
> > >> > >>> > > > > > delegation
> > >> > >>> > tokens
> > >> > >>> > > > but
> > >> > >>> > > > > > even here I would recommend to avoid creating a chain
> > >> > >>> > > > > > where a
> > >> > >>> > client
> > >> > >>> > > > > > authenticated via delegation token request for more
> > >> > delegation
> > >> > >>> > > tokens.
> > >> > >>> > > > > > Basically anyone can request delegation token, as long
> > as
> > >> > they
> > >> > >>> > > > > authenticate
> > >> > >>> > > > > > via a non delegation token mechanism.
> > >> > >>> > > > > >
> > >> > >>> > > > > > Aren't classes listed here
> > >> > >>> > > > > > <
> > >> > >>> > > > > >
> > >> > >>> > > > >
> > >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >> > >>> > > 48+Delegation+token+support+fo
> r+Kafka#KIP-48Delegationtokens
> > >> > >>> upportforKaf
> > >> > >>> > > ka-PublicInterfaces
> > >> > >>> > > > > > >
> > >> > >>> > > > > > sufficient?
> > >> > >>> > > > > >
> > >> > >>> > > > > > Thanks
> > >> > >>> > > > > > Parth
> > >> > >>> > > > > >
> > >> > >>> > > > > >
> > >> > >>> > > > > >
> > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> > >> > >>> > > > > > <ju...@confluent.io>
> > >> > >>> wrote:
> > >> > >>> > > > > >
> > >> > >>> > > > > > > Parth,
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > Thanks for the reply. A couple of comments inline
> > below.
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original
> > requester
> > >> > >>> only? or
> > >> > >>> > > > using
> > >> > >>> > > > > > > > Kerberos
> > >> > >>> > > > > > > > auth only?
> > >> > >>> > > > > > > > My recommendation is to do this only using
> Kerberos
> > >> > >>> > > > > > > > auth
> > >> > and
> > >> > >>> > only
> > >> > >>> > > > > threw
> > >> > >>> > > > > > > the
> > >> > >>> > > > > > > > renewer specified during the acquisition request.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying
> that
> > >> > >>> > > > > > > any
> > >> > >>> client
> > >> > >>> > > > > > > authenticated with the delegation token can renew,
> > i.e.
> > >> > there
> > >> > >>> is
> > >> > >>> > no
> > >> > >>> > > > > > renewer
> > >> > >>> > > > > > > needed?
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > Also, just to be clear, any authenticated client
> > (either
> > >> > >>> through
> > >> > >>> > > SASL
> > >> > >>> > > > > or
> > >> > >>> > > > > > > SSL) can request a delegation token for the
> > >> > >>> > > > > > > authenticated
> > >> > >>> user,
> > >> > >>> > > > right?
> > >> > >>> > > > > > >
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> > >> > >>> > > > > > > > My recommendation is still to store in ZK or not
> > store
> > >> > them
> > >> > >>> at
> > >> > >>> > > all.
> > >> > >>> > > > > The
> > >> > >>> > > > > > > > whole controller based distribution is too much
> > >> > >>> > > > > > > > overhead
> > >> > >>> with
> > >> > >>> > not
> > >> > >>> > > > > much
> > >> > >>> > > > > > to
> > >> > >>> > > > > > > > achieve.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> > >> > >>> > > > > > > > Either by expiration time out or through an
> explicit
> > >> > >>> request to
> > >> > >>> > > > > > > invalidate.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > >> > >>> > > > > > > > SCRAM
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't
> in
> > >> > >>> > > > > > > > the
> > >> > KIP
> > >> > >>> but
> > >> > >>> > > was
> > >> > >>> > > > > > > > discussed
> > >> > >>> > > > > > > > in this thread)?
> > >> > >>> > > > > > > > There is no imperonation proposal. I tried and
> > >> > >>> > > > > > > > explained
> > >> > how
> > >> > >>> > its
> > >> > >>> > > a
> > >> > >>> > > > > > > > different problem and why its not really necessary
> > to
> > >> > >>> discuss
> > >> > >>> > > that
> > >> > >>> > > > as
> > >> > >>> > > > > > > part
> > >> > >>> > > > > > > > of this KIP. This KIP will not support any
> > >> > impersonation,
> > >> > >>> it
> > >> > >>> > > will
> > >> > >>> > > > > just
> > >> > >>> > > > > > > be
> > >> > >>> > > > > > > > another way to authenticate.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> > >> > >>> > > > > > > > We do not need new ACLs.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > Could we document the format of the new
> > request/response
> > >> > and
> > >> > >>> > their
> > >> > >>> > > > > > > associated Resource and Operation for ACL?
> > >> > >>> > > > > > >
> > >> > >>> > > > > > >
> > >> > >>> > > > > > > > 7. How would the delegation token be configured in
> > the
> > >> > >>> client?
> > >> > >>> > > > > > > > Should be through config. I wasn't planning on
> > >> > >>> > > > > > > > supporting
> > >> > >>> JAAS
> > >> > >>> > > for
> > >> > >>> > > > > > > tokens.
> > >> > >>> > > > > > > > I don't believe hadoop does this either.
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > Thanks
> > >> > >>> > > > > > > > Parth
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> > >> > jun@confluent.io>
> > >> > >>> > > wrote:
> > >> > >>> > > > > > > >
> > >> > >>> > > > > > > > > Harsha,
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > Another question.
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > 9. How would the delegation token be configured
> in
> > >> > >>> > > > > > > > > the
> > >> > >>> > client?
> > >> > >>> > > > The
> > >> > >>> > > > > > > > standard
> > >> > >>> > > > > > > > > way is to do this through JAAS. However, we will
> > >> > >>> > > > > > > > > need
> > >> > to
> > >> > >>> > think
> > >> > >>> > > > > > through
> > >> > >>> > > > > > > if
> > >> > >>> > > > > > > > > this is convenient in a shared environment. For
> > >> > example,
> > >> > >>> > when a
> > >> > >>> > > > new
> > >> > >>> > > > > > > task
> > >> > >>> > > > > > > > is
> > >> > >>> > > > > > > > > added to a Storm worker node, do we need to
> > >> > >>> > > > > > > > > dynamically
> > >> > >>> add a
> > >> > >>> > > new
> > >> > >>> > > > > > > section
> > >> > >>> > > > > > > > > in the JAAS file? It may be more convenient if
> we
> > >> > >>> > > > > > > > > can
> > >> > >>> pass in
> > >> > >>> > > the
> > >> > >>> > > > > > token
> > >> > >>> > > > > > > > > through the config directly w/o going through
> > JAAS.
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > Are you or Parth still actively working on this
> > KIP?
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > Thanks,
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > Jun
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> > >> > >>> jun@confluent.io>
> > >> > >>> > > > wrote:
> > >> > >>> > > > > > > > >
> > >> > >>> > > > > > > > > > Just to add on that list.
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > > 2. It would be good to document the format of
> > the
> > >> > data
> > >> > >>> > stored
> > >> > >>> > > > in
> > >> > >>> > > > > > ZK.
> > >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether
> > the
> > >> > tokens
> > >> > >>> > > should
> > >> > >>> > > > > be
> > >> > >>> > > > > > > > > > propagated through ZK like config/acl/quota,
> or
> > >> > through
> > >> > >>> the
> > >> > >>> > > > > > > controller.
> > >> > >>> > > > > > > > > > Currently, the controller is only designed for
> > >> > >>> propagating
> > >> > >>> > > > topic
> > >> > >>> > > > > > > > > metadata,
> > >> > >>> > > > > > > > > > but not other data.
> > >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token
> instead
> > >> > >>> > > > > > > > > > of
> > >> > >>> > > DIGEST-MD5
> > >> > >>> > > > > > since
> > >> > >>> > > > > > > > it's
> > >> > >>> > > > > > > > > > deprecated?
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > > Thanks,
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > > Jun
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen
> Shapira <
> > >> > >>> > > > > gwen@confluent.io>
> > >> > >>> > > > > > > > > wrote:
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > >> From what I can see, remaining questions are:
> > >> > >>> > > > > > > > > >>
> > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
> > >> > requester
> > >> > >>> > only?
> > >> > >>> > > > or
> > >> > >>> > > > > > > using
> > >> > >>> > > > > > > > > >> Kerberos auth only?
> > >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > >> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> > >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> > >> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it
> > wasn't
> > >> > >>> > > > > > > > > >> in
> > >> > the
> > >> > >>> > KIP
> > >> > >>> > > > but
> > >> > >>> > > > > > was
> > >> > >>> > > > > > > > > >> discussed in this thread)?
> > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what
> > actions?
> > >> > >>> > > > > > > > > >>
> > >> > >>> > > > > > > > > >> Gwen
> > >> > >>> > > > > > > > > >>
> > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> > >> > >>> kafka@harsha.io>
> > >> > >>> > > > wrote:
> > >> > >>> > > > > > > > > >> > Jun & Ismael,
> > >> > >>> > > > > > > > > >> > Unfortunately I
> > >> > >>> > > > > > > > > >> > couldn't
> > >> > >>> attend
> > >> > >>> > > the
> > >> > >>> > > > > KIP
> > >> > >>> > > > > > > > > meeting
> > >> > >>> > > > > > > > > >> > when delegation
> > tokens
> > >> > >>> > discussed.
> > >> > >>> > > > > > > > Appreciate
> > >> > >>> > > > > > > > > if
> > >> > >>> > > > > > > > > >> > you can update the
> > >> > thread if
> > >> > >>> > you
> > >> > >>> > > > have
> > >> > >>> > > > > > any
> > >> > >>> > > > > > > > > >> > further questions.
> > >> > >>> > > > > > > > > >> > Thanks,
> > >> > >>> > > > > > > > > >> > Harsha
> > >> > >>> > > > > > > > > >> >
> > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan
> Pei
> > >> > wrote:
> > >> > >>> > > > > > > > > >> >> It seems that the links to images in the
> KIP
> > >> > >>> > > > > > > > > >> >> are
> > >> > >>> > broken.
> > >> > >>> > > > > > > > > >> >>
> > >> > >>> > > > > > > > > >> >> Liquan
> > >> > >>> > > > > > > > > >> >>
> > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> > >> > brahmbhatt <
> > >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > >> > >>> > > > > > > > > >> >>
> > >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs
> mean?
> > >> > >>> > > > > > > > > >> >> > In the current proposal we only allow a
> > user
> > >> > >>> > > > > > > > > >> >> > to
> > >> > >>> get
> > >> > >>> > > > > > delegation
> > >> > >>> > > > > > > > > token
> > >> > >>> > > > > > > > > >> for
> > >> > >>> > > > > > > > > >> >> > the identity that it authenticated as
> > using
> > >> > >>> another
> > >> > >>> > > > > > mechanism,
> > >> > >>> > > > > > > > i.e.
> > >> > >>> > > > > > > > > >> A user
> > >> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
> > >> > >>> > > > > > > > > >> >> > principal
> > >> > >>> > > > > > > user1@EXAMPLE.COM
> > >> > >>> > > > > > > > > >> will get
> > >> > >>> > > > > > > > > >> >> > delegation tokens for that user only. In
> > >> > future I
> > >> > >>> > think
> > >> > >>> > > > we
> > >> > >>> > > > > > will
> > >> > >>> > > > > > > > > have
> > >> > >>> > > > > > > > > >> to
> > >> > >>> > > > > > > > > >> >> > extend support such that we allow some
> set
> > >> > >>> > > > > > > > > >> >> > of
> > >> > >>> users (
> > >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > >> > >>> > storm-nimbus@EXAMPLE.COM)
> > >> > >>> > > > to
> > >> > >>> > > > > > > > acquire
> > >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other
> users
> > >> > whose
> > >> > >>> > > identity
> > >> > >>> > > > > > they
> > >> > >>> > > > > > > > have
> > >> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers
> > will
> > >> > have
> > >> > >>> ACLs
> > >> > >>> > > to
> > >> > >>> > > > > > > control
> > >> > >>> > > > > > > > > >> which
> > >> > >>> > > > > > > > > >> >> > users are allowed to impersonate other
> > users
> > >> > and
> > >> > >>> get
> > >> > >>> > > > tokens
> > >> > >>> > > > > > on
> > >> > >>> > > > > > > > > >> behalf of
> > >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
> > >> > different
> > >> > >>> > > problem
> > >> > >>> > > > in
> > >> > >>> > > > > > my
> > >> > >>> > > > > > > > > >> opinion and
> > >> > >>> > > > > > > > > >> >> > I think we can tackle it in separate
> KIP.
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting
> > and
> > >> > >>> renewing
> > >> > >>> > > > > > delegation
> > >> > >>> > > > > > > > > >> tokens?
> > >> > >>> > > > > > > > > >> >> > Typically this should be very very low,
> 1
> > >> > request
> > >> > >>> per
> > >> > >>> > > > > minute
> > >> > >>> > > > > > > is a
> > >> > >>> > > > > > > > > >> >> > relatively high estimate. However it
> > depends
> > >> > >>> > > > > > > > > >> >> > on
> > >> > >>> the
> > >> > >>> > > token
> > >> > >>> > > > > > > > > >> expiration. I am
> > >> > >>> > > > > > > > > >> >> > less worried about the extra load it
> puts
> > on
> > >> > >>> > controller
> > >> > >>> > > > vs
> > >> > >>> > > > > > the
> > >> > >>> > > > > > > > > added
> > >> > >>> > > > > > > > > >> >> > complexity and the value it offers.
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> > Thanks
> > >> > >>> > > > > > > > > >> >> > Parth
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael
> > Juma
> > >> > >>> > > > > > > > > >> >> > <
> > >> > >>> > > > > > > ismael@juma.me.uk>
> > >> > >>> > > > > > > > > >> wrote:
> > >> > >>> > > > > > > > > >> >> >
> > >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably
> > require a
> > >> > >>> separate
> > >> > >>> > > KIP
> > >> > >>> > > > > as
> > >> > >>> > > > > > it
> > >> > >>> > > > > > > > > will
> > >> > >>> > > > > > > > > >> >> > > introduce user visible changes. We
> could
> > >> > >>> > > > > > > > > >> >> > > also
> > >> > >>> > update
> > >> > >>> > > > > KIP-48
> > >> > >>> > > > > > > to
> > >> > >>> > > > > > > > > >> have this
> > >> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to
> do
> > it
> > >> > >>> > > separately.
> > >> > >>> > > > We
> > >> > >>> > > > > > can
> > >> > >>> > > > > > > > > >> discuss
> > >> > >>> > > > > > > > > >> >> > that
> > >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > >> > >>> > > > > > > > > >> >> > >
> > >> > >>> > > > > > > > > >> >> > > Ismael
> > >> > >>> > > > > > > > > >> >> > >
> > >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM,
> Rajini
> > >> > Sivaram
> > >> > >>> <
> > >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > >> > >>> > > > > > > > > >> >> > >
> > >> > >>> > > > > > > > > >> >> > > > Ismael,
> > >> > >>> > > > > > > > > >> >> > > >
> > >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > >> > jira/browse/KAFKA-3751)
> > >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL
> mechanism.
> > >> > >>> > > > > > > > > >> >> > > > Would
> > >> > >>> that
> > >> > >>> > > need
> > >> > >>> > > > > > > another
> > >> > >>> > > > > > > > > >> KIP? If
> > >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can
> > this
> > >> > just
> > >> > >>> be
> > >> > >>> > a
> > >> > >>> > > > JIRA
> > >> > >>> > > > > > > that
> > >> > >>> > > > > > > > > gets
> > >> > >>> > > > > > > > > >> >> > > reviewed
> > >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > >> > >>> > > > > > > > > >> >> > > >
> > >> > >>> > > > > > > > > >> >> > > > Thank you,
> > >> > >>> > > > > > > > > >> >> > > >
> > >> > >>> > > > > > > > > >> >> > > > Rajini
> > >> > >>> > > > > > > > > >> >> > > >
> > >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM,
> > Ismael
> > >> > Juma <
> > >> > >>> > > > > > > > > ismael@juma.me.uk>
> > >> > >>> > > > > > > > > >> >> > wrote:
> > >> > >>> > > > > > > > > >> >> > > >
> > >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a
> > good
> > >> > >>> > candidate.
> > >> > >>> > > > > > > > > >> >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned
> > this
> > >> > >>> > > > > > > > > >> >> > > > > as
> > >> > a
> > >> > >>> SASL
> > >> > >>> > > > > > mechanism
> > >> > >>> > > > > > > > > that
> > >> > >>> > > > > > > > > >> might
> > >> > >>> > > > > > > > > >> >> > be
> > >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been
> > >> > >>> > > > > > > > > >> >> > > > > meaning
> > >> > to
> > >> > >>> > check
> > >> > >>> > > > it
> > >> > >>> > > > > in
> > >> > >>> > > > > > > > more
> > >> > >>> > > > > > > > > >> detail.
> > >> > >>> > > > > > > > > >> >> > > Good
> > >> > >>> > > > > > > > > >> >> > > > > to know that you are willing to
> > >> > contribute
> > >> > >>> an
> > >> > >>> > > > > > > > implementation.
> > >> > >>> > > > > > > > > >> Maybe
> > >> > >>> > > > > > > > > >> >> > we
> > >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for
> > this?
> > >> > >>> > > > > > > > > >> >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > Ismael
> > >> > >>> > > > > > > > > >> >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM,
> > >> > >>> > > > > > > > > >> >> > > > > Rajini
> > >> > >>> > Sivaram <
> > >> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com>
> > wrote:
> > >> > >>> > > > > > > > > >> >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> > >> > >>> > Authentication
> > >> > >>> > > > > > > > Mechanism)
> > >> > >>> > > > > > > > > >> is a
> > >> > >>> > > > > > > > > >> >> > > better
> > >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java
> > >> > >>> > > > > > > > > >> >> > > > > > doesn't
> > >> > >>> come
> > >> > >>> > > > with a
> > >> > >>> > > > > > > > > built-in
> > >> > >>> > > > > > > > > >> SCRAM
> > >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I
> > will
> > >> > >>> > > > > > > > > >> >> > > > > > be
> > >> > >>> happy
> > >> > >>> > > to
> > >> > >>> > > > > add
> > >> > >>> > > > > > > > > support
> > >> > >>> > > > > > > > > >> in
> > >> > >>> > > > > > > > > >> >> > Kafka
> > >> > >>> > > > > > > > > >> >> > > > > since
> > >> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism
> to
> > >> > support
> > >> > >>> > > anyway.
> > >> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/
> > rfc7677
> > >> > >>> > describes
> > >> > >>> > > > the
> > >> > >>> > > > > > > > protocol
> > >> > >>> > > > > > > > > >> for
> > >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > >> > >>> > > > > > > > > >> >> > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM,
> > Jun
> > >> > Rao <
> > >> > >>> > > > > > > > jun@confluent.io
> > >> > >>> > > > > > > > > >
> > >> > >>> > > > > > > > > >> wrote:
> > >> > >>> > > > > > > > > >> >> > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A
> > >> > >>> > > > > > > > > >> >> > > > > > > couple
> > >> > of
> > >> > >>> > more
> > >> > >>> > > > > > > questions.
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> > >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> > >> > >>> mean?
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate
> of
> > >> > getting
> > >> > >>> and
> > >> > >>> > > > > > renewing
> > >> > >>> > > > > > > > > >> delegation
> > >> > >>> > > > > > > > > >> >> > > > tokens?
> > >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on
> > whether
> > >> > they
> > >> > >>> > > should
> > >> > >>> > > > be
> > >> > >>> > > > > > > > > directed
> > >> > >>> > > > > > > > > >> to the
> > >> > >>> > > > > > > > > >> >> > > > > > > controller.
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > Jun
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19
> PM,
> > >> > parth
> > >> > >>> > > > > brahmbhatt <
> > >> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com>
> > wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster
> > action
> > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >> > add
> > >> > >>> > acls
> > >> > >>> > > > on
> > >> > >>> > > > > > who
> > >> > >>> > > > > > > > can
> > >> > >>> > > > > > > > > >> request
> > >> > >>> > > > > > > > > >> >> > > > > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use
> > case
> > >> > for
> > >> > >>> that
> > >> > >>> > > yet
> > >> > >>> > > > > but
> > >> > >>> > > > > > > > down
> > >> > >>> > > > > > > > > >> the line
> > >> > >>> > > > > > > > > >> >> > > > when
> > >> > >>> > > > > > > > > >> >> > > > > we
> > >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > >> > getDelegationTokenAs
> > >> > >>> it
> > >> > >>> > > will
> > >> > >>> > > > > be
> > >> > >>> > > > > > > > > >> necessary.
> > >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to
> > be
> > >> > only
> > >> > >>> > > > > > > used/distributed
> > >> > >>> > > > > > > > > >> over
> > >> > >>> > > > > > > > > >> >> > secure
> > >> > >>> > > > > > > > > >> >> > > > > > > channels.
> > >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design
> we
> > >> > >>> > > > > > > > > >> >> > > > > > > > end
> > >> > up
> > >> > >>> > > choosing
> > >> > >>> > > > > > > > > >> Invalidation will
> > >> > >>> > > > > > > > > >> >> > > be
> > >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every
> broker
> > >> > >>> > > > > > > > > >> >> > > > > > > > or
> > >> > >>> > > controller.
> > >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
> > documented
> > >> > >>> somewhere
> > >> > >>> > > > that
> > >> > >>> > > > > > > > > >> invalidation
> > >> > >>> > > > > > > > > >> >> > will
> > >> > >>> > > > > > > > > >> >> > > > > > directly
> > >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but
> that
> > is
> > >> > not
> > >> > >>> the
> > >> > >>> > > > > intent.
> > >> > >>> > > > > > > > > >> Invalidation
> > >> > >>> > > > > > > > > >> >> > > will
> > >> > >>> > > > > > > > > >> >> > > > > > either
> > >> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to
> > >> > >>> expiration. No
> > >> > >>> > > > > direct
> > >> > >>> > > > > > > > > >> zookeeper
> > >> > >>> > > > > > > > > >> >> > > > > interaction
> > >> > >>> > > > > > > > > >> >> > > > > > > from
> > >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> > >> > >>> DelegationToken
> > >> > >>> > > > > without
> > >> > >>> > > > > > > the
> > >> > >>> > > > > > > > > >> hmac in
> > >> > >>> > > > > > > > > >> >> > the
> > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about
> the
> > >> > >>> confusion.
> > >> > >>> > > The
> > >> > >>> > > > > sole
> > >> > >>> > > > > > > > > >> purpose of
> > >> > >>> > > > > > > > > >> >> > > > > zookeeper
> > >> > >>> > > > > > > > > >> >> > > > > > in
> > >> > >>> > > > > > > > > >> >> > > > > > > > this design is as
> distribution
> > >> > channel
> > >> > >>> > for
> > >> > >>> > > > > tokens
> > >> > >>> > > > > > > > > >> between all
> > >> > >>> > > > > > > > > >> >> > > > brokers
> > >> > >>> > > > > > > > > >> >> > > > > > > and a
> > >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only
> tokens
> > >> > >>> > > > > > > > > >> >> > > > > > > > that
> > >> > >>> were
> > >> > >>> > > > > > generated
> > >> > >>> > > > > > > by
> > >> > >>> > > > > > > > > >> making a
> > >> > >>> > > > > > > > > >> >> > > > > request
> > >> > >>> > > > > > > > > >> >> > > > > > > to a
> > >> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted
> (more
> > on
> > >> > this
> > >> > >>> in
> > >> > >>> > > > second
> > >> > >>> > > > > > > > > >> paragraph). The
> > >> > >>> > > > > > > > > >> >> > > > token
> > >> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements
> > (owner,
> > >> > >>> renewer,
> > >> > >>> > > > uuid
> > >> > >>> > > > > ,
> > >> > >>> > > > > > > > > >> expiration,
> > >> > >>> > > > > > > > > >> >> > > hmac)
> > >> > >>> > > > > > > > > >> >> > > > ,
> > >> > >>> > > > > > > > > >> >> > > > > > one
> > >> > >>> > > > > > > > > >> >> > > > > > > of
> > >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally
> generated
> > >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> > >> > >>> but
> > >> > >>> > > hmac
> > >> > >>> > > > it
> > >> > >>> > > > > > > self
> > >> > >>> > > > > > > > is
> > >> > >>> > > > > > > > > >> >> > derivable
> > >> > >>> > > > > > > > > >> >> > > > if
> > >> > >>> > > > > > > > > >> >> > > > > > you
> > >> > >>> > > > > > > > > >> >> > > > > > > > have all the other elements
> of
> > >> > >>> > > > > > > > > >> >> > > > > > > > the
> > >> > >>> token
> > >> > >>> > +
> > >> > >>> > > > > secret
> > >> > >>> > > > > > > key
> > >> > >>> > > > > > > > > to
> > >> > >>> > > > > > > > > >> >> > generate
> > >> > >>> > > > > > > > > >> >> > > > > hmac.
> > >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not
> > provide
> > >> > SSL
> > >> > >>> > > support
> > >> > >>> > > > we
> > >> > >>> > > > > > do
> > >> > >>> > > > > > > > not
> > >> > >>> > > > > > > > > >> want the
> > >> > >>> > > > > > > > > >> >> > > > > entire
> > >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred
> > to
> > >> > >>> zookeeper
> > >> > >>> > > as
> > >> > >>> > > > > that
> > >> > >>> > > > > > > > will
> > >> > >>> > > > > > > > > >> be an
> > >> > >>> > > > > > > > > >> >> > > > insecure
> > >> > >>> > > > > > > > > >> >> > > > > > > wire
> > >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only
> > store
> > >> > >>> > > > > > > > > >> >> > > > > > > > all
> > >> > >>> the
> > >> > >>> > > other
> > >> > >>> > > > > > > > elements
> > >> > >>> > > > > > > > > >> of a
> > >> > >>> > > > > > > > > >> >> > > > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read
> these
> > >> > >>> elements
> > >> > >>> > and
> > >> > >>> > > > > > because
> > >> > >>> > > > > > > > > they
> > >> > >>> > > > > > > > > >> also
> > >> > >>> > > > > > > > > >> >> > > have
> > >> > >>> > > > > > > > > >> >> > > > > > access
> > >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be
> > able
> > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >> > >>> > generate
> > >> > >>> > > > > hmac
> > >> > >>> > > > > > on
> > >> > >>> > > > > > > > > >> their end.
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative
> > proposed
> > >> > >>> > > > > > > > > >> >> > > > > > > > is
> > >> > to
> > >> > >>> > avoid
> > >> > >>> > > > > > > zookeeper
> > >> > >>> > > > > > > > > >> >> > > altogether. A
> > >> > >>> > > > > > > > > >> >> > > > > > > Client
> > >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with
> required
> > >> > >>> > information
> > >> > >>> > > > > > (owner,
> > >> > >>> > > > > > > > > >> renwer,
> > >> > >>> > > > > > > > > >> >> > > > > expiration)
> > >> > >>> > > > > > > > > >> >> > > > > > > and
> > >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac,
> uuid).
> > >> > Broker
> > >> > >>> > won't
> > >> > >>> > > > > store
> > >> > >>> > > > > > > this
> > >> > >>> > > > > > > > > in
> > >> > >>> > > > > > > > > >> >> > > zookeeper.
> > >> > >>> > > > > > > > > >> >> > > > > > From
> > >> > >>> > > > > > > > > >> >> > > > > > > > this point a client can
> > contact
> > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > >> > >>> > broker
> > >> > >>> > > > with
> > >> > >>> > > > > > all
> > >> > >>> > > > > > > > the
> > >> > >>> > > > > > > > > >> >> > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > token
> > >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
> > expiration,
> > >> > hmac,
> > >> > >>> > > uuid)
> > >> > >>> > > > > the
> > >> > >>> > > > > > > > borker
> > >> > >>> > > > > > > > > >> will
> > >> > >>> > > > > > > > > >> >> > > > > regenerate
> > >> > >>> > > > > > > > > >> >> > > > > > > the
> > >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it
> matches
> > >> > >>> > > > > > > > > >> >> > > > > > > > with
> > >> > >>> hmac
> > >> > >>> > > > > > presented
> > >> > >>> > > > > > > by
> > >> > >>> > > > > > > > > >> client ,
> > >> > >>> > > > > > > > > >> >> > > > broker
> > >> > >>> > > > > > > > > >> >> > > > > > > will
> > >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> > >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> > >> > >>> Only
> > >> > >>> > > > > problem
> > >> > >>> > > > > > > with
> > >> > >>> > > > > > > > > >> this
> > >> > >>> > > > > > > > > >> >> > > approach
> > >> > >>> > > > > > > > > >> >> > > > > is
> > >> > >>> > > > > > > > > >> >> > > > > > if
> > >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is
> compromised
> > >> > >>> > > > > > > > > >> >> > > > > > > > any
> > >> > >>> client
> > >> > >>> > > can
> > >> > >>> > > > > now
> > >> > >>> > > > > > > > > generate
> > >> > >>> > > > > > > > > >> >> > random
> > >> > >>> > > > > > > > > >> >> > > > > tokens
> > >> > >>> > > > > > > > > >> >> > > > > > > and
> > >> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to
> > >> > >>> authenticate
> > >> > >>> > as
> > >> > >>> > > > any
> > >> > >>> > > > > > user
> > >> > >>> > > > > > > > > they
> > >> > >>> > > > > > > > > >> like.
> > >> > >>> > > > > > > > > >> >> > > with
> > >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that
> > only
> > >> > >>> tokens
> > >> > >>> > > > > acquired
> > >> > >>> > > > > > > via
> > >> > >>> > > > > > > > a
> > >> > >>> > > > > > > > > >> broker
> > >> > >>> > > > > > > > > >> >> > > > (using
> > >> > >>> > > > > > > > > >> >> > > > > > some
> > >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
> > >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> > >> > >>> token)
> > >> > >>> > > will
> > >> > >>> > > > > be
> > >> > >>> > > > > > > > > >> accepted. We
> > >> > >>> > > > > > > > > >> >> > > need
> > >> > >>> > > > > > > > > >> >> > > > to
> > >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes
> > >> > >>> > > > > > > > > >> >> > > > > > > > more
> > >> > >>> sense
> > >> > >>> > and
> > >> > >>> > > > we
> > >> > >>> > > > > > can
> > >> > >>> > > > > > > go
> > >> > >>> > > > > > > > > >> over it
> > >> > >>> > > > > > > > > >> >> > in
> > >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the
> > invite
> > >> > >>> > > > > > > > > >> >> > > > > > > > to
> > >> > >>> me?
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at
> 10:35
> > >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> > >> > Jun
> > >> > >>> > Rao <
> > >> > >>> > > > > > > > > >> jun@confluent.io>
> > >> > >>> > > > > > > > > >> >> > > > wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
> > >> > comments.
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can
> be
> > >> > useful
> > >> > >>> for
> > >> > >>> > > > Kafka
> > >> > >>> > > > > > > > Connect
> > >> > >>> > > > > > > > > >> and
> > >> > >>> > > > > > > > > >> >> > Kafka
> > >> > >>> > > > > > > > > >> >> > > > > rest
> > >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will
> > need
> > >> > >>> > > > > > > > > >> >> > > > > > > > > to
> > >> > >>> run a
> > >> > >>> > > > task
> > >> > >>> > > > > on
> > >> > >>> > > > > > > > > behalf
> > >> > >>> > > > > > > > > >> of a
> > >> > >>> > > > > > > > > >> >> > > > client.
> > >> > >>> > > > > > > > > >> >> > > > > > We
> > >> > >>> > > > > > > > > >> >> > > > > > > > will
> > >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how
> > >> > >>> > > > > > > > > >> >> > > > > > > > > those
> > >> > >>> > services
> > >> > >>> > > > use
> > >> > >>> > > > > > > Kafka
> > >> > >>> > > > > > > > > >> clients
> > >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer).
> Instead
> > >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> > >> > >>> > shared
> > >> > >>> > > > > client
> > >> > >>> > > > > > > per
> > >> > >>> > > > > > > > > >> worker,
> > >> > >>> > > > > > > > > >> >> > we
> > >> > >>> > > > > > > > > >> >> > > > will
> > >> > >>> > > > > > > > > >> >> > > > > > > need
> > >> > >>> > > > > > > > > >> >> > > > > > > > a
> > >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task since
> > the
> > >> > >>> > > > authentication
> > >> > >>> > > > > > > > happens
> > >> > >>> > > > > > > > > >> at the
> > >> > >>> > > > > > > > > >> >> > > > > > connection
> > >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect,
> > the
> > >> > >>> renewer
> > >> > >>> > > will
> > >> > >>> > > > be
> > >> > >>> > > > > > the
> > >> > >>> > > > > > > > > >> workers.
> > >> > >>> > > > > > > > > >> >> > So,
> > >> > >>> > > > > > > > > >> >> > > we
> > >> > >>> > > > > > > > > >> >> > > > > > > > probably
> > >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
> > >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> > >> > For
> > >> > >>> > > Kafka
> > >> > >>> > > > > rest
> > >> > >>> > > > > > > > > proxy,
> > >> > >>> > > > > > > > > >> the
> > >> > >>> > > > > > > > > >> >> > > > renewer
> > >> > >>> > > > > > > > > >> >> > > > > > can
> > >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the
> creator
> > >> > >>> > > > > > > > > >> >> > > > > > > > > of
> > >> > the
> > >> > >>> > > token.
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on
> > who
> > >> > can
> > >> > >>> > > request
> > >> > >>> > > > > > > > delegation
> > >> > >>> > > > > > > > > >> tokens?
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend
> people
> > to
> > >> > send
> > >> > >>> > > > > delegation
> > >> > >>> > > > > > > > tokens
> > >> > >>> > > > > > > > > >> in an
> > >> > >>> > > > > > > > > >> >> > > > > encrypted
> > >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible
> for
> > >> > expiring
> > >> > >>> > > > tokens,
> > >> > >>> > > > > > > every
> > >> > >>> > > > > > > > > >> broker?
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating
> > tokens,
> > >> > would
> > >> > >>> it
> > >> > >>> > be
> > >> > >>> > > > > > better
> > >> > >>> > > > > > > to
> > >> > >>> > > > > > > > > do
> > >> > >>> > > > > > > > > >> it in
> > >> > >>> > > > > > > > > >> >> > a
> > >> > >>> > > > > > > > > >> >> > > > > > request
> > >> > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK
> > >> > >>> > > > > > > > > >> >> > > > > > > > > directly?
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of
> > client
> > >> > >>> > > > > > > > > >> >> > > > > > > > > in
> > >> > >>> the
> > >> > >>> > > wiki
> > >> > >>> > > > > > > > sometimes
> > >> > >>> > > > > > > > > >> refers
> > >> > >>> > > > > > > > > >> >> > to
> > >> > >>> > > > > > > > > >> >> > > > the
> > >> > >>> > > > > > > > > >> >> > > > > > end
> > >> > >>> > > > > > > > > >> >> > > > > > > > > client and some other
> times
> > >> > refers
> > >> > >>> to
> > >> > >>> > the
> > >> > >>> > > > > > client
> > >> > >>> > > > > > > > > using
> > >> > >>> > > > > > > > > >> the
> > >> > >>> > > > > > > > > >> >> > > > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful
> > to
> > >> > >>> > distinguish
> > >> > >>> > > > > > between
> > >> > >>> > > > > > > > the
> > >> > >>> > > > > > > > > >> two.
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
> > >> > sentence
> > >> > >>> > > "Broker
> > >> > >>> > > > > > also
> > >> > >>> > > > > > > > > >> stores the
> > >> > >>> > > > > > > > > >> >> > > > > > > > DelegationToken
> > >> > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
> > >> > zookeeper."
> > >> > >>> a
> > >> > >>> > bit
> > >> > >>> > > > > > more? I
> > >> > >>> > > > > > > > > >> thought the
> > >> > >>> > > > > > > > > >> >> > > > > > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks,
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > Jun
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at
> 9:22
> > >> > >>> > > > > > > > > >> >> > > > > > > > > AM,
> > >> > Jun
> > >> > >>> > Rao
> > >> > >>> > > <
> > >> > >>> > > > > > > > > >> jun@confluent.io>
> > >> > >>> > > > > > > > > >> >> > > > wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP
> > meeting
> > >> > >>> invite.
> > >> > >>> > We
> > >> > >>> > > > can
> > >> > >>> > > > > > > > discuss
> > >> > >>> > > > > > > > > >> this in
> > >> > >>> > > > > > > > > >> >> > > the
> > >> > >>> > > > > > > > > >> >> > > > > > > meeting
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > Jun
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at
> > 8:47
> > >> > AM,
> > >> > >>> > > Harsha <
> > >> > >>> > > > > > > > > >> kafka@harsha.io>
> > >> > >>> > > > > > > > > >> >> > > > wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have
> a
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> KIP
> > >> > >>> meeting
> > >> > >>> > > > > around
> > >> > >>> > > > > > > > this.
> > >> > >>> > > > > > > > > >> The KIP
> > >> > >>> > > > > > > > > >> >> > is
> > >> > >>> > > > > > > > > >> >> > > > up
> > >> > >>> > > > > > > > > >> >> > > > > > for
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and
> > if
> > >> > there
> > >> > >>> are
> > >> > >>> > > any
> > >> > >>> > > > > > > > questions
> > >> > >>> > > > > > > > > >> lets
> > >> > >>> > > > > > > > > >> >> > > > quickly
> > >> > >>> > > > > > > > > >> >> > > > > > hash
> > >> > >>> > > > > > > > > >> >> > > > > > > > out
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> details.
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >>
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >>
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016,
> at
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> 08:40
> > >> > >>> AM,
> > >> > >>> > > parth
> > >> > >>> > > > > > > > > brahmbhatt
> > >> > >>> > > > > > > > > >> wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the
> hadoop
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > echo
> > >> > >>> > system
> > >> > >>> > > > uses
> > >> > >>> > > > > > so
> > >> > >>> > > > > > > no
> > >> > >>> > > > > > > > > >> good
> > >> > >>> > > > > > > > > >> >> > reason
> > >> > >>> > > > > > > > > >> >> > > > > > really.
> > >> > >>> > > > > > > > > >> >> > > > > > > > We
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > could
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever
> > is
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > the
> > >> > >>> > newest
> > >> > >>> > > > > > > > recommended
> > >> > >>> > > > > > > > > >> standard
> > >> > >>> > > > > > > > > >> >> > > is.
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016
> at
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > 3:33
> > >> > >>> AM,
> > >> > >>> > > > Ismael
> > >> > >>> > > > > > > Juma <
> > >> > >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> > >> > >>> > > > > > > > > >> >> > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the
> KIP. I
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > only
> > >> > >>> > started
> > >> > >>> > > > > > > reviewing
> > >> > >>> > > > > > > > > >> this and
> > >> > >>> > > > > > > > > >> >> > > may
> > >> > >>> > > > > > > > > >> >> > > > > have
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> additional
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later.
> The
> > >> > >>> immediate
> > >> > >>> > > > > question
> > >> > >>> > > > > > > that
> > >> > >>> > > > > > > > > >> came to
> > >> > >>> > > > > > > > > >> >> > > mind
> > >> > >>> > > > > > > > > >> >> > > > is
> > >> > >>> > > > > > > > > >> >> > > > > > our
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> choice of
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > though
> > >> > it's
> > >> > >>> > > marked
> > >> > >>> > > > > as
> > >> > >>> > > > > > > > > >> OBSOLETE in
> > >> > >>> > > > > > > > > >> >> > the
> > >> > >>> > > > > > > > > >> >> > > > IANA
> > >> > >>> > > > > > > > > >> >> > > > > > > > > Registry
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> of
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and
> > the
> > >> > >>> original
> > >> > >>> > > RFC
> > >> > >>> > > > > > > (2831)
> > >> > >>> > > > > > > > > has
> > >> > >>> > > > > > > > > >> been
> > >> > >>> > > > > > > > > >> >> > > moved
> > >> > >>> > > > > > > > > >> >> > > > > to
> > >> > >>> > > > > > > > > >> >> > > > > > > > > Historic
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > https://tools.ietf.org/html/
> > >> > >>> > > rfc6331
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > >
> > >> > >>> > > > > > > > > >> >> > >
> > >> > >>> > > > > > > > > >>
> > >> > >>> > > > > > >
> > >> > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
> > >> > >>> mechanisms.xhtml
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the
> reasoning
> > >> > behind
> > >> > >>> > that
> > >> > >>> > > > > > choice?
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13,
> 2016
> > at
> > >> > 11:29
> > >> > >>> > PM,
> > >> > >>> > > > Gwen
> > >> > >>> > > > > > > > > Shapira <
> > >> > >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> > >> > >>> > > > > > > > > >> >> > > > > > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments
> > inline
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > :)
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > emphasize
> > >> > >>> that
> > >> > >>> > > even
> > >> > >>> > > > > > though
> > >> > >>> > > > > > > > > >> delegation
> > >> > >>> > > > > > > > > >> >> > > > tokens
> > >> > >>> > > > > > > > > >> >> > > > > > > are a
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I
> > feel
> > >> > very
> > >> > >>> > > strongly
> > >> > >>> > > > > > about
> > >> > >>> > > > > > > > not
> > >> > >>> > > > > > > > > >> adding
> > >> > >>> > > > > > > > > >> >> > > > > > dependency
> > >> > >>> > > > > > > > > >> >> > > > > > > > on
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when
> implementing
> > >> > >>> delegation
> > >> > >>> > > > > tokens
> > >> > >>> > > > > > > for
> > >> > >>> > > > > > > > > >> Kafka. The
> > >> > >>> > > > > > > > > >> >> > > KIP
> > >> > >>> > > > > > > > > >> >> > > > > > > doesn't
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> imply
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such
> dependency,
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but
> > >> > if
> > >> > >>> you
> > >> > >>> > > can
> > >> > >>> > > > > > > > clarify...
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
> > >> > dependency.*
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add
> this
> > to
> > >> > the
> > >> > >>> KIP
> > >> > >>> > so
> > >> > >>> > > > no
> > >> > >>> > > > > > one
> > >> > >>> > > > > > > > will
> > >> > >>> > > > > > > > > >> read
> > >> > >>> > > > > > > > > >> >> > the
> > >> > >>> > > > > > > > > >> >> > > > KIP
> > >> > >>> > > > > > > > > >> >> > > > > > and
> > >> > >>> > > > > > > > > >> >> > > > > > > > > panic
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks
> before
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > the
> > >> > next
> > >> > >>> > > > > release...
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
> > >> > delegation
> > >> > >>> > token
> > >> > >>> > > at
> > >> > >>> > > > > any
> > >> > >>> > > > > > > > time
> > >> > >>> > > > > > > > > >> after
> > >> > >>> > > > > > > > > >> >> > > > > > > > authenticating?
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> only
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately
> > after?
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you
> > are
> > >> > >>> > > > authenticated
> > >> > >>> > > > > > you
> > >> > >>> > > > > > > > can
> > >> > >>> > > > > > > > > >> get
> > >> > >>> > > > > > > > > >> >> > > > delegation
> > >> > >>> > > > > > > > > >> >> > > > > > > > tokens.
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> We
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > need
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a
> > client
> > >> > >>> > > > authenticated
> > >> > >>> > > > > > > using
> > >> > >>> > > > > > > > > >> delegation
> > >> > >>> > > > > > > > > >> >> > > > > token,
> > >> > >>> > > > > > > > > >> >> > > > > > > can
> > >> > >>> > > > > > > > > >> >> > > > > > > > > also
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation
> token
> > >> > again or
> > >> > >>> > not.
> > >> > >>> > > > > Also
> > >> > >>> > > > > > > > there
> > >> > >>> > > > > > > > > >> is the
> > >> > >>> > > > > > > > > >> >> > > > > question
> > >> > >>> > > > > > > > > >> >> > > > > > of
> > >> > >>> > > > > > > > > >> >> > > > > > > > do
> > >> > >>> > > > > > > > > >> >> > > > > > > > > we
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
> > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to
> acquire
> > >> > >>> delegation
> > >> > >>> > > > token
> > >> > >>> > > > > > or
> > >> > >>> > > > > > > we
> > >> > >>> > > > > > > > > >> want
> > >> > >>> > > > > > > > > >> >> > > specific
> > >> > >>> > > > > > > > > >>
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>
>
>
> --
>
> Regards,
> Ashish
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ashish Singh <as...@cloudera.com>.
Harsha/ Gwen,
How do we proceed here? I am willing to help out with here.
On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Is it updated? are all concerns addressed? do you want to start a vote?
>
> Sorry for being pushy, I do appreciate that we are all volunteers and
> finding time is difficult. This feature is important for anything that
> integrates with Kafka (stream processors, Flume, NiFi, etc) and I
> don't want to see this getting stuck because we lack coordination
> within the community.
>
> On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
> > The only pending update for the KIP is to write up the protocol changes
> like
> > we've it KIP-4. I'll update the wiki.
> >
> >
> > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com>
> wrote:
> >>
> >> I think we decided to not support secret rotation, I guess this can be
> >> stated clearly on the KIP. Also, more details on how clients will
> perform
> >> token distribution and how CLI will look like will be helpful.
> >>
> >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >>
> >> > Hi Guys,
> >> >
> >> > This discussion was dead for a while. Are there still contentious
> >> > points? If not, why are there no votes?
> >> >
> >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> >> > > Ashish,
> >> > >
> >> > > Yes, I will send out a KIP invite for next week to discuss KIP-48
> and
> >> > other
> >> > > remaining KIPs.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com>
> >> > wrote:
> >> > >
> >> > >> Thanks Harsha!
> >> > >>
> >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did
> not
> >> > >> actually make a call on when we should have next KIP call. As there
> >> > >> are
> >> > a
> >> > >> few outstanding KIPs that could not be discussed this week, can we
> >> > >> have
> >> > a
> >> > >> KIP hangout call next week?
> >> > >>
> >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
> >> > >> <ka...@harsha.io>
> >> > >> wrote:
> >> > >>
> >> > >>> Ashish,
> >> > >>> Yes we are working on it. Lets discuss in the next KIP
> >> > >>> meeting.
> >> > >>> I'll join.
> >> > >>> -Harsha
> >> > >>>
> >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <
> asingh@cloudera.com>
> >> > >>> wrote:
> >> > >>>
> >> > >>> > Hello Harsha,
> >> > >>> >
> >> > >>> > Are you still working on this? Wondering if we can discuss this
> in
> >> > next
> >> > >>> KIP
> >> > >>> > meeting, if you can join.
> >> > >>> >
> >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> >> > kafka@harsha.io>
> >> > >>> > wrote:
> >> > >>> >
> >> > >>> > > Hi Grant,
> >> > >>> > > We are working on it. Will add the details to KIP
> >> > >>> > > about
> >> > the
> >> > >>> > > request protocol.
> >> > >>> > >
> >> > >>> > > Thanks,
> >> > >>> > > Harsha
> >> > >>> > >
> >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
> >> > >>> > > <gh...@cloudera.com>
> >> > >>> wrote:
> >> > >>> > >
> >> > >>> > > > Hi Parth,
> >> > >>> > > >
> >> > >>> > > > Are you still working on this? If you need any help please
> >> > >>> > > > don't
> >> > >>> > hesitate
> >> > >>> > > > to ask.
> >> > >>> > > >
> >> > >>> > > > Thanks,
> >> > >>> > > > Grant
> >> > >>> > > >
> >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io>
> >> > wrote:
> >> > >>> > > >
> >> > >>> > > > > Parth,
> >> > >>> > > > >
> >> > >>> > > > > Thanks for the reply.
> >> > >>> > > > >
> >> > >>> > > > > It makes sense to only allow the renewal by users that
> >> > >>> authenticated
> >> > >>> > > > using
> >> > >>> > > > > *non* delegation token mechanism. Then, should we make the
> >> > >>> renewal a
> >> > >>> > > > list?
> >> > >>> > > > > For example, in the case of rest proxy, it will be useful
> >> > >>> > > > > for
> >> > >>> every
> >> > >>> > > > > instance of rest proxy to be able to renew the tokens.
> >> > >>> > > > >
> >> > >>> > > > > It would be clearer if we can document the request
> protocol
> >> > like
> >> > >>> > > > >
> >> > >>> > > > >
> >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> > >>> > >
> >> > >>> > > 4+-+Command+line+and+centralized+administrative+
> operations#KIP-4-
> >> > >>> > > Commandlineandcentralizedadministrativeoperations-
> >> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.
> 10.1.0)
> >> > >>> > > > > .
> >> > >>> > > > >
> >> > >>> > > > > It would also be useful to document the client APIs.
> >> > >>> > > > >
> >> > >>> > > > > Thanks,
> >> > >>> > > > >
> >> > >>> > > > > Jun
> >> > >>> > > > >
> >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> >> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> >> > >>> > > > >
> >> > >>> > > > > > Hi,
> >> > >>> > > > > >
> >> > >>> > > > > > I am suggesting that we will only allow the renewal by
> >> > >>> > > > > > users
> >> > >>> that
> >> > >>> > > > > > authenticated using *non* delegation token mechanism.
> For
> >> > >>> example,
> >> > >>> > If
> >> > >>> > > > > user
> >> > >>> > > > > > Alice authenticated using kerberos and requested
> >> > >>> > > > > > delegation
> >> > >>> tokens,
> >> > >>> > > > only
> >> > >>> > > > > > user Alice authenticated via non delegation token
> >> > >>> > > > > > mechanism
> >> > can
> >> > >>> > > renew.
> >> > >>> > > > > > Clients that have access to delegation tokens can not
> >> > >>> > > > > > issue
> >> > >>> > renewal
> >> > >>> > > > > > request for renewing their own token and this is
> primarily
> >> > >>> > important
> >> > >>> > > to
> >> > >>> > > > > > reduce the time window for which a compromised token
> will
> >> > >>> > > > > > be
> >> > >>> valid.
> >> > >>> > > > > >
> >> > >>> > > > > > To clarify, Yes any authenticated user can request
> >> > >>> > > > > > delegation
> >> > >>> > tokens
> >> > >>> > > > but
> >> > >>> > > > > > even here I would recommend to avoid creating a chain
> >> > >>> > > > > > where a
> >> > >>> > client
> >> > >>> > > > > > authenticated via delegation token request for more
> >> > delegation
> >> > >>> > > tokens.
> >> > >>> > > > > > Basically anyone can request delegation token, as long
> as
> >> > they
> >> > >>> > > > > authenticate
> >> > >>> > > > > > via a non delegation token mechanism.
> >> > >>> > > > > >
> >> > >>> > > > > > Aren't classes listed here
> >> > >>> > > > > > <
> >> > >>> > > > > >
> >> > >>> > > > >
> >> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> > >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
> >> > >>> upportforKaf
> >> > >>> > > ka-PublicInterfaces
> >> > >>> > > > > > >
> >> > >>> > > > > > sufficient?
> >> > >>> > > > > >
> >> > >>> > > > > > Thanks
> >> > >>> > > > > > Parth
> >> > >>> > > > > >
> >> > >>> > > > > >
> >> > >>> > > > > >
> >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
> >> > >>> > > > > > <ju...@confluent.io>
> >> > >>> wrote:
> >> > >>> > > > > >
> >> > >>> > > > > > > Parth,
> >> > >>> > > > > > >
> >> > >>> > > > > > > Thanks for the reply. A couple of comments inline
> below.
> >> > >>> > > > > > >
> >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> >> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >> > >>> > > > > > >
> >> > >>> > > > > > > > 1. Who / how are tokens renewed? By original
> requester
> >> > >>> only? or
> >> > >>> > > > using
> >> > >>> > > > > > > > Kerberos
> >> > >>> > > > > > > > auth only?
> >> > >>> > > > > > > > My recommendation is to do this only using Kerberos
> >> > >>> > > > > > > > auth
> >> > and
> >> > >>> > only
> >> > >>> > > > > threw
> >> > >>> > > > > > > the
> >> > >>> > > > > > > > renewer specified during the acquisition request.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > >
> >> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying that
> >> > >>> > > > > > > any
> >> > >>> client
> >> > >>> > > > > > > authenticated with the delegation token can renew,
> i.e.
> >> > there
> >> > >>> is
> >> > >>> > no
> >> > >>> > > > > > renewer
> >> > >>> > > > > > > needed?
> >> > >>> > > > > > >
> >> > >>> > > > > > > Also, just to be clear, any authenticated client
> (either
> >> > >>> through
> >> > >>> > > SASL
> >> > >>> > > > > or
> >> > >>> > > > > > > SSL) can request a delegation token for the
> >> > >>> > > > > > > authenticated
> >> > >>> user,
> >> > >>> > > > right?
> >> > >>> > > > > > >
> >> > >>> > > > > > >
> >> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> >> > >>> > > > > > > > My recommendation is still to store in ZK or not
> store
> >> > them
> >> > >>> at
> >> > >>> > > all.
> >> > >>> > > > > The
> >> > >>> > > > > > > > whole controller based distribution is too much
> >> > >>> > > > > > > > overhead
> >> > >>> with
> >> > >>> > not
> >> > >>> > > > > much
> >> > >>> > > > > > to
> >> > >>> > > > > > > > achieve.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> >> > >>> > > > > > > > Either by expiration time out or through an explicit
> >> > >>> request to
> >> > >>> > > > > > > invalidate.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > 4. Which encryption algorithm is used?
> >> > >>> > > > > > > > SCRAM
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in
> >> > >>> > > > > > > > the
> >> > KIP
> >> > >>> but
> >> > >>> > > was
> >> > >>> > > > > > > > discussed
> >> > >>> > > > > > > > in this thread)?
> >> > >>> > > > > > > > There is no imperonation proposal. I tried and
> >> > >>> > > > > > > > explained
> >> > how
> >> > >>> > its
> >> > >>> > > a
> >> > >>> > > > > > > > different problem and why its not really necessary
> to
> >> > >>> discuss
> >> > >>> > > that
> >> > >>> > > > as
> >> > >>> > > > > > > part
> >> > >>> > > > > > > > of this KIP. This KIP will not support any
> >> > impersonation,
> >> > >>> it
> >> > >>> > > will
> >> > >>> > > > > just
> >> > >>> > > > > > > be
> >> > >>> > > > > > > > another way to authenticate.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> >> > >>> > > > > > > > We do not need new ACLs.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > >
> >> > >>> > > > > > > Could we document the format of the new
> request/response
> >> > and
> >> > >>> > their
> >> > >>> > > > > > > associated Resource and Operation for ACL?
> >> > >>> > > > > > >
> >> > >>> > > > > > >
> >> > >>> > > > > > > > 7. How would the delegation token be configured in
> the
> >> > >>> client?
> >> > >>> > > > > > > > Should be through config. I wasn't planning on
> >> > >>> > > > > > > > supporting
> >> > >>> JAAS
> >> > >>> > > for
> >> > >>> > > > > > > tokens.
> >> > >>> > > > > > > > I don't believe hadoop does this either.
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > Thanks
> >> > >>> > > > > > > > Parth
> >> > >>> > > > > > > >
> >> > >>> > > > > > > >
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> >> > jun@confluent.io>
> >> > >>> > > wrote:
> >> > >>> > > > > > > >
> >> > >>> > > > > > > > > Harsha,
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > Another question.
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > 9. How would the delegation token be configured in
> >> > >>> > > > > > > > > the
> >> > >>> > client?
> >> > >>> > > > The
> >> > >>> > > > > > > > standard
> >> > >>> > > > > > > > > way is to do this through JAAS. However, we will
> >> > >>> > > > > > > > > need
> >> > to
> >> > >>> > think
> >> > >>> > > > > > through
> >> > >>> > > > > > > if
> >> > >>> > > > > > > > > this is convenient in a shared environment. For
> >> > example,
> >> > >>> > when a
> >> > >>> > > > new
> >> > >>> > > > > > > task
> >> > >>> > > > > > > > is
> >> > >>> > > > > > > > > added to a Storm worker node, do we need to
> >> > >>> > > > > > > > > dynamically
> >> > >>> add a
> >> > >>> > > new
> >> > >>> > > > > > > section
> >> > >>> > > > > > > > > in the JAAS file? It may be more convenient if we
> >> > >>> > > > > > > > > can
> >> > >>> pass in
> >> > >>> > > the
> >> > >>> > > > > > token
> >> > >>> > > > > > > > > through the config directly w/o going through
> JAAS.
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > Are you or Parth still actively working on this
> KIP?
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > Thanks,
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > Jun
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> >> > >>> jun@confluent.io>
> >> > >>> > > > wrote:
> >> > >>> > > > > > > > >
> >> > >>> > > > > > > > > > Just to add on that list.
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > > 2. It would be good to document the format of
> the
> >> > data
> >> > >>> > stored
> >> > >>> > > > in
> >> > >>> > > > > > ZK.
> >> > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether
> the
> >> > tokens
> >> > >>> > > should
> >> > >>> > > > > be
> >> > >>> > > > > > > > > > propagated through ZK like config/acl/quota, or
> >> > through
> >> > >>> the
> >> > >>> > > > > > > controller.
> >> > >>> > > > > > > > > > Currently, the controller is only designed for
> >> > >>> propagating
> >> > >>> > > > topic
> >> > >>> > > > > > > > > metadata,
> >> > >>> > > > > > > > > > but not other data.
> >> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead
> >> > >>> > > > > > > > > > of
> >> > >>> > > DIGEST-MD5
> >> > >>> > > > > > since
> >> > >>> > > > > > > > it's
> >> > >>> > > > > > > > > > deprecated?
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > > Thanks,
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > > Jun
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> >> > >>> > > > > gwen@confluent.io>
> >> > >>> > > > > > > > > wrote:
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > >> From what I can see, remaining questions are:
> >> > >>> > > > > > > > > >>
> >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
> >> > requester
> >> > >>> > only?
> >> > >>> > > > or
> >> > >>> > > > > > > using
> >> > >>> > > > > > > > > >> Kerberos auth only?
> >> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> >> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> >> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> >> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it
> wasn't
> >> > >>> > > > > > > > > >> in
> >> > the
> >> > >>> > KIP
> >> > >>> > > > but
> >> > >>> > > > > > was
> >> > >>> > > > > > > > > >> discussed in this thread)?
> >> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what
> actions?
> >> > >>> > > > > > > > > >>
> >> > >>> > > > > > > > > >> Gwen
> >> > >>> > > > > > > > > >>
> >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> >> > >>> kafka@harsha.io>
> >> > >>> > > > wrote:
> >> > >>> > > > > > > > > >> > Jun & Ismael,
> >> > >>> > > > > > > > > >> > Unfortunately I
> >> > >>> > > > > > > > > >> > couldn't
> >> > >>> attend
> >> > >>> > > the
> >> > >>> > > > > KIP
> >> > >>> > > > > > > > > meeting
> >> > >>> > > > > > > > > >> > when delegation
> tokens
> >> > >>> > discussed.
> >> > >>> > > > > > > > Appreciate
> >> > >>> > > > > > > > > if
> >> > >>> > > > > > > > > >> > you can update the
> >> > thread if
> >> > >>> > you
> >> > >>> > > > have
> >> > >>> > > > > > any
> >> > >>> > > > > > > > > >> > further questions.
> >> > >>> > > > > > > > > >> > Thanks,
> >> > >>> > > > > > > > > >> > Harsha
> >> > >>> > > > > > > > > >> >
> >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei
> >> > wrote:
> >> > >>> > > > > > > > > >> >> It seems that the links to images in the KIP
> >> > >>> > > > > > > > > >> >> are
> >> > >>> > broken.
> >> > >>> > > > > > > > > >> >>
> >> > >>> > > > > > > > > >> >> Liquan
> >> > >>> > > > > > > > > >> >>
> >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> >> > brahmbhatt <
> >> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> >> > >>> > > > > > > > > >> >>
> >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> >> > >>> > > > > > > > > >> >> > In the current proposal we only allow a
> user
> >> > >>> > > > > > > > > >> >> > to
> >> > >>> get
> >> > >>> > > > > > delegation
> >> > >>> > > > > > > > > token
> >> > >>> > > > > > > > > >> for
> >> > >>> > > > > > > > > >> >> > the identity that it authenticated as
> using
> >> > >>> another
> >> > >>> > > > > > mechanism,
> >> > >>> > > > > > > > i.e.
> >> > >>> > > > > > > > > >> A user
> >> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
> >> > >>> > > > > > > > > >> >> > principal
> >> > >>> > > > > > > user1@EXAMPLE.COM
> >> > >>> > > > > > > > > >> will get
> >> > >>> > > > > > > > > >> >> > delegation tokens for that user only. In
> >> > future I
> >> > >>> > think
> >> > >>> > > > we
> >> > >>> > > > > > will
> >> > >>> > > > > > > > > have
> >> > >>> > > > > > > > > >> to
> >> > >>> > > > > > > > > >> >> > extend support such that we allow some set
> >> > >>> > > > > > > > > >> >> > of
> >> > >>> users (
> >> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> >> > >>> > storm-nimbus@EXAMPLE.COM)
> >> > >>> > > > to
> >> > >>> > > > > > > > acquire
> >> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other users
> >> > whose
> >> > >>> > > identity
> >> > >>> > > > > > they
> >> > >>> > > > > > > > have
> >> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers
> will
> >> > have
> >> > >>> ACLs
> >> > >>> > > to
> >> > >>> > > > > > > control
> >> > >>> > > > > > > > > >> which
> >> > >>> > > > > > > > > >> >> > users are allowed to impersonate other
> users
> >> > and
> >> > >>> get
> >> > >>> > > > tokens
> >> > >>> > > > > > on
> >> > >>> > > > > > > > > >> behalf of
> >> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
> >> > different
> >> > >>> > > problem
> >> > >>> > > > in
> >> > >>> > > > > > my
> >> > >>> > > > > > > > > >> opinion and
> >> > >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting
> and
> >> > >>> renewing
> >> > >>> > > > > > delegation
> >> > >>> > > > > > > > > >> tokens?
> >> > >>> > > > > > > > > >> >> > Typically this should be very very low, 1
> >> > request
> >> > >>> per
> >> > >>> > > > > minute
> >> > >>> > > > > > > is a
> >> > >>> > > > > > > > > >> >> > relatively high estimate. However it
> depends
> >> > >>> > > > > > > > > >> >> > on
> >> > >>> the
> >> > >>> > > token
> >> > >>> > > > > > > > > >> expiration. I am
> >> > >>> > > > > > > > > >> >> > less worried about the extra load it puts
> on
> >> > >>> > controller
> >> > >>> > > > vs
> >> > >>> > > > > > the
> >> > >>> > > > > > > > > added
> >> > >>> > > > > > > > > >> >> > complexity and the value it offers.
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> > Thanks
> >> > >>> > > > > > > > > >> >> > Parth
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael
> Juma
> >> > >>> > > > > > > > > >> >> > <
> >> > >>> > > > > > > ismael@juma.me.uk>
> >> > >>> > > > > > > > > >> wrote:
> >> > >>> > > > > > > > > >> >> >
> >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably
> require a
> >> > >>> separate
> >> > >>> > > KIP
> >> > >>> > > > > as
> >> > >>> > > > > > it
> >> > >>> > > > > > > > > will
> >> > >>> > > > > > > > > >> >> > > introduce user visible changes. We could
> >> > >>> > > > > > > > > >> >> > > also
> >> > >>> > update
> >> > >>> > > > > KIP-48
> >> > >>> > > > > > > to
> >> > >>> > > > > > > > > >> have this
> >> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to do
> it
> >> > >>> > > separately.
> >> > >>> > > > We
> >> > >>> > > > > > can
> >> > >>> > > > > > > > > >> discuss
> >> > >>> > > > > > > > > >> >> > that
> >> > >>> > > > > > > > > >> >> > > in the KIP call today.
> >> > >>> > > > > > > > > >> >> > >
> >> > >>> > > > > > > > > >> >> > > Ismael
> >> > >>> > > > > > > > > >> >> > >
> >> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini
> >> > Sivaram
> >> > >>> <
> >> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> >> > >>> > > > > > > > > >> >> > >
> >> > >>> > > > > > > > > >> >> > > > Ismael,
> >> > >>> > > > > > > > > >> >> > > >
> >> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> >> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> >> > jira/browse/KAFKA-3751)
> >> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism.
> >> > >>> > > > > > > > > >> >> > > > Would
> >> > >>> that
> >> > >>> > > need
> >> > >>> > > > > > > another
> >> > >>> > > > > > > > > >> KIP? If
> >> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can
> this
> >> > just
> >> > >>> be
> >> > >>> > a
> >> > >>> > > > JIRA
> >> > >>> > > > > > > that
> >> > >>> > > > > > > > > gets
> >> > >>> > > > > > > > > >> >> > > reviewed
> >> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> >> > >>> > > > > > > > > >> >> > > >
> >> > >>> > > > > > > > > >> >> > > > Thank you,
> >> > >>> > > > > > > > > >> >> > > >
> >> > >>> > > > > > > > > >> >> > > > Rajini
> >> > >>> > > > > > > > > >> >> > > >
> >> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM,
> Ismael
> >> > Juma <
> >> > >>> > > > > > > > > ismael@juma.me.uk>
> >> > >>> > > > > > > > > >> >> > wrote:
> >> > >>> > > > > > > > > >> >> > > >
> >> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a
> good
> >> > >>> > candidate.
> >> > >>> > > > > > > > > >> >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned
> this
> >> > >>> > > > > > > > > >> >> > > > > as
> >> > a
> >> > >>> SASL
> >> > >>> > > > > > mechanism
> >> > >>> > > > > > > > > that
> >> > >>> > > > > > > > > >> might
> >> > >>> > > > > > > > > >> >> > be
> >> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been
> >> > >>> > > > > > > > > >> >> > > > > meaning
> >> > to
> >> > >>> > check
> >> > >>> > > > it
> >> > >>> > > > > in
> >> > >>> > > > > > > > more
> >> > >>> > > > > > > > > >> detail.
> >> > >>> > > > > > > > > >> >> > > Good
> >> > >>> > > > > > > > > >> >> > > > > to know that you are willing to
> >> > contribute
> >> > >>> an
> >> > >>> > > > > > > > implementation.
> >> > >>> > > > > > > > > >> Maybe
> >> > >>> > > > > > > > > >> >> > we
> >> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for
> this?
> >> > >>> > > > > > > > > >> >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > Ismael
> >> > >>> > > > > > > > > >> >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM,
> >> > >>> > > > > > > > > >> >> > > > > Rajini
> >> > >>> > Sivaram <
> >> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com>
> wrote:
> >> > >>> > > > > > > > > >> >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> >> > >>> > Authentication
> >> > >>> > > > > > > > Mechanism)
> >> > >>> > > > > > > > > >> is a
> >> > >>> > > > > > > > > >> >> > > better
> >> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java
> >> > >>> > > > > > > > > >> >> > > > > > doesn't
> >> > >>> come
> >> > >>> > > > with a
> >> > >>> > > > > > > > > built-in
> >> > >>> > > > > > > > > >> SCRAM
> >> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I
> will
> >> > >>> > > > > > > > > >> >> > > > > > be
> >> > >>> happy
> >> > >>> > > to
> >> > >>> > > > > add
> >> > >>> > > > > > > > > support
> >> > >>> > > > > > > > > >> in
> >> > >>> > > > > > > > > >> >> > Kafka
> >> > >>> > > > > > > > > >> >> > > > > since
> >> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to
> >> > support
> >> > >>> > > anyway.
> >> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/
> rfc7677
> >> > >>> > describes
> >> > >>> > > > the
> >> > >>> > > > > > > > protocol
> >> > >>> > > > > > > > > >> for
> >> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> >> > >>> > > > > > > > > >> >> > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM,
> Jun
> >> > Rao <
> >> > >>> > > > > > > > jun@confluent.io
> >> > >>> > > > > > > > > >
> >> > >>> > > > > > > > > >> wrote:
> >> > >>> > > > > > > > > >> >> > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > Parth,
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A
> >> > >>> > > > > > > > > >> >> > > > > > > couple
> >> > of
> >> > >>> > more
> >> > >>> > > > > > > questions.
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > 110. What does
> >> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
> >> > >>> mean?
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of
> >> > getting
> >> > >>> and
> >> > >>> > > > > > renewing
> >> > >>> > > > > > > > > >> delegation
> >> > >>> > > > > > > > > >> >> > > > tokens?
> >> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on
> whether
> >> > they
> >> > >>> > > should
> >> > >>> > > > be
> >> > >>> > > > > > > > > directed
> >> > >>> > > > > > > > > >> to the
> >> > >>> > > > > > > > > >> >> > > > > > > controller.
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > Jun
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM,
> >> > parth
> >> > >>> > > > > brahmbhatt <
> >> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com>
> wrote:
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster
> action
> >> > >>> > > > > > > > > >> >> > > > > > > > to
> >> > add
> >> > >>> > acls
> >> > >>> > > > on
> >> > >>> > > > > > who
> >> > >>> > > > > > > > can
> >> > >>> > > > > > > > > >> request
> >> > >>> > > > > > > > > >> >> > > > > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use
> case
> >> > for
> >> > >>> that
> >> > >>> > > yet
> >> > >>> > > > > but
> >> > >>> > > > > > > > down
> >> > >>> > > > > > > > > >> the line
> >> > >>> > > > > > > > > >> >> > > > when
> >> > >>> > > > > > > > > >> >> > > > > we
> >> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> >> > getDelegationTokenAs
> >> > >>> it
> >> > >>> > > will
> >> > >>> > > > > be
> >> > >>> > > > > > > > > >> necessary.
> >> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to
> be
> >> > only
> >> > >>> > > > > > > used/distributed
> >> > >>> > > > > > > > > >> over
> >> > >>> > > > > > > > > >> >> > secure
> >> > >>> > > > > > > > > >> >> > > > > > > channels.
> >> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we
> >> > >>> > > > > > > > > >> >> > > > > > > > end
> >> > up
> >> > >>> > > choosing
> >> > >>> > > > > > > > > >> Invalidation will
> >> > >>> > > > > > > > > >> >> > > be
> >> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker
> >> > >>> > > > > > > > > >> >> > > > > > > > or
> >> > >>> > > controller.
> >> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I
> documented
> >> > >>> somewhere
> >> > >>> > > > that
> >> > >>> > > > > > > > > >> invalidation
> >> > >>> > > > > > > > > >> >> > will
> >> > >>> > > > > > > > > >> >> > > > > > directly
> >> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that
> is
> >> > not
> >> > >>> the
> >> > >>> > > > > intent.
> >> > >>> > > > > > > > > >> Invalidation
> >> > >>> > > > > > > > > >> >> > > will
> >> > >>> > > > > > > > > >> >> > > > > > either
> >> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to
> >> > >>> expiration. No
> >> > >>> > > > > direct
> >> > >>> > > > > > > > > >> zookeeper
> >> > >>> > > > > > > > > >> >> > > > > interaction
> >> > >>> > > > > > > > > >> >> > > > > > > from
> >> > >>> > > > > > > > > >> >> > > > > > > > any client.
> >> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> >> > >>> DelegationToken
> >> > >>> > > > > without
> >> > >>> > > > > > > the
> >> > >>> > > > > > > > > >> hmac in
> >> > >>> > > > > > > > > >> >> > the
> >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
> >> > >>> confusion.
> >> > >>> > > The
> >> > >>> > > > > sole
> >> > >>> > > > > > > > > >> purpose of
> >> > >>> > > > > > > > > >> >> > > > > zookeeper
> >> > >>> > > > > > > > > >> >> > > > > > in
> >> > >>> > > > > > > > > >> >> > > > > > > > this design is as distribution
> >> > channel
> >> > >>> > for
> >> > >>> > > > > tokens
> >> > >>> > > > > > > > > >> between all
> >> > >>> > > > > > > > > >> >> > > > brokers
> >> > >>> > > > > > > > > >> >> > > > > > > and a
> >> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens
> >> > >>> > > > > > > > > >> >> > > > > > > > that
> >> > >>> were
> >> > >>> > > > > > generated
> >> > >>> > > > > > > by
> >> > >>> > > > > > > > > >> making a
> >> > >>> > > > > > > > > >> >> > > > > request
> >> > >>> > > > > > > > > >> >> > > > > > > to a
> >> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more
> on
> >> > this
> >> > >>> in
> >> > >>> > > > second
> >> > >>> > > > > > > > > >> paragraph). The
> >> > >>> > > > > > > > > >> >> > > > token
> >> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements
> (owner,
> >> > >>> renewer,
> >> > >>> > > > uuid
> >> > >>> > > > > ,
> >> > >>> > > > > > > > > >> expiration,
> >> > >>> > > > > > > > > >> >> > > hmac)
> >> > >>> > > > > > > > > >> >> > > > ,
> >> > >>> > > > > > > > > >> >> > > > > > one
> >> > >>> > > > > > > > > >> >> > > > > > > of
> >> > >>> > > > > > > > > >> >> > > > > > > > which is the finally generated
> >> > >>> > > > > > > > > >> >> > > > > > > > hmac
> >> > >>> but
> >> > >>> > > hmac
> >> > >>> > > > it
> >> > >>> > > > > > > self
> >> > >>> > > > > > > > is
> >> > >>> > > > > > > > > >> >> > derivable
> >> > >>> > > > > > > > > >> >> > > > if
> >> > >>> > > > > > > > > >> >> > > > > > you
> >> > >>> > > > > > > > > >> >> > > > > > > > have all the other elements of
> >> > >>> > > > > > > > > >> >> > > > > > > > the
> >> > >>> token
> >> > >>> > +
> >> > >>> > > > > secret
> >> > >>> > > > > > > key
> >> > >>> > > > > > > > > to
> >> > >>> > > > > > > > > >> >> > generate
> >> > >>> > > > > > > > > >> >> > > > > hmac.
> >> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not
> provide
> >> > SSL
> >> > >>> > > support
> >> > >>> > > > we
> >> > >>> > > > > > do
> >> > >>> > > > > > > > not
> >> > >>> > > > > > > > > >> want the
> >> > >>> > > > > > > > > >> >> > > > > entire
> >> > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred
> to
> >> > >>> zookeeper
> >> > >>> > > as
> >> > >>> > > > > that
> >> > >>> > > > > > > > will
> >> > >>> > > > > > > > > >> be an
> >> > >>> > > > > > > > > >> >> > > > insecure
> >> > >>> > > > > > > > > >> >> > > > > > > wire
> >> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only
> store
> >> > >>> > > > > > > > > >> >> > > > > > > > all
> >> > >>> the
> >> > >>> > > other
> >> > >>> > > > > > > > elements
> >> > >>> > > > > > > > > >> of a
> >> > >>> > > > > > > > > >> >> > > > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
> >> > >>> elements
> >> > >>> > and
> >> > >>> > > > > > because
> >> > >>> > > > > > > > > they
> >> > >>> > > > > > > > > >> also
> >> > >>> > > > > > > > > >> >> > > have
> >> > >>> > > > > > > > > >> >> > > > > > access
> >> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be
> able
> >> > >>> > > > > > > > > >> >> > > > > > > > to
> >> > >>> > generate
> >> > >>> > > > > hmac
> >> > >>> > > > > > on
> >> > >>> > > > > > > > > >> their end.
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative
> proposed
> >> > >>> > > > > > > > > >> >> > > > > > > > is
> >> > to
> >> > >>> > avoid
> >> > >>> > > > > > > zookeeper
> >> > >>> > > > > > > > > >> >> > > altogether. A
> >> > >>> > > > > > > > > >> >> > > > > > > Client
> >> > >>> > > > > > > > > >> >> > > > > > > > will call broker with required
> >> > >>> > information
> >> > >>> > > > > > (owner,
> >> > >>> > > > > > > > > >> renwer,
> >> > >>> > > > > > > > > >> >> > > > > expiration)
> >> > >>> > > > > > > > > >> >> > > > > > > and
> >> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid).
> >> > Broker
> >> > >>> > won't
> >> > >>> > > > > store
> >> > >>> > > > > > > this
> >> > >>> > > > > > > > > in
> >> > >>> > > > > > > > > >> >> > > zookeeper.
> >> > >>> > > > > > > > > >> >> > > > > > From
> >> > >>> > > > > > > > > >> >> > > > > > > > this point a client can
> contact
> >> > >>> > > > > > > > > >> >> > > > > > > > any
> >> > >>> > broker
> >> > >>> > > > with
> >> > >>> > > > > > all
> >> > >>> > > > > > > > the
> >> > >>> > > > > > > > > >> >> > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > token
> >> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner,
> expiration,
> >> > hmac,
> >> > >>> > > uuid)
> >> > >>> > > > > the
> >> > >>> > > > > > > > borker
> >> > >>> > > > > > > > > >> will
> >> > >>> > > > > > > > > >> >> > > > > regenerate
> >> > >>> > > > > > > > > >> >> > > > > > > the
> >> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches
> >> > >>> > > > > > > > > >> >> > > > > > > > with
> >> > >>> hmac
> >> > >>> > > > > > presented
> >> > >>> > > > > > > by
> >> > >>> > > > > > > > > >> client ,
> >> > >>> > > > > > > > > >> >> > > > broker
> >> > >>> > > > > > > > > >> >> > > > > > > will
> >> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> >> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
> >> > >>> Only
> >> > >>> > > > > problem
> >> > >>> > > > > > > with
> >> > >>> > > > > > > > > >> this
> >> > >>> > > > > > > > > >> >> > > approach
> >> > >>> > > > > > > > > >> >> > > > > is
> >> > >>> > > > > > > > > >> >> > > > > > if
> >> > >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised
> >> > >>> > > > > > > > > >> >> > > > > > > > any
> >> > >>> client
> >> > >>> > > can
> >> > >>> > > > > now
> >> > >>> > > > > > > > > generate
> >> > >>> > > > > > > > > >> >> > random
> >> > >>> > > > > > > > > >> >> > > > > tokens
> >> > >>> > > > > > > > > >> >> > > > > > > and
> >> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to
> >> > >>> authenticate
> >> > >>> > as
> >> > >>> > > > any
> >> > >>> > > > > > user
> >> > >>> > > > > > > > > they
> >> > >>> > > > > > > > > >> like.
> >> > >>> > > > > > > > > >> >> > > with
> >> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that
> only
> >> > >>> tokens
> >> > >>> > > > > acquired
> >> > >>> > > > > > > via
> >> > >>> > > > > > > > a
> >> > >>> > > > > > > > > >> broker
> >> > >>> > > > > > > > > >> >> > > > (using
> >> > >>> > > > > > > > > >> >> > > > > > some
> >> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
> >> > >>> > > > > > > > > >> >> > > > > > > > delegation
> >> > >>> token)
> >> > >>> > > will
> >> > >>> > > > > be
> >> > >>> > > > > > > > > >> accepted. We
> >> > >>> > > > > > > > > >> >> > > need
> >> > >>> > > > > > > > > >> >> > > > to
> >> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes
> >> > >>> > > > > > > > > >> >> > > > > > > > more
> >> > >>> sense
> >> > >>> > and
> >> > >>> > > > we
> >> > >>> > > > > > can
> >> > >>> > > > > > > go
> >> > >>> > > > > > > > > >> over it
> >> > >>> > > > > > > > > >> >> > in
> >> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> >> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the
> invite
> >> > >>> > > > > > > > > >> >> > > > > > > > to
> >> > >>> me?
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> >> > >>> > > > > > > > > >> >> > > > > > > > Parth
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35
> >> > >>> > > > > > > > > >> >> > > > > > > > AM,
> >> > Jun
> >> > >>> > Rao <
> >> > >>> > > > > > > > > >> jun@confluent.io>
> >> > >>> > > > > > > > > >> >> > > > wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
> >> > comments.
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be
> >> > useful
> >> > >>> for
> >> > >>> > > > Kafka
> >> > >>> > > > > > > > Connect
> >> > >>> > > > > > > > > >> and
> >> > >>> > > > > > > > > >> >> > Kafka
> >> > >>> > > > > > > > > >> >> > > > > rest
> >> > >>> > > > > > > > > >> >> > > > > > > > proxy
> >> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will
> need
> >> > >>> > > > > > > > > >> >> > > > > > > > > to
> >> > >>> run a
> >> > >>> > > > task
> >> > >>> > > > > on
> >> > >>> > > > > > > > > behalf
> >> > >>> > > > > > > > > >> of a
> >> > >>> > > > > > > > > >> >> > > > client.
> >> > >>> > > > > > > > > >> >> > > > > > We
> >> > >>> > > > > > > > > >> >> > > > > > > > will
> >> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how
> >> > >>> > > > > > > > > >> >> > > > > > > > > those
> >> > >>> > services
> >> > >>> > > > use
> >> > >>> > > > > > > Kafka
> >> > >>> > > > > > > > > >> clients
> >> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead
> >> > >>> > > > > > > > > >> >> > > > > > > > > of a
> >> > >>> > shared
> >> > >>> > > > > client
> >> > >>> > > > > > > per
> >> > >>> > > > > > > > > >> worker,
> >> > >>> > > > > > > > > >> >> > we
> >> > >>> > > > > > > > > >> >> > > > will
> >> > >>> > > > > > > > > >> >> > > > > > > need
> >> > >>> > > > > > > > > >> >> > > > > > > > a
> >> > >>> > > > > > > > > >> >> > > > > > > > > client per user task since
> the
> >> > >>> > > > authentication
> >> > >>> > > > > > > > happens
> >> > >>> > > > > > > > > >> at the
> >> > >>> > > > > > > > > >> >> > > > > > connection
> >> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect,
> the
> >> > >>> renewer
> >> > >>> > > will
> >> > >>> > > > be
> >> > >>> > > > > > the
> >> > >>> > > > > > > > > >> workers.
> >> > >>> > > > > > > > > >> >> > So,
> >> > >>> > > > > > > > > >> >> > > we
> >> > >>> > > > > > > > > >> >> > > > > > > > probably
> >> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
> >> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
> >> > For
> >> > >>> > > Kafka
> >> > >>> > > > > rest
> >> > >>> > > > > > > > > proxy,
> >> > >>> > > > > > > > > >> the
> >> > >>> > > > > > > > > >> >> > > > renewer
> >> > >>> > > > > > > > > >> >> > > > > > can
> >> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator
> >> > >>> > > > > > > > > >> >> > > > > > > > > of
> >> > the
> >> > >>> > > token.
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on
> who
> >> > can
> >> > >>> > > request
> >> > >>> > > > > > > > delegation
> >> > >>> > > > > > > > > >> tokens?
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people
> to
> >> > send
> >> > >>> > > > > delegation
> >> > >>> > > > > > > > tokens
> >> > >>> > > > > > > > > >> in an
> >> > >>> > > > > > > > > >> >> > > > > encrypted
> >> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for
> >> > expiring
> >> > >>> > > > tokens,
> >> > >>> > > > > > > every
> >> > >>> > > > > > > > > >> broker?
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating
> tokens,
> >> > would
> >> > >>> it
> >> > >>> > be
> >> > >>> > > > > > better
> >> > >>> > > > > > > to
> >> > >>> > > > > > > > > do
> >> > >>> > > > > > > > > >> it in
> >> > >>> > > > > > > > > >> >> > a
> >> > >>> > > > > > > > > >> >> > > > > > request
> >> > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK
> >> > >>> > > > > > > > > >> >> > > > > > > > > directly?
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of
> client
> >> > >>> > > > > > > > > >> >> > > > > > > > > in
> >> > >>> the
> >> > >>> > > wiki
> >> > >>> > > > > > > > sometimes
> >> > >>> > > > > > > > > >> refers
> >> > >>> > > > > > > > > >> >> > to
> >> > >>> > > > > > > > > >> >> > > > the
> >> > >>> > > > > > > > > >> >> > > > > > end
> >> > >>> > > > > > > > > >> >> > > > > > > > > client and some other times
> >> > refers
> >> > >>> to
> >> > >>> > the
> >> > >>> > > > > > client
> >> > >>> > > > > > > > > using
> >> > >>> > > > > > > > > >> the
> >> > >>> > > > > > > > > >> >> > > > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful
> to
> >> > >>> > distinguish
> >> > >>> > > > > > between
> >> > >>> > > > > > > > the
> >> > >>> > > > > > > > > >> two.
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
> >> > sentence
> >> > >>> > > "Broker
> >> > >>> > > > > > also
> >> > >>> > > > > > > > > >> stores the
> >> > >>> > > > > > > > > >> >> > > > > > > > DelegationToken
> >> > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
> >> > zookeeper."
> >> > >>> a
> >> > >>> > bit
> >> > >>> > > > > > more? I
> >> > >>> > > > > > > > > >> thought the
> >> > >>> > > > > > > > > >> >> > > > > > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > Thanks,
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > Jun
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22
> >> > >>> > > > > > > > > >> >> > > > > > > > > AM,
> >> > Jun
> >> > >>> > Rao
> >> > >>> > > <
> >> > >>> > > > > > > > > >> jun@confluent.io>
> >> > >>> > > > > > > > > >> >> > > > wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP
> meeting
> >> > >>> invite.
> >> > >>> > We
> >> > >>> > > > can
> >> > >>> > > > > > > > discuss
> >> > >>> > > > > > > > > >> this in
> >> > >>> > > > > > > > > >> >> > > the
> >> > >>> > > > > > > > > >> >> > > > > > > meeting
> >> > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> >> > >>> > > > > > > > > >> >> > > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > > Jun
> >> > >>> > > > > > > > > >> >> > > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at
> 8:47
> >> > AM,
> >> > >>> > > Harsha <
> >> > >>> > > > > > > > > >> kafka@harsha.io>
> >> > >>> > > > > > > > > >> >> > > > wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> KIP
> >> > >>> meeting
> >> > >>> > > > > around
> >> > >>> > > > > > > > this.
> >> > >>> > > > > > > > > >> The KIP
> >> > >>> > > > > > > > > >> >> > is
> >> > >>> > > > > > > > > >> >> > > > up
> >> > >>> > > > > > > > > >> >> > > > > > for
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and
> if
> >> > there
> >> > >>> are
> >> > >>> > > any
> >> > >>> > > > > > > > questions
> >> > >>> > > > > > > > > >> lets
> >> > >>> > > > > > > > > >> >> > > > quickly
> >> > >>> > > > > > > > > >> >> > > > > > hash
> >> > >>> > > > > > > > > >> >> > > > > > > > out
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> details.
> >> > >>> > > > > > > > > >> >> > > > > > > > > >>
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
> >> > >>> > > > > > > > > >> >> > > > > > > > > >>
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> 08:40
> >> > >>> AM,
> >> > >>> > > parth
> >> > >>> > > > > > > > > brahmbhatt
> >> > >>> > > > > > > > > >> wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > echo
> >> > >>> > system
> >> > >>> > > > uses
> >> > >>> > > > > > so
> >> > >>> > > > > > > no
> >> > >>> > > > > > > > > >> good
> >> > >>> > > > > > > > > >> >> > reason
> >> > >>> > > > > > > > > >> >> > > > > > really.
> >> > >>> > > > > > > > > >> >> > > > > > > > We
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > could
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever
> is
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > the
> >> > >>> > newest
> >> > >>> > > > > > > > recommended
> >> > >>> > > > > > > > > >> standard
> >> > >>> > > > > > > > > >> >> > > is.
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > 3:33
> >> > >>> AM,
> >> > >>> > > > Ismael
> >> > >>> > > > > > > Juma <
> >> > >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> >> > >>> > > > > > > > > >> >> > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > only
> >> > >>> > started
> >> > >>> > > > > > > reviewing
> >> > >>> > > > > > > > > >> this and
> >> > >>> > > > > > > > > >> >> > > may
> >> > >>> > > > > > > > > >> >> > > > > have
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> additional
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
> >> > >>> immediate
> >> > >>> > > > > question
> >> > >>> > > > > > > that
> >> > >>> > > > > > > > > >> came to
> >> > >>> > > > > > > > > >> >> > > mind
> >> > >>> > > > > > > > > >> >> > > > is
> >> > >>> > > > > > > > > >> >> > > > > > our
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> choice of
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > though
> >> > it's
> >> > >>> > > marked
> >> > >>> > > > > as
> >> > >>> > > > > > > > > >> OBSOLETE in
> >> > >>> > > > > > > > > >> >> > the
> >> > >>> > > > > > > > > >> >> > > > IANA
> >> > >>> > > > > > > > > >> >> > > > > > > > > Registry
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> of
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and
> the
> >> > >>> original
> >> > >>> > > RFC
> >> > >>> > > > > > > (2831)
> >> > >>> > > > > > > > > has
> >> > >>> > > > > > > > > >> been
> >> > >>> > > > > > > > > >> >> > > moved
> >> > >>> > > > > > > > > >> >> > > > > to
> >> > >>> > > > > > > > > >> >> > > > > > > > > Historic
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > https://tools.ietf.org/html/
> >> > >>> > > rfc6331
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > >
> >> > >>> > > > > > > > > >> >> > >
> >> > >>> > > > > > > > > >>
> >> > >>> > > > > > >
> >> > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
> >> > >>> mechanisms.xhtml
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning
> >> > behind
> >> > >>> > that
> >> > >>> > > > > > choice?
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016
> at
> >> > 11:29
> >> > >>> > PM,
> >> > >>> > > > Gwen
> >> > >>> > > > > > > > > Shapira <
> >> > >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> >> > >>> > > > > > > > > >> >> > > > > > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments
> inline
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > :)
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > emphasize
> >> > >>> that
> >> > >>> > > even
> >> > >>> > > > > > though
> >> > >>> > > > > > > > > >> delegation
> >> > >>> > > > > > > > > >> >> > > > tokens
> >> > >>> > > > > > > > > >> >> > > > > > > are a
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I
> feel
> >> > very
> >> > >>> > > strongly
> >> > >>> > > > > > about
> >> > >>> > > > > > > > not
> >> > >>> > > > > > > > > >> adding
> >> > >>> > > > > > > > > >> >> > > > > > dependency
> >> > >>> > > > > > > > > >> >> > > > > > > > on
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
> >> > >>> delegation
> >> > >>> > > > > tokens
> >> > >>> > > > > > > for
> >> > >>> > > > > > > > > >> Kafka. The
> >> > >>> > > > > > > > > >> >> > > KIP
> >> > >>> > > > > > > > > >> >> > > > > > > doesn't
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> imply
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency,
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but
> >> > if
> >> > >>> you
> >> > >>> > > can
> >> > >>> > > > > > > > clarify...
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
> >> > dependency.*
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this
> to
> >> > the
> >> > >>> KIP
> >> > >>> > so
> >> > >>> > > > no
> >> > >>> > > > > > one
> >> > >>> > > > > > > > will
> >> > >>> > > > > > > > > >> read
> >> > >>> > > > > > > > > >> >> > the
> >> > >>> > > > > > > > > >> >> > > > KIP
> >> > >>> > > > > > > > > >> >> > > > > > and
> >> > >>> > > > > > > > > >> >> > > > > > > > > panic
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > the
> >> > next
> >> > >>> > > > > release...
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
> >> > delegation
> >> > >>> > token
> >> > >>> > > at
> >> > >>> > > > > any
> >> > >>> > > > > > > > time
> >> > >>> > > > > > > > > >> after
> >> > >>> > > > > > > > > >> >> > > > > > > > authenticating?
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> only
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately
> after?
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you
> are
> >> > >>> > > > authenticated
> >> > >>> > > > > > you
> >> > >>> > > > > > > > can
> >> > >>> > > > > > > > > >> get
> >> > >>> > > > > > > > > >> >> > > > delegation
> >> > >>> > > > > > > > > >> >> > > > > > > > tokens.
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> We
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > need
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a
> client
> >> > >>> > > > authenticated
> >> > >>> > > > > > > using
> >> > >>> > > > > > > > > >> delegation
> >> > >>> > > > > > > > > >> >> > > > > token,
> >> > >>> > > > > > > > > >> >> > > > > > > can
> >> > >>> > > > > > > > > >> >> > > > > > > > > also
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token
> >> > again or
> >> > >>> > not.
> >> > >>> > > > > Also
> >> > >>> > > > > > > > there
> >> > >>> > > > > > > > > >> is the
> >> > >>> > > > > > > > > >> >> > > > > question
> >> > >>> > > > > > > > > >> >> > > > > > of
> >> > >>> > > > > > > > > >> >> > > > > > > > do
> >> > >>> > > > > > > > > >> >> > > > > > > > > we
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
> >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
> >> > >>> delegation
> >> > >>> > > > token
> >> > >>> > > > > > or
> >> > >>> > > > > > > we
> >> > >>> > > > > > > > > >> want
> >> > >>> > > > > > > > > >> >> > > specific
> >> > >>> > > > > > > > > >>
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
--
Regards,
Ashish
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Gwen Shapira <gw...@confluent.io>.
Is it updated? are all concerns addressed? do you want to start a vote?
Sorry for being pushy, I do appreciate that we are all volunteers and
finding time is difficult. This feature is important for anything that
integrates with Kafka (stream processors, Flume, NiFi, etc) and I
don't want to see this getting stuck because we lack coordination
within the community.
On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani <ka...@harsha.io> wrote:
> The only pending update for the KIP is to write up the protocol changes like
> we've it KIP-4. I'll update the wiki.
>
>
> On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com> wrote:
>>
>> I think we decided to not support secret rotation, I guess this can be
>> stated clearly on the KIP. Also, more details on how clients will perform
>> token distribution and how CLI will look like will be helpful.
>>
>> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>> > Hi Guys,
>> >
>> > This discussion was dead for a while. Are there still contentious
>> > points? If not, why are there no votes?
>> >
>> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
>> > > Ashish,
>> > >
>> > > Yes, I will send out a KIP invite for next week to discuss KIP-48 and
>> > other
>> > > remaining KIPs.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com>
>> > wrote:
>> > >
>> > >> Thanks Harsha!
>> > >>
>> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
>> > >> actually make a call on when we should have next KIP call. As there
>> > >> are
>> > a
>> > >> few outstanding KIPs that could not be discussed this week, can we
>> > >> have
>> > a
>> > >> KIP hangout call next week?
>> > >>
>> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani
>> > >> <ka...@harsha.io>
>> > >> wrote:
>> > >>
>> > >>> Ashish,
>> > >>> Yes we are working on it. Lets discuss in the next KIP
>> > >>> meeting.
>> > >>> I'll join.
>> > >>> -Harsha
>> > >>>
>> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
>> > >>> wrote:
>> > >>>
>> > >>> > Hello Harsha,
>> > >>> >
>> > >>> > Are you still working on this? Wondering if we can discuss this in
>> > next
>> > >>> KIP
>> > >>> > meeting, if you can join.
>> > >>> >
>> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
>> > kafka@harsha.io>
>> > >>> > wrote:
>> > >>> >
>> > >>> > > Hi Grant,
>> > >>> > > We are working on it. Will add the details to KIP
>> > >>> > > about
>> > the
>> > >>> > > request protocol.
>> > >>> > >
>> > >>> > > Thanks,
>> > >>> > > Harsha
>> > >>> > >
>> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke
>> > >>> > > <gh...@cloudera.com>
>> > >>> wrote:
>> > >>> > >
>> > >>> > > > Hi Parth,
>> > >>> > > >
>> > >>> > > > Are you still working on this? If you need any help please
>> > >>> > > > don't
>> > >>> > hesitate
>> > >>> > > > to ask.
>> > >>> > > >
>> > >>> > > > Thanks,
>> > >>> > > > Grant
>> > >>> > > >
>> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io>
>> > wrote:
>> > >>> > > >
>> > >>> > > > > Parth,
>> > >>> > > > >
>> > >>> > > > > Thanks for the reply.
>> > >>> > > > >
>> > >>> > > > > It makes sense to only allow the renewal by users that
>> > >>> authenticated
>> > >>> > > > using
>> > >>> > > > > *non* delegation token mechanism. Then, should we make the
>> > >>> renewal a
>> > >>> > > > list?
>> > >>> > > > > For example, in the case of rest proxy, it will be useful
>> > >>> > > > > for
>> > >>> every
>> > >>> > > > > instance of rest proxy to be able to renew the tokens.
>> > >>> > > > >
>> > >>> > > > > It would be clearer if we can document the request protocol
>> > like
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > >>> > >
>> > >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
>> > >>> > > Commandlineandcentralizedadministrativeoperations-
>> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
>> > >>> > > > > .
>> > >>> > > > >
>> > >>> > > > > It would also be useful to document the client APIs.
>> > >>> > > > >
>> > >>> > > > > Thanks,
>> > >>> > > > >
>> > >>> > > > > Jun
>> > >>> > > > >
>> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
>> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
>> > >>> > > > >
>> > >>> > > > > > Hi,
>> > >>> > > > > >
>> > >>> > > > > > I am suggesting that we will only allow the renewal by
>> > >>> > > > > > users
>> > >>> that
>> > >>> > > > > > authenticated using *non* delegation token mechanism. For
>> > >>> example,
>> > >>> > If
>> > >>> > > > > user
>> > >>> > > > > > Alice authenticated using kerberos and requested
>> > >>> > > > > > delegation
>> > >>> tokens,
>> > >>> > > > only
>> > >>> > > > > > user Alice authenticated via non delegation token
>> > >>> > > > > > mechanism
>> > can
>> > >>> > > renew.
>> > >>> > > > > > Clients that have access to delegation tokens can not
>> > >>> > > > > > issue
>> > >>> > renewal
>> > >>> > > > > > request for renewing their own token and this is primarily
>> > >>> > important
>> > >>> > > to
>> > >>> > > > > > reduce the time window for which a compromised token will
>> > >>> > > > > > be
>> > >>> valid.
>> > >>> > > > > >
>> > >>> > > > > > To clarify, Yes any authenticated user can request
>> > >>> > > > > > delegation
>> > >>> > tokens
>> > >>> > > > but
>> > >>> > > > > > even here I would recommend to avoid creating a chain
>> > >>> > > > > > where a
>> > >>> > client
>> > >>> > > > > > authenticated via delegation token request for more
>> > delegation
>> > >>> > > tokens.
>> > >>> > > > > > Basically anyone can request delegation token, as long as
>> > they
>> > >>> > > > > authenticate
>> > >>> > > > > > via a non delegation token mechanism.
>> > >>> > > > > >
>> > >>> > > > > > Aren't classes listed here
>> > >>> > > > > > <
>> > >>> > > > > >
>> > >>> > > > >
>> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
>> > >>> upportforKaf
>> > >>> > > ka-PublicInterfaces
>> > >>> > > > > > >
>> > >>> > > > > > sufficient?
>> > >>> > > > > >
>> > >>> > > > > > Thanks
>> > >>> > > > > > Parth
>> > >>> > > > > >
>> > >>> > > > > >
>> > >>> > > > > >
>> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao
>> > >>> > > > > > <ju...@confluent.io>
>> > >>> wrote:
>> > >>> > > > > >
>> > >>> > > > > > > Parth,
>> > >>> > > > > > >
>> > >>> > > > > > > Thanks for the reply. A couple of comments inline below.
>> > >>> > > > > > >
>> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
>> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> > >>> > > > > > >
>> > >>> > > > > > > > 1. Who / how are tokens renewed? By original requester
>> > >>> only? or
>> > >>> > > > using
>> > >>> > > > > > > > Kerberos
>> > >>> > > > > > > > auth only?
>> > >>> > > > > > > > My recommendation is to do this only using Kerberos
>> > >>> > > > > > > > auth
>> > and
>> > >>> > only
>> > >>> > > > > threw
>> > >>> > > > > > > the
>> > >>> > > > > > > > renewer specified during the acquisition request.
>> > >>> > > > > > > >
>> > >>> > > > > > > >
>> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying that
>> > >>> > > > > > > any
>> > >>> client
>> > >>> > > > > > > authenticated with the delegation token can renew, i.e.
>> > there
>> > >>> is
>> > >>> > no
>> > >>> > > > > > renewer
>> > >>> > > > > > > needed?
>> > >>> > > > > > >
>> > >>> > > > > > > Also, just to be clear, any authenticated client (either
>> > >>> through
>> > >>> > > SASL
>> > >>> > > > > or
>> > >>> > > > > > > SSL) can request a delegation token for the
>> > >>> > > > > > > authenticated
>> > >>> user,
>> > >>> > > > right?
>> > >>> > > > > > >
>> > >>> > > > > > >
>> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
>> > >>> > > > > > > > My recommendation is still to store in ZK or not store
>> > them
>> > >>> at
>> > >>> > > all.
>> > >>> > > > > The
>> > >>> > > > > > > > whole controller based distribution is too much
>> > >>> > > > > > > > overhead
>> > >>> with
>> > >>> > not
>> > >>> > > > > much
>> > >>> > > > > > to
>> > >>> > > > > > > > achieve.
>> > >>> > > > > > > >
>> > >>> > > > > > > > 3. How are tokens invalidated / expired?
>> > >>> > > > > > > > Either by expiration time out or through an explicit
>> > >>> request to
>> > >>> > > > > > > invalidate.
>> > >>> > > > > > > >
>> > >>> > > > > > > > 4. Which encryption algorithm is used?
>> > >>> > > > > > > > SCRAM
>> > >>> > > > > > > >
>> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in
>> > >>> > > > > > > > the
>> > KIP
>> > >>> but
>> > >>> > > was
>> > >>> > > > > > > > discussed
>> > >>> > > > > > > > in this thread)?
>> > >>> > > > > > > > There is no imperonation proposal. I tried and
>> > >>> > > > > > > > explained
>> > how
>> > >>> > its
>> > >>> > > a
>> > >>> > > > > > > > different problem and why its not really necessary to
>> > >>> discuss
>> > >>> > > that
>> > >>> > > > as
>> > >>> > > > > > > part
>> > >>> > > > > > > > of this KIP. This KIP will not support any
>> > impersonation,
>> > >>> it
>> > >>> > > will
>> > >>> > > > > just
>> > >>> > > > > > > be
>> > >>> > > > > > > > another way to authenticate.
>> > >>> > > > > > > >
>> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
>> > >>> > > > > > > > We do not need new ACLs.
>> > >>> > > > > > > >
>> > >>> > > > > > > >
>> > >>> > > > > > > Could we document the format of the new request/response
>> > and
>> > >>> > their
>> > >>> > > > > > > associated Resource and Operation for ACL?
>> > >>> > > > > > >
>> > >>> > > > > > >
>> > >>> > > > > > > > 7. How would the delegation token be configured in the
>> > >>> client?
>> > >>> > > > > > > > Should be through config. I wasn't planning on
>> > >>> > > > > > > > supporting
>> > >>> JAAS
>> > >>> > > for
>> > >>> > > > > > > tokens.
>> > >>> > > > > > > > I don't believe hadoop does this either.
>> > >>> > > > > > > >
>> > >>> > > > > > > > Thanks
>> > >>> > > > > > > > Parth
>> > >>> > > > > > > >
>> > >>> > > > > > > >
>> > >>> > > > > > > >
>> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
>> > jun@confluent.io>
>> > >>> > > wrote:
>> > >>> > > > > > > >
>> > >>> > > > > > > > > Harsha,
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > Another question.
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > 9. How would the delegation token be configured in
>> > >>> > > > > > > > > the
>> > >>> > client?
>> > >>> > > > The
>> > >>> > > > > > > > standard
>> > >>> > > > > > > > > way is to do this through JAAS. However, we will
>> > >>> > > > > > > > > need
>> > to
>> > >>> > think
>> > >>> > > > > > through
>> > >>> > > > > > > if
>> > >>> > > > > > > > > this is convenient in a shared environment. For
>> > example,
>> > >>> > when a
>> > >>> > > > new
>> > >>> > > > > > > task
>> > >>> > > > > > > > is
>> > >>> > > > > > > > > added to a Storm worker node, do we need to
>> > >>> > > > > > > > > dynamically
>> > >>> add a
>> > >>> > > new
>> > >>> > > > > > > section
>> > >>> > > > > > > > > in the JAAS file? It may be more convenient if we
>> > >>> > > > > > > > > can
>> > >>> pass in
>> > >>> > > the
>> > >>> > > > > > token
>> > >>> > > > > > > > > through the config directly w/o going through JAAS.
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > Are you or Parth still actively working on this KIP?
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > Thanks,
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > Jun
>> > >>> > > > > > > > >
>> > >>> > > > > > > > >
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
>> > >>> jun@confluent.io>
>> > >>> > > > wrote:
>> > >>> > > > > > > > >
>> > >>> > > > > > > > > > Just to add on that list.
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > > 2. It would be good to document the format of the
>> > data
>> > >>> > stored
>> > >>> > > > in
>> > >>> > > > > > ZK.
>> > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the
>> > tokens
>> > >>> > > should
>> > >>> > > > > be
>> > >>> > > > > > > > > > propagated through ZK like config/acl/quota, or
>> > through
>> > >>> the
>> > >>> > > > > > > controller.
>> > >>> > > > > > > > > > Currently, the controller is only designed for
>> > >>> propagating
>> > >>> > > > topic
>> > >>> > > > > > > > > metadata,
>> > >>> > > > > > > > > > but not other data.
>> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead
>> > >>> > > > > > > > > > of
>> > >>> > > DIGEST-MD5
>> > >>> > > > > > since
>> > >>> > > > > > > > it's
>> > >>> > > > > > > > > > deprecated?
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > > Thanks,
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > > Jun
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
>> > >>> > > > > gwen@confluent.io>
>> > >>> > > > > > > > > wrote:
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > >> From what I can see, remaining questions are:
>> > >>> > > > > > > > > >>
>> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
>> > requester
>> > >>> > only?
>> > >>> > > > or
>> > >>> > > > > > > using
>> > >>> > > > > > > > > >> Kerberos auth only?
>> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
>> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
>> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
>> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't
>> > >>> > > > > > > > > >> in
>> > the
>> > >>> > KIP
>> > >>> > > > but
>> > >>> > > > > > was
>> > >>> > > > > > > > > >> discussed in this thread)?
>> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
>> > >>> > > > > > > > > >>
>> > >>> > > > > > > > > >> Gwen
>> > >>> > > > > > > > > >>
>> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
>> > >>> kafka@harsha.io>
>> > >>> > > > wrote:
>> > >>> > > > > > > > > >> > Jun & Ismael,
>> > >>> > > > > > > > > >> > Unfortunately I
>> > >>> > > > > > > > > >> > couldn't
>> > >>> attend
>> > >>> > > the
>> > >>> > > > > KIP
>> > >>> > > > > > > > > meeting
>> > >>> > > > > > > > > >> > when delegation tokens
>> > >>> > discussed.
>> > >>> > > > > > > > Appreciate
>> > >>> > > > > > > > > if
>> > >>> > > > > > > > > >> > you can update the
>> > thread if
>> > >>> > you
>> > >>> > > > have
>> > >>> > > > > > any
>> > >>> > > > > > > > > >> > further questions.
>> > >>> > > > > > > > > >> > Thanks,
>> > >>> > > > > > > > > >> > Harsha
>> > >>> > > > > > > > > >> >
>> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei
>> > wrote:
>> > >>> > > > > > > > > >> >> It seems that the links to images in the KIP
>> > >>> > > > > > > > > >> >> are
>> > >>> > broken.
>> > >>> > > > > > > > > >> >>
>> > >>> > > > > > > > > >> >> Liquan
>> > >>> > > > > > > > > >> >>
>> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
>> > brahmbhatt <
>> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
>> > >>> > > > > > > > > >> >>
>> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
>> > >>> > > > > > > > > >> >> > In the current proposal we only allow a user
>> > >>> > > > > > > > > >> >> > to
>> > >>> get
>> > >>> > > > > > delegation
>> > >>> > > > > > > > > token
>> > >>> > > > > > > > > >> for
>> > >>> > > > > > > > > >> >> > the identity that it authenticated as using
>> > >>> another
>> > >>> > > > > > mechanism,
>> > >>> > > > > > > > i.e.
>> > >>> > > > > > > > > >> A user
>> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
>> > >>> > > > > > > > > >> >> > principal
>> > >>> > > > > > > user1@EXAMPLE.COM
>> > >>> > > > > > > > > >> will get
>> > >>> > > > > > > > > >> >> > delegation tokens for that user only. In
>> > future I
>> > >>> > think
>> > >>> > > > we
>> > >>> > > > > > will
>> > >>> > > > > > > > > have
>> > >>> > > > > > > > > >> to
>> > >>> > > > > > > > > >> >> > extend support such that we allow some set
>> > >>> > > > > > > > > >> >> > of
>> > >>> users (
>> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
>> > >>> > storm-nimbus@EXAMPLE.COM)
>> > >>> > > > to
>> > >>> > > > > > > > acquire
>> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other users
>> > whose
>> > >>> > > identity
>> > >>> > > > > > they
>> > >>> > > > > > > > have
>> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers will
>> > have
>> > >>> ACLs
>> > >>> > > to
>> > >>> > > > > > > control
>> > >>> > > > > > > > > >> which
>> > >>> > > > > > > > > >> >> > users are allowed to impersonate other users
>> > and
>> > >>> get
>> > >>> > > > tokens
>> > >>> > > > > > on
>> > >>> > > > > > > > > >> behalf of
>> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
>> > different
>> > >>> > > problem
>> > >>> > > > in
>> > >>> > > > > > my
>> > >>> > > > > > > > > >> opinion and
>> > >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
>> > >>> renewing
>> > >>> > > > > > delegation
>> > >>> > > > > > > > > >> tokens?
>> > >>> > > > > > > > > >> >> > Typically this should be very very low, 1
>> > request
>> > >>> per
>> > >>> > > > > minute
>> > >>> > > > > > > is a
>> > >>> > > > > > > > > >> >> > relatively high estimate. However it depends
>> > >>> > > > > > > > > >> >> > on
>> > >>> the
>> > >>> > > token
>> > >>> > > > > > > > > >> expiration. I am
>> > >>> > > > > > > > > >> >> > less worried about the extra load it puts on
>> > >>> > controller
>> > >>> > > > vs
>> > >>> > > > > > the
>> > >>> > > > > > > > > added
>> > >>> > > > > > > > > >> >> > complexity and the value it offers.
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> > Thanks
>> > >>> > > > > > > > > >> >> > Parth
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma
>> > >>> > > > > > > > > >> >> > <
>> > >>> > > > > > > ismael@juma.me.uk>
>> > >>> > > > > > > > > >> wrote:
>> > >>> > > > > > > > > >> >> >
>> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
>> > >>> separate
>> > >>> > > KIP
>> > >>> > > > > as
>> > >>> > > > > > it
>> > >>> > > > > > > > > will
>> > >>> > > > > > > > > >> >> > > introduce user visible changes. We could
>> > >>> > > > > > > > > >> >> > > also
>> > >>> > update
>> > >>> > > > > KIP-48
>> > >>> > > > > > > to
>> > >>> > > > > > > > > >> have this
>> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
>> > >>> > > separately.
>> > >>> > > > We
>> > >>> > > > > > can
>> > >>> > > > > > > > > >> discuss
>> > >>> > > > > > > > > >> >> > that
>> > >>> > > > > > > > > >> >> > > in the KIP call today.
>> > >>> > > > > > > > > >> >> > >
>> > >>> > > > > > > > > >> >> > > Ismael
>> > >>> > > > > > > > > >> >> > >
>> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini
>> > Sivaram
>> > >>> <
>> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
>> > >>> > > > > > > > > >> >> > >
>> > >>> > > > > > > > > >> >> > > > Ismael,
>> > >>> > > > > > > > > >> >> > > >
>> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
>> > >>> > > > > > > > > >> >> > https://issues.apache.org/
>> > jira/browse/KAFKA-3751)
>> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism.
>> > >>> > > > > > > > > >> >> > > > Would
>> > >>> that
>> > >>> > > need
>> > >>> > > > > > > another
>> > >>> > > > > > > > > >> KIP? If
>> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this
>> > just
>> > >>> be
>> > >>> > a
>> > >>> > > > JIRA
>> > >>> > > > > > > that
>> > >>> > > > > > > > > gets
>> > >>> > > > > > > > > >> >> > > reviewed
>> > >>> > > > > > > > > >> >> > > > when the PR is ready?
>> > >>> > > > > > > > > >> >> > > >
>> > >>> > > > > > > > > >> >> > > > Thank you,
>> > >>> > > > > > > > > >> >> > > >
>> > >>> > > > > > > > > >> >> > > > Rajini
>> > >>> > > > > > > > > >> >> > > >
>> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael
>> > Juma <
>> > >>> > > > > > > > > ismael@juma.me.uk>
>> > >>> > > > > > > > > >> >> > wrote:
>> > >>> > > > > > > > > >> >> > > >
>> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
>> > >>> > candidate.
>> > >>> > > > > > > > > >> >> > > > >
>> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this
>> > >>> > > > > > > > > >> >> > > > > as
>> > a
>> > >>> SASL
>> > >>> > > > > > mechanism
>> > >>> > > > > > > > > that
>> > >>> > > > > > > > > >> might
>> > >>> > > > > > > > > >> >> > be
>> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been
>> > >>> > > > > > > > > >> >> > > > > meaning
>> > to
>> > >>> > check
>> > >>> > > > it
>> > >>> > > > > in
>> > >>> > > > > > > > more
>> > >>> > > > > > > > > >> detail.
>> > >>> > > > > > > > > >> >> > > Good
>> > >>> > > > > > > > > >> >> > > > > to know that you are willing to
>> > contribute
>> > >>> an
>> > >>> > > > > > > > implementation.
>> > >>> > > > > > > > > >> Maybe
>> > >>> > > > > > > > > >> >> > we
>> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
>> > >>> > > > > > > > > >> >> > > > >
>> > >>> > > > > > > > > >> >> > > > > Ismael
>> > >>> > > > > > > > > >> >> > > > >
>> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM,
>> > >>> > > > > > > > > >> >> > > > > Rajini
>> > >>> > Sivaram <
>> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
>> > >>> > > > > > > > > >> >> > > > >
>> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
>> > >>> > Authentication
>> > >>> > > > > > > > Mechanism)
>> > >>> > > > > > > > > >> is a
>> > >>> > > > > > > > > >> >> > > better
>> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java
>> > >>> > > > > > > > > >> >> > > > > > doesn't
>> > >>> come
>> > >>> > > > with a
>> > >>> > > > > > > > > built-in
>> > >>> > > > > > > > > >> SCRAM
>> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will
>> > >>> > > > > > > > > >> >> > > > > > be
>> > >>> happy
>> > >>> > > to
>> > >>> > > > > add
>> > >>> > > > > > > > > support
>> > >>> > > > > > > > > >> in
>> > >>> > > > > > > > > >> >> > Kafka
>> > >>> > > > > > > > > >> >> > > > > since
>> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to
>> > support
>> > >>> > > anyway.
>> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
>> > >>> > describes
>> > >>> > > > the
>> > >>> > > > > > > > protocol
>> > >>> > > > > > > > > >> for
>> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
>> > >>> > > > > > > > > >> >> > > > > >
>> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun
>> > Rao <
>> > >>> > > > > > > > jun@confluent.io
>> > >>> > > > > > > > > >
>> > >>> > > > > > > > > >> wrote:
>> > >>> > > > > > > > > >> >> > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > Parth,
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A
>> > >>> > > > > > > > > >> >> > > > > > > couple
>> > of
>> > >>> > more
>> > >>> > > > > > > questions.
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > 110. What does
>> > >>> > > > > > > > > >> >> > > > > > > getDelegationTokenAs
>> > >>> mean?
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of
>> > getting
>> > >>> and
>> > >>> > > > > > renewing
>> > >>> > > > > > > > > >> delegation
>> > >>> > > > > > > > > >> >> > > > tokens?
>> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether
>> > they
>> > >>> > > should
>> > >>> > > > be
>> > >>> > > > > > > > > directed
>> > >>> > > > > > > > > >> to the
>> > >>> > > > > > > > > >> >> > > > > > > controller.
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > Jun
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM,
>> > parth
>> > >>> > > > > brahmbhatt <
>> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action
>> > >>> > > > > > > > > >> >> > > > > > > > to
>> > add
>> > >>> > acls
>> > >>> > > > on
>> > >>> > > > > > who
>> > >>> > > > > > > > can
>> > >>> > > > > > > > > >> request
>> > >>> > > > > > > > > >> >> > > > > > delegation
>> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case
>> > for
>> > >>> that
>> > >>> > > yet
>> > >>> > > > > but
>> > >>> > > > > > > > down
>> > >>> > > > > > > > > >> the line
>> > >>> > > > > > > > > >> >> > > > when
>> > >>> > > > > > > > > >> >> > > > > we
>> > >>> > > > > > > > > >> >> > > > > > > > start supporting
>> > getDelegationTokenAs
>> > >>> it
>> > >>> > > will
>> > >>> > > > > be
>> > >>> > > > > > > > > >> necessary.
>> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be
>> > only
>> > >>> > > > > > > used/distributed
>> > >>> > > > > > > > > >> over
>> > >>> > > > > > > > > >> >> > secure
>> > >>> > > > > > > > > >> >> > > > > > > channels.
>> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we
>> > >>> > > > > > > > > >> >> > > > > > > > end
>> > up
>> > >>> > > choosing
>> > >>> > > > > > > > > >> Invalidation will
>> > >>> > > > > > > > > >> >> > > be
>> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker
>> > >>> > > > > > > > > >> >> > > > > > > > or
>> > >>> > > controller.
>> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
>> > >>> somewhere
>> > >>> > > > that
>> > >>> > > > > > > > > >> invalidation
>> > >>> > > > > > > > > >> >> > will
>> > >>> > > > > > > > > >> >> > > > > > directly
>> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is
>> > not
>> > >>> the
>> > >>> > > > > intent.
>> > >>> > > > > > > > > >> Invalidation
>> > >>> > > > > > > > > >> >> > > will
>> > >>> > > > > > > > > >> >> > > > > > either
>> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to
>> > >>> expiration. No
>> > >>> > > > > direct
>> > >>> > > > > > > > > >> zookeeper
>> > >>> > > > > > > > > >> >> > > > > interaction
>> > >>> > > > > > > > > >> >> > > > > > > from
>> > >>> > > > > > > > > >> >> > > > > > > > any client.
>> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
>> > >>> DelegationToken
>> > >>> > > > > without
>> > >>> > > > > > > the
>> > >>> > > > > > > > > >> hmac in
>> > >>> > > > > > > > > >> >> > the
>> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
>> > >>> confusion.
>> > >>> > > The
>> > >>> > > > > sole
>> > >>> > > > > > > > > >> purpose of
>> > >>> > > > > > > > > >> >> > > > > zookeeper
>> > >>> > > > > > > > > >> >> > > > > > in
>> > >>> > > > > > > > > >> >> > > > > > > > this design is as distribution
>> > channel
>> > >>> > for
>> > >>> > > > > tokens
>> > >>> > > > > > > > > >> between all
>> > >>> > > > > > > > > >> >> > > > brokers
>> > >>> > > > > > > > > >> >> > > > > > > and a
>> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens
>> > >>> > > > > > > > > >> >> > > > > > > > that
>> > >>> were
>> > >>> > > > > > generated
>> > >>> > > > > > > by
>> > >>> > > > > > > > > >> making a
>> > >>> > > > > > > > > >> >> > > > > request
>> > >>> > > > > > > > > >> >> > > > > > > to a
>> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on
>> > this
>> > >>> in
>> > >>> > > > second
>> > >>> > > > > > > > > >> paragraph). The
>> > >>> > > > > > > > > >> >> > > > token
>> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
>> > >>> renewer,
>> > >>> > > > uuid
>> > >>> > > > > ,
>> > >>> > > > > > > > > >> expiration,
>> > >>> > > > > > > > > >> >> > > hmac)
>> > >>> > > > > > > > > >> >> > > > ,
>> > >>> > > > > > > > > >> >> > > > > > one
>> > >>> > > > > > > > > >> >> > > > > > > of
>> > >>> > > > > > > > > >> >> > > > > > > > which is the finally generated
>> > >>> > > > > > > > > >> >> > > > > > > > hmac
>> > >>> but
>> > >>> > > hmac
>> > >>> > > > it
>> > >>> > > > > > > self
>> > >>> > > > > > > > is
>> > >>> > > > > > > > > >> >> > derivable
>> > >>> > > > > > > > > >> >> > > > if
>> > >>> > > > > > > > > >> >> > > > > > you
>> > >>> > > > > > > > > >> >> > > > > > > > have all the other elements of
>> > >>> > > > > > > > > >> >> > > > > > > > the
>> > >>> token
>> > >>> > +
>> > >>> > > > > secret
>> > >>> > > > > > > key
>> > >>> > > > > > > > > to
>> > >>> > > > > > > > > >> >> > generate
>> > >>> > > > > > > > > >> >> > > > > hmac.
>> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide
>> > SSL
>> > >>> > > support
>> > >>> > > > we
>> > >>> > > > > > do
>> > >>> > > > > > > > not
>> > >>> > > > > > > > > >> want the
>> > >>> > > > > > > > > >> >> > > > > entire
>> > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
>> > >>> zookeeper
>> > >>> > > as
>> > >>> > > > > that
>> > >>> > > > > > > > will
>> > >>> > > > > > > > > >> be an
>> > >>> > > > > > > > > >> >> > > > insecure
>> > >>> > > > > > > > > >> >> > > > > > > wire
>> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store
>> > >>> > > > > > > > > >> >> > > > > > > > all
>> > >>> the
>> > >>> > > other
>> > >>> > > > > > > > elements
>> > >>> > > > > > > > > >> of a
>> > >>> > > > > > > > > >> >> > > > > delegation
>> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
>> > >>> elements
>> > >>> > and
>> > >>> > > > > > because
>> > >>> > > > > > > > > they
>> > >>> > > > > > > > > >> also
>> > >>> > > > > > > > > >> >> > > have
>> > >>> > > > > > > > > >> >> > > > > > access
>> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able
>> > >>> > > > > > > > > >> >> > > > > > > > to
>> > >>> > generate
>> > >>> > > > > hmac
>> > >>> > > > > > on
>> > >>> > > > > > > > > >> their end.
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed
>> > >>> > > > > > > > > >> >> > > > > > > > is
>> > to
>> > >>> > avoid
>> > >>> > > > > > > zookeeper
>> > >>> > > > > > > > > >> >> > > altogether. A
>> > >>> > > > > > > > > >> >> > > > > > > Client
>> > >>> > > > > > > > > >> >> > > > > > > > will call broker with required
>> > >>> > information
>> > >>> > > > > > (owner,
>> > >>> > > > > > > > > >> renwer,
>> > >>> > > > > > > > > >> >> > > > > expiration)
>> > >>> > > > > > > > > >> >> > > > > > > and
>> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid).
>> > Broker
>> > >>> > won't
>> > >>> > > > > store
>> > >>> > > > > > > this
>> > >>> > > > > > > > > in
>> > >>> > > > > > > > > >> >> > > zookeeper.
>> > >>> > > > > > > > > >> >> > > > > > From
>> > >>> > > > > > > > > >> >> > > > > > > > this point a client can contact
>> > >>> > > > > > > > > >> >> > > > > > > > any
>> > >>> > broker
>> > >>> > > > with
>> > >>> > > > > > all
>> > >>> > > > > > > > the
>> > >>> > > > > > > > > >> >> > > delegation
>> > >>> > > > > > > > > >> >> > > > > > token
>> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration,
>> > hmac,
>> > >>> > > uuid)
>> > >>> > > > > the
>> > >>> > > > > > > > borker
>> > >>> > > > > > > > > >> will
>> > >>> > > > > > > > > >> >> > > > > regenerate
>> > >>> > > > > > > > > >> >> > > > > > > the
>> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches
>> > >>> > > > > > > > > >> >> > > > > > > > with
>> > >>> hmac
>> > >>> > > > > > presented
>> > >>> > > > > > > by
>> > >>> > > > > > > > > >> client ,
>> > >>> > > > > > > > > >> >> > > > broker
>> > >>> > > > > > > > > >> >> > > > > > > will
>> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
>> > >>> > > > > > > > > >> >> > > > > > > > authenticate.
>> > >>> Only
>> > >>> > > > > problem
>> > >>> > > > > > > with
>> > >>> > > > > > > > > >> this
>> > >>> > > > > > > > > >> >> > > approach
>> > >>> > > > > > > > > >> >> > > > > is
>> > >>> > > > > > > > > >> >> > > > > > if
>> > >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised
>> > >>> > > > > > > > > >> >> > > > > > > > any
>> > >>> client
>> > >>> > > can
>> > >>> > > > > now
>> > >>> > > > > > > > > generate
>> > >>> > > > > > > > > >> >> > random
>> > >>> > > > > > > > > >> >> > > > > tokens
>> > >>> > > > > > > > > >> >> > > > > > > and
>> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to
>> > >>> authenticate
>> > >>> > as
>> > >>> > > > any
>> > >>> > > > > > user
>> > >>> > > > > > > > > they
>> > >>> > > > > > > > > >> like.
>> > >>> > > > > > > > > >> >> > > with
>> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
>> > >>> tokens
>> > >>> > > > > acquired
>> > >>> > > > > > > via
>> > >>> > > > > > > > a
>> > >>> > > > > > > > > >> broker
>> > >>> > > > > > > > > >> >> > > > (using
>> > >>> > > > > > > > > >> >> > > > > > some
>> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than
>> > >>> > > > > > > > > >> >> > > > > > > > delegation
>> > >>> token)
>> > >>> > > will
>> > >>> > > > > be
>> > >>> > > > > > > > > >> accepted. We
>> > >>> > > > > > > > > >> >> > > need
>> > >>> > > > > > > > > >> >> > > > to
>> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes
>> > >>> > > > > > > > > >> >> > > > > > > > more
>> > >>> sense
>> > >>> > and
>> > >>> > > > we
>> > >>> > > > > > can
>> > >>> > > > > > > go
>> > >>> > > > > > > > > >> over it
>> > >>> > > > > > > > > >> >> > in
>> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
>> > >>> > > > > > > > > >> >> > > > > > > > meeting.
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite
>> > >>> > > > > > > > > >> >> > > > > > > > to
>> > >>> me?
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > Thanks
>> > >>> > > > > > > > > >> >> > > > > > > > Parth
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35
>> > >>> > > > > > > > > >> >> > > > > > > > AM,
>> > Jun
>> > >>> > Rao <
>> > >>> > > > > > > > > >> jun@confluent.io>
>> > >>> > > > > > > > > >> >> > > > wrote:
>> > >>> > > > > > > > > >> >> > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
>> > comments.
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be
>> > useful
>> > >>> for
>> > >>> > > > Kafka
>> > >>> > > > > > > > Connect
>> > >>> > > > > > > > > >> and
>> > >>> > > > > > > > > >> >> > Kafka
>> > >>> > > > > > > > > >> >> > > > > rest
>> > >>> > > > > > > > > >> >> > > > > > > > proxy
>> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need
>> > >>> > > > > > > > > >> >> > > > > > > > > to
>> > >>> run a
>> > >>> > > > task
>> > >>> > > > > on
>> > >>> > > > > > > > > behalf
>> > >>> > > > > > > > > >> of a
>> > >>> > > > > > > > > >> >> > > > client.
>> > >>> > > > > > > > > >> >> > > > > > We
>> > >>> > > > > > > > > >> >> > > > > > > > will
>> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how
>> > >>> > > > > > > > > >> >> > > > > > > > > those
>> > >>> > services
>> > >>> > > > use
>> > >>> > > > > > > Kafka
>> > >>> > > > > > > > > >> clients
>> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead
>> > >>> > > > > > > > > >> >> > > > > > > > > of a
>> > >>> > shared
>> > >>> > > > > client
>> > >>> > > > > > > per
>> > >>> > > > > > > > > >> worker,
>> > >>> > > > > > > > > >> >> > we
>> > >>> > > > > > > > > >> >> > > > will
>> > >>> > > > > > > > > >> >> > > > > > > need
>> > >>> > > > > > > > > >> >> > > > > > > > a
>> > >>> > > > > > > > > >> >> > > > > > > > > client per user task since the
>> > >>> > > > authentication
>> > >>> > > > > > > > happens
>> > >>> > > > > > > > > >> at the
>> > >>> > > > > > > > > >> >> > > > > > connection
>> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
>> > >>> renewer
>> > >>> > > will
>> > >>> > > > be
>> > >>> > > > > > the
>> > >>> > > > > > > > > >> workers.
>> > >>> > > > > > > > > >> >> > So,
>> > >>> > > > > > > > > >> >> > > we
>> > >>> > > > > > > > > >> >> > > > > > > > probably
>> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
>> > >>> > > > > > > > > >> >> > > > > > > > > renewers.
>> > For
>> > >>> > > Kafka
>> > >>> > > > > rest
>> > >>> > > > > > > > > proxy,
>> > >>> > > > > > > > > >> the
>> > >>> > > > > > > > > >> >> > > > renewer
>> > >>> > > > > > > > > >> >> > > > > > can
>> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator
>> > >>> > > > > > > > > >> >> > > > > > > > > of
>> > the
>> > >>> > > token.
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who
>> > can
>> > >>> > > request
>> > >>> > > > > > > > delegation
>> > >>> > > > > > > > > >> tokens?
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to
>> > send
>> > >>> > > > > delegation
>> > >>> > > > > > > > tokens
>> > >>> > > > > > > > > >> in an
>> > >>> > > > > > > > > >> >> > > > > encrypted
>> > >>> > > > > > > > > >> >> > > > > > > > > channel?
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for
>> > expiring
>> > >>> > > > tokens,
>> > >>> > > > > > > every
>> > >>> > > > > > > > > >> broker?
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens,
>> > would
>> > >>> it
>> > >>> > be
>> > >>> > > > > > better
>> > >>> > > > > > > to
>> > >>> > > > > > > > > do
>> > >>> > > > > > > > > >> it in
>> > >>> > > > > > > > > >> >> > a
>> > >>> > > > > > > > > >> >> > > > > > request
>> > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK
>> > >>> > > > > > > > > >> >> > > > > > > > > directly?
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client
>> > >>> > > > > > > > > >> >> > > > > > > > > in
>> > >>> the
>> > >>> > > wiki
>> > >>> > > > > > > > sometimes
>> > >>> > > > > > > > > >> refers
>> > >>> > > > > > > > > >> >> > to
>> > >>> > > > > > > > > >> >> > > > the
>> > >>> > > > > > > > > >> >> > > > > > end
>> > >>> > > > > > > > > >> >> > > > > > > > > client and some other times
>> > refers
>> > >>> to
>> > >>> > the
>> > >>> > > > > > client
>> > >>> > > > > > > > > using
>> > >>> > > > > > > > > >> the
>> > >>> > > > > > > > > >> >> > > > > delegation
>> > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
>> > >>> > distinguish
>> > >>> > > > > > between
>> > >>> > > > > > > > the
>> > >>> > > > > > > > > >> two.
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
>> > sentence
>> > >>> > > "Broker
>> > >>> > > > > > also
>> > >>> > > > > > > > > >> stores the
>> > >>> > > > > > > > > >> >> > > > > > > > DelegationToken
>> > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
>> > zookeeper."
>> > >>> a
>> > >>> > bit
>> > >>> > > > > > more? I
>> > >>> > > > > > > > > >> thought the
>> > >>> > > > > > > > > >> >> > > > > > > delegation
>> > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > Thanks,
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > Jun
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22
>> > >>> > > > > > > > > >> >> > > > > > > > > AM,
>> > Jun
>> > >>> > Rao
>> > >>> > > <
>> > >>> > > > > > > > > >> jun@confluent.io>
>> > >>> > > > > > > > > >> >> > > > wrote:
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
>> > >>> > > > > > > > > >> >> > > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
>> > >>> invite.
>> > >>> > We
>> > >>> > > > can
>> > >>> > > > > > > > discuss
>> > >>> > > > > > > > > >> this in
>> > >>> > > > > > > > > >> >> > > the
>> > >>> > > > > > > > > >> >> > > > > > > meeting
>> > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
>> > >>> > > > > > > > > >> >> > > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
>> > >>> > > > > > > > > >> >> > > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > > Jun
>> > >>> > > > > > > > > >> >> > > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47
>> > AM,
>> > >>> > > Harsha <
>> > >>> > > > > > > > > >> kafka@harsha.io>
>> > >>> > > > > > > > > >> >> > > > wrote:
>> > >>> > > > > > > > > >> >> > > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a
>> > >>> > > > > > > > > >> >> > > > > > > > > >> KIP
>> > >>> meeting
>> > >>> > > > > around
>> > >>> > > > > > > > this.
>> > >>> > > > > > > > > >> The KIP
>> > >>> > > > > > > > > >> >> > is
>> > >>> > > > > > > > > >> >> > > > up
>> > >>> > > > > > > > > >> >> > > > > > for
>> > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if
>> > there
>> > >>> are
>> > >>> > > any
>> > >>> > > > > > > > questions
>> > >>> > > > > > > > > >> lets
>> > >>> > > > > > > > > >> >> > > > quickly
>> > >>> > > > > > > > > >> >> > > > > > hash
>> > >>> > > > > > > > > >> >> > > > > > > > out
>> > >>> > > > > > > > > >> >> > > > > > > > > >> details.
>> > >>> > > > > > > > > >> >> > > > > > > > > >>
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
>> > >>> > > > > > > > > >> >> > > > > > > > > >>
>> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at
>> > >>> > > > > > > > > >> >> > > > > > > > > >> 08:40
>> > >>> AM,
>> > >>> > > parth
>> > >>> > > > > > > > > brahmbhatt
>> > >>> > > > > > > > > >> wrote:
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > echo
>> > >>> > system
>> > >>> > > > uses
>> > >>> > > > > > so
>> > >>> > > > > > > no
>> > >>> > > > > > > > > >> good
>> > >>> > > > > > > > > >> >> > reason
>> > >>> > > > > > > > > >> >> > > > > > really.
>> > >>> > > > > > > > > >> >> > > > > > > > We
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > could
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > the
>> > >>> > newest
>> > >>> > > > > > > > recommended
>> > >>> > > > > > > > > >> standard
>> > >>> > > > > > > > > >> >> > > is.
>> > >>> > > > > > > > > >> >> > > > > > > > > >> >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
>> > >>> > > > > > > > > >> >> > > > > > > > > >> >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > 3:33
>> > >>> AM,
>> > >>> > > > Ismael
>> > >>> > > > > > > Juma <
>> > >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
>> > >>> > > > > > > > > >> >> > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > wrote:
>> > >>> > > > > > > > > >> >> > > > > > > > > >> >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > only
>> > >>> > started
>> > >>> > > > > > > reviewing
>> > >>> > > > > > > > > >> this and
>> > >>> > > > > > > > > >> >> > > may
>> > >>> > > > > > > > > >> >> > > > > have
>> > >>> > > > > > > > > >> >> > > > > > > > > >> additional
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
>> > >>> immediate
>> > >>> > > > > question
>> > >>> > > > > > > that
>> > >>> > > > > > > > > >> came to
>> > >>> > > > > > > > > >> >> > > mind
>> > >>> > > > > > > > > >> >> > > > is
>> > >>> > > > > > > > > >> >> > > > > > our
>> > >>> > > > > > > > > >> >> > > > > > > > > >> choice of
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > though
>> > it's
>> > >>> > > marked
>> > >>> > > > > as
>> > >>> > > > > > > > > >> OBSOLETE in
>> > >>> > > > > > > > > >> >> > the
>> > >>> > > > > > > > > >> >> > > > IANA
>> > >>> > > > > > > > > >> >> > > > > > > > > Registry
>> > >>> > > > > > > > > >> >> > > > > > > > > >> of
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
>> > >>> original
>> > >>> > > RFC
>> > >>> > > > > > > (2831)
>> > >>> > > > > > > > > has
>> > >>> > > > > > > > > >> been
>> > >>> > > > > > > > > >> >> > > moved
>> > >>> > > > > > > > > >> >> > > > > to
>> > >>> > > > > > > > > >> >> > > > > > > > > Historic
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > https://tools.ietf.org/html/
>> > >>> > > rfc6331
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > >
>> > >>> > > > > > > > > >> >> > >
>> > >>> > > > > > > > > >>
>> > >>> > > > > > >
>> > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
>> > >>> mechanisms.xhtml
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning
>> > behind
>> > >>> > that
>> > >>> > > > > > choice?
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at
>> > 11:29
>> > >>> > PM,
>> > >>> > > > Gwen
>> > >>> > > > > > > > > Shapira <
>> > >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
>> > >>> > > > > > > > > >> >> > > > > > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > :)
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > emphasize
>> > >>> that
>> > >>> > > even
>> > >>> > > > > > though
>> > >>> > > > > > > > > >> delegation
>> > >>> > > > > > > > > >> >> > > > tokens
>> > >>> > > > > > > > > >> >> > > > > > > are a
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel
>> > very
>> > >>> > > strongly
>> > >>> > > > > > about
>> > >>> > > > > > > > not
>> > >>> > > > > > > > > >> adding
>> > >>> > > > > > > > > >> >> > > > > > dependency
>> > >>> > > > > > > > > >> >> > > > > > > > on
>> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
>> > >>> delegation
>> > >>> > > > > tokens
>> > >>> > > > > > > for
>> > >>> > > > > > > > > >> Kafka. The
>> > >>> > > > > > > > > >> >> > > KIP
>> > >>> > > > > > > > > >> >> > > > > > > doesn't
>> > >>> > > > > > > > > >> >> > > > > > > > > >> imply
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency,
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but
>> > if
>> > >>> you
>> > >>> > > can
>> > >>> > > > > > > > clarify...
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
>> > dependency.*
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to
>> > the
>> > >>> KIP
>> > >>> > so
>> > >>> > > > no
>> > >>> > > > > > one
>> > >>> > > > > > > > will
>> > >>> > > > > > > > > >> read
>> > >>> > > > > > > > > >> >> > the
>> > >>> > > > > > > > > >> >> > > > KIP
>> > >>> > > > > > > > > >> >> > > > > > and
>> > >>> > > > > > > > > >> >> > > > > > > > > panic
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > the
>> > next
>> > >>> > > > > release...
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
>> > delegation
>> > >>> > token
>> > >>> > > at
>> > >>> > > > > any
>> > >>> > > > > > > > time
>> > >>> > > > > > > > > >> after
>> > >>> > > > > > > > > >> >> > > > > > > > authenticating?
>> > >>> > > > > > > > > >> >> > > > > > > > > >> only
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
>> > >>> > > > authenticated
>> > >>> > > > > > you
>> > >>> > > > > > > > can
>> > >>> > > > > > > > > >> get
>> > >>> > > > > > > > > >> >> > > > delegation
>> > >>> > > > > > > > > >> >> > > > > > > > tokens.
>> > >>> > > > > > > > > >> >> > > > > > > > > >> We
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > need
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
>> > >>> > > > authenticated
>> > >>> > > > > > > using
>> > >>> > > > > > > > > >> delegation
>> > >>> > > > > > > > > >> >> > > > > token,
>> > >>> > > > > > > > > >> >> > > > > > > can
>> > >>> > > > > > > > > >> >> > > > > > > > > also
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token
>> > again or
>> > >>> > not.
>> > >>> > > > > Also
>> > >>> > > > > > > > there
>> > >>> > > > > > > > > >> is the
>> > >>> > > > > > > > > >> >> > > > > question
>> > >>> > > > > > > > > >> >> > > > > > of
>> > >>> > > > > > > > > >> >> > > > > > > > do
>> > >>> > > > > > > > > >> >> > > > > > > > > we
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
>> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
>> > >>> delegation
>> > >>> > > > token
>> > >>> > > > > > or
>> > >>> > > > > > > we
>> > >>> > > > > > > > > >> want
>> > >>> > > > > > > > > >> >> > > specific
>> > >>> > > > > > > > > >>
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Harsha Chintalapani <ka...@harsha.io>.
The only pending update for the KIP is to write up the protocol changes
like we've it KIP-4. I'll update the wiki.
On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh <as...@cloudera.com> wrote:
> I think we decided to not support secret rotation, I guess this can be
> stated clearly on the KIP. Also, more details on how clients will perform
> token distribution and how CLI will look like will be helpful.
>
> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Hi Guys,
> >
> > This discussion was dead for a while. Are there still contentious
> > points? If not, why are there no votes?
> >
> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> > > Ashish,
> > >
> > > Yes, I will send out a KIP invite for next week to discuss KIP-48 and
> > other
> > > remaining KIPs.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com>
> > wrote:
> > >
> > >> Thanks Harsha!
> > >>
> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
> > >> actually make a call on when we should have next KIP call. As there
> are
> > a
> > >> few outstanding KIPs that could not be discussed this week, can we
> have
> > a
> > >> KIP hangout call next week?
> > >>
> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <kafka@harsha.io
> >
> > >> wrote:
> > >>
> > >>> Ashish,
> > >>> Yes we are working on it. Lets discuss in the next KIP
> meeting.
> > >>> I'll join.
> > >>> -Harsha
> > >>>
> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
> > >>> wrote:
> > >>>
> > >>> > Hello Harsha,
> > >>> >
> > >>> > Are you still working on this? Wondering if we can discuss this in
> > next
> > >>> KIP
> > >>> > meeting, if you can join.
> > >>> >
> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> > kafka@harsha.io>
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Grant,
> > >>> > > We are working on it. Will add the details to KIP about
> > the
> > >>> > > request protocol.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > Harsha
> > >>> > >
> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <ghenke@cloudera.com
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Hi Parth,
> > >>> > > >
> > >>> > > > Are you still working on this? If you need any help please
> don't
> > >>> > hesitate
> > >>> > > > to ask.
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > > Grant
> > >>> > > >
> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > >>> > > >
> > >>> > > > > Parth,
> > >>> > > > >
> > >>> > > > > Thanks for the reply.
> > >>> > > > >
> > >>> > > > > It makes sense to only allow the renewal by users that
> > >>> authenticated
> > >>> > > > using
> > >>> > > > > *non* delegation token mechanism. Then, should we make the
> > >>> renewal a
> > >>> > > > list?
> > >>> > > > > For example, in the case of rest proxy, it will be useful for
> > >>> every
> > >>> > > > > instance of rest proxy to be able to renew the tokens.
> > >>> > > > >
> > >>> > > > > It would be clearer if we can document the request protocol
> > like
> > >>> > > > >
> > >>> > > > >
> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> > >>> > > Commandlineandcentralizedadministrativeoperations-
> > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> > >>> > > > > .
> > >>> > > > >
> > >>> > > > > It would also be useful to document the client APIs.
> > >>> > > > >
> > >>> > > > > Thanks,
> > >>> > > > >
> > >>> > > > > Jun
> > >>> > > > >
> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> > >>> > > > >
> > >>> > > > > > Hi,
> > >>> > > > > >
> > >>> > > > > > I am suggesting that we will only allow the renewal by
> users
> > >>> that
> > >>> > > > > > authenticated using *non* delegation token mechanism. For
> > >>> example,
> > >>> > If
> > >>> > > > > user
> > >>> > > > > > Alice authenticated using kerberos and requested delegation
> > >>> tokens,
> > >>> > > > only
> > >>> > > > > > user Alice authenticated via non delegation token mechanism
> > can
> > >>> > > renew.
> > >>> > > > > > Clients that have access to delegation tokens can not
> issue
> > >>> > renewal
> > >>> > > > > > request for renewing their own token and this is primarily
> > >>> > important
> > >>> > > to
> > >>> > > > > > reduce the time window for which a compromised token will
> be
> > >>> valid.
> > >>> > > > > >
> > >>> > > > > > To clarify, Yes any authenticated user can request
> delegation
> > >>> > tokens
> > >>> > > > but
> > >>> > > > > > even here I would recommend to avoid creating a chain
> where a
> > >>> > client
> > >>> > > > > > authenticated via delegation token request for more
> > delegation
> > >>> > > tokens.
> > >>> > > > > > Basically anyone can request delegation token, as long as
> > they
> > >>> > > > > authenticate
> > >>> > > > > > via a non delegation token mechanism.
> > >>> > > > > >
> > >>> > > > > > Aren't classes listed here
> > >>> > > > > > <
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
> > >>> upportforKaf
> > >>> > > ka-PublicInterfaces
> > >>> > > > > > >
> > >>> > > > > > sufficient?
> > >>> > > > > >
> > >>> > > > > > Thanks
> > >>> > > > > > Parth
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <jun@confluent.io
> >
> > >>> wrote:
> > >>> > > > > >
> > >>> > > > > > > Parth,
> > >>> > > > > > >
> > >>> > > > > > > Thanks for the reply. A couple of comments inline below.
> > >>> > > > > > >
> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > >>> > > > > > >
> > >>> > > > > > > > 1. Who / how are tokens renewed? By original requester
> > >>> only? or
> > >>> > > > using
> > >>> > > > > > > > Kerberos
> > >>> > > > > > > > auth only?
> > >>> > > > > > > > My recommendation is to do this only using Kerberos
> auth
> > and
> > >>> > only
> > >>> > > > > threw
> > >>> > > > > > > the
> > >>> > > > > > > > renewer specified during the acquisition request.
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > Hmm, not sure that I follow this. Are you saying that any
> > >>> client
> > >>> > > > > > > authenticated with the delegation token can renew, i.e.
> > there
> > >>> is
> > >>> > no
> > >>> > > > > > renewer
> > >>> > > > > > > needed?
> > >>> > > > > > >
> > >>> > > > > > > Also, just to be clear, any authenticated client (either
> > >>> through
> > >>> > > SASL
> > >>> > > > > or
> > >>> > > > > > > SSL) can request a delegation token for the authenticated
> > >>> user,
> > >>> > > > right?
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> > >>> > > > > > > > My recommendation is still to store in ZK or not store
> > them
> > >>> at
> > >>> > > all.
> > >>> > > > > The
> > >>> > > > > > > > whole controller based distribution is too much
> overhead
> > >>> with
> > >>> > not
> > >>> > > > > much
> > >>> > > > > > to
> > >>> > > > > > > > achieve.
> > >>> > > > > > > >
> > >>> > > > > > > > 3. How are tokens invalidated / expired?
> > >>> > > > > > > > Either by expiration time out or through an explicit
> > >>> request to
> > >>> > > > > > > invalidate.
> > >>> > > > > > > >
> > >>> > > > > > > > 4. Which encryption algorithm is used?
> > >>> > > > > > > > SCRAM
> > >>> > > > > > > >
> > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the
> > KIP
> > >>> but
> > >>> > > was
> > >>> > > > > > > > discussed
> > >>> > > > > > > > in this thread)?
> > >>> > > > > > > > There is no imperonation proposal. I tried and
> explained
> > how
> > >>> > its
> > >>> > > a
> > >>> > > > > > > > different problem and why its not really necessary to
> > >>> discuss
> > >>> > > that
> > >>> > > > as
> > >>> > > > > > > part
> > >>> > > > > > > > of this KIP. This KIP will not support any
> > impersonation,
> > >>> it
> > >>> > > will
> > >>> > > > > just
> > >>> > > > > > > be
> > >>> > > > > > > > another way to authenticate.
> > >>> > > > > > > >
> > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> > >>> > > > > > > > We do not need new ACLs.
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > Could we document the format of the new request/response
> > and
> > >>> > their
> > >>> > > > > > > associated Resource and Operation for ACL?
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > > > > 7. How would the delegation token be configured in the
> > >>> client?
> > >>> > > > > > > > Should be through config. I wasn't planning on
> supporting
> > >>> JAAS
> > >>> > > for
> > >>> > > > > > > tokens.
> > >>> > > > > > > > I don't believe hadoop does this either.
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks
> > >>> > > > > > > > Parth
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> > jun@confluent.io>
> > >>> > > wrote:
> > >>> > > > > > > >
> > >>> > > > > > > > > Harsha,
> > >>> > > > > > > > >
> > >>> > > > > > > > > Another question.
> > >>> > > > > > > > >
> > >>> > > > > > > > > 9. How would the delegation token be configured in
> the
> > >>> > client?
> > >>> > > > The
> > >>> > > > > > > > standard
> > >>> > > > > > > > > way is to do this through JAAS. However, we will need
> > to
> > >>> > think
> > >>> > > > > > through
> > >>> > > > > > > if
> > >>> > > > > > > > > this is convenient in a shared environment. For
> > example,
> > >>> > when a
> > >>> > > > new
> > >>> > > > > > > task
> > >>> > > > > > > > is
> > >>> > > > > > > > > added to a Storm worker node, do we need to
> dynamically
> > >>> add a
> > >>> > > new
> > >>> > > > > > > section
> > >>> > > > > > > > > in the JAAS file? It may be more convenient if we can
> > >>> pass in
> > >>> > > the
> > >>> > > > > > token
> > >>> > > > > > > > > through the config directly w/o going through JAAS.
> > >>> > > > > > > > >
> > >>> > > > > > > > > Are you or Parth still actively working on this KIP?
> > >>> > > > > > > > >
> > >>> > > > > > > > > Thanks,
> > >>> > > > > > > > >
> > >>> > > > > > > > > Jun
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> > >>> jun@confluent.io>
> > >>> > > > wrote:
> > >>> > > > > > > > >
> > >>> > > > > > > > > > Just to add on that list.
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > 2. It would be good to document the format of the
> > data
> > >>> > stored
> > >>> > > > in
> > >>> > > > > > ZK.
> > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the
> > tokens
> > >>> > > should
> > >>> > > > > be
> > >>> > > > > > > > > > propagated through ZK like config/acl/quota, or
> > through
> > >>> the
> > >>> > > > > > > controller.
> > >>> > > > > > > > > > Currently, the controller is only designed for
> > >>> propagating
> > >>> > > > topic
> > >>> > > > > > > > > metadata,
> > >>> > > > > > > > > > but not other data.
> > >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
> > >>> > > DIGEST-MD5
> > >>> > > > > > since
> > >>> > > > > > > > it's
> > >>> > > > > > > > > > deprecated?
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Also, the images in the wiki seem broken.
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Thanks,
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Jun
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> > >>> > > > > gwen@confluent.io>
> > >>> > > > > > > > > wrote:
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >> From what I can see, remaining questions are:
> > >>> > > > > > > > > >>
> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
> > requester
> > >>> > only?
> > >>> > > > or
> > >>> > > > > > > using
> > >>> > > > > > > > > >> Kerberos auth only?
> > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> > >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> > >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't
> in
> > the
> > >>> > KIP
> > >>> > > > but
> > >>> > > > > > was
> > >>> > > > > > > > > >> discussed in this thread)?
> > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > >>> > > > > > > > > >>
> > >>> > > > > > > > > >> Gwen
> > >>> > > > > > > > > >>
> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> > >>> kafka@harsha.io>
> > >>> > > > wrote:
> > >>> > > > > > > > > >> > Jun & Ismael,
> > >>> > > > > > > > > >> > Unfortunately I
> couldn't
> > >>> attend
> > >>> > > the
> > >>> > > > > KIP
> > >>> > > > > > > > > meeting
> > >>> > > > > > > > > >> > when delegation tokens
> > >>> > discussed.
> > >>> > > > > > > > Appreciate
> > >>> > > > > > > > > if
> > >>> > > > > > > > > >> > you can update the
> > thread if
> > >>> > you
> > >>> > > > have
> > >>> > > > > > any
> > >>> > > > > > > > > >> > further questions.
> > >>> > > > > > > > > >> > Thanks,
> > >>> > > > > > > > > >> > Harsha
> > >>> > > > > > > > > >> >
> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei
> > wrote:
> > >>> > > > > > > > > >> >> It seems that the links to images in the KIP
> are
> > >>> > broken.
> > >>> > > > > > > > > >> >>
> > >>> > > > > > > > > >> >> Liquan
> > >>> > > > > > > > > >> >>
> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> > brahmbhatt <
> > >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > >>> > > > > > > > > >> >>
> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > >>> > > > > > > > > >> >> > In the current proposal we only allow a user
> to
> > >>> get
> > >>> > > > > > delegation
> > >>> > > > > > > > > token
> > >>> > > > > > > > > >> for
> > >>> > > > > > > > > >> >> > the identity that it authenticated as using
> > >>> another
> > >>> > > > > > mechanism,
> > >>> > > > > > > > i.e.
> > >>> > > > > > > > > >> A user
> > >>> > > > > > > > > >> >> > that authenticate using a keytab for
> principal
> > >>> > > > > > > user1@EXAMPLE.COM
> > >>> > > > > > > > > >> will get
> > >>> > > > > > > > > >> >> > delegation tokens for that user only. In
> > future I
> > >>> > think
> > >>> > > > we
> > >>> > > > > > will
> > >>> > > > > > > > > have
> > >>> > > > > > > > > >> to
> > >>> > > > > > > > > >> >> > extend support such that we allow some set of
> > >>> users (
> > >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > >>> > storm-nimbus@EXAMPLE.COM)
> > >>> > > > to
> > >>> > > > > > > > acquire
> > >>> > > > > > > > > >> >> > delegation tokens on behalf of other users
> > whose
> > >>> > > identity
> > >>> > > > > > they
> > >>> > > > > > > > have
> > >>> > > > > > > > > >> >> > verified independently. Kafka brokers will
> > have
> > >>> ACLs
> > >>> > > to
> > >>> > > > > > > control
> > >>> > > > > > > > > >> which
> > >>> > > > > > > > > >> >> > users are allowed to impersonate other users
> > and
> > >>> get
> > >>> > > > tokens
> > >>> > > > > > on
> > >>> > > > > > > > > >> behalf of
> > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
> > different
> > >>> > > problem
> > >>> > > > in
> > >>> > > > > > my
> > >>> > > > > > > > > >> opinion and
> > >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
> > >>> renewing
> > >>> > > > > > delegation
> > >>> > > > > > > > > >> tokens?
> > >>> > > > > > > > > >> >> > Typically this should be very very low, 1
> > request
> > >>> per
> > >>> > > > > minute
> > >>> > > > > > > is a
> > >>> > > > > > > > > >> >> > relatively high estimate. However it depends
> on
> > >>> the
> > >>> > > token
> > >>> > > > > > > > > >> expiration. I am
> > >>> > > > > > > > > >> >> > less worried about the extra load it puts on
> > >>> > controller
> > >>> > > > vs
> > >>> > > > > > the
> > >>> > > > > > > > > added
> > >>> > > > > > > > > >> >> > complexity and the value it offers.
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> > Thanks
> > >>> > > > > > > > > >> >> > Parth
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma
> <
> > >>> > > > > > > ismael@juma.me.uk>
> > >>> > > > > > > > > >> wrote:
> > >>> > > > > > > > > >> >> >
> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
> > >>> separate
> > >>> > > KIP
> > >>> > > > > as
> > >>> > > > > > it
> > >>> > > > > > > > > will
> > >>> > > > > > > > > >> >> > > introduce user visible changes. We could
> also
> > >>> > update
> > >>> > > > > KIP-48
> > >>> > > > > > > to
> > >>> > > > > > > > > >> have this
> > >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
> > >>> > > separately.
> > >>> > > > We
> > >>> > > > > > can
> > >>> > > > > > > > > >> discuss
> > >>> > > > > > > > > >> >> > that
> > >>> > > > > > > > > >> >> > > in the KIP call today.
> > >>> > > > > > > > > >> >> > >
> > >>> > > > > > > > > >> >> > > Ismael
> > >>> > > > > > > > > >> >> > >
> > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini
> > Sivaram
> > >>> <
> > >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > >>> > > > > > > > > >> >> > >
> > >>> > > > > > > > > >> >> > > > Ismael,
> > >>> > > > > > > > > >> >> > > >
> > >>> > > > > > > > > >> >> > > > I have created a JIRA (
> > >>> > > > > > > > > >> >> > https://issues.apache.org/
> > jira/browse/KAFKA-3751)
> > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism.
> Would
> > >>> that
> > >>> > > need
> > >>> > > > > > > another
> > >>> > > > > > > > > >> KIP? If
> > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this
> > just
> > >>> be
> > >>> > a
> > >>> > > > JIRA
> > >>> > > > > > > that
> > >>> > > > > > > > > gets
> > >>> > > > > > > > > >> >> > > reviewed
> > >>> > > > > > > > > >> >> > > > when the PR is ready?
> > >>> > > > > > > > > >> >> > > >
> > >>> > > > > > > > > >> >> > > > Thank you,
> > >>> > > > > > > > > >> >> > > >
> > >>> > > > > > > > > >> >> > > > Rajini
> > >>> > > > > > > > > >> >> > > >
> > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael
> > Juma <
> > >>> > > > > > > > > ismael@juma.me.uk>
> > >>> > > > > > > > > >> >> > wrote:
> > >>> > > > > > > > > >> >> > > >
> > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
> > >>> > candidate.
> > >>> > > > > > > > > >> >> > > > >
> > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this
> as
> > a
> > >>> SASL
> > >>> > > > > > mechanism
> > >>> > > > > > > > > that
> > >>> > > > > > > > > >> might
> > >>> > > > > > > > > >> >> > be
> > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been
> meaning
> > to
> > >>> > check
> > >>> > > > it
> > >>> > > > > in
> > >>> > > > > > > > more
> > >>> > > > > > > > > >> detail.
> > >>> > > > > > > > > >> >> > > Good
> > >>> > > > > > > > > >> >> > > > > to know that you are willing to
> > contribute
> > >>> an
> > >>> > > > > > > > implementation.
> > >>> > > > > > > > > >> Maybe
> > >>> > > > > > > > > >> >> > we
> > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
> > >>> > > > > > > > > >> >> > > > >
> > >>> > > > > > > > > >> >> > > > > Ismael
> > >>> > > > > > > > > >> >> > > > >
> > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
> > >>> > Sivaram <
> > >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > >>> > > > > > > > > >> >> > > > >
> > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> > >>> > Authentication
> > >>> > > > > > > > Mechanism)
> > >>> > > > > > > > > >> is a
> > >>> > > > > > > > > >> >> > > better
> > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java
> doesn't
> > >>> come
> > >>> > > > with a
> > >>> > > > > > > > > built-in
> > >>> > > > > > > > > >> SCRAM
> > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will
> be
> > >>> happy
> > >>> > > to
> > >>> > > > > add
> > >>> > > > > > > > > support
> > >>> > > > > > > > > >> in
> > >>> > > > > > > > > >> >> > Kafka
> > >>> > > > > > > > > >> >> > > > > since
> > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to
> > support
> > >>> > > anyway.
> > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
> > >>> > describes
> > >>> > > > the
> > >>> > > > > > > > protocol
> > >>> > > > > > > > > >> for
> > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > >>> > > > > > > > > >> >> > > > > >
> > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun
> > Rao <
> > >>> > > > > > > > jun@confluent.io
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >> wrote:
> > >>> > > > > > > > > >> >> > > > > >
> > >>> > > > > > > > > >> >> > > > > > > Parth,
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A
> couple
> > of
> > >>> > more
> > >>> > > > > > > questions.
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs
> > >>> mean?
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of
> > getting
> > >>> and
> > >>> > > > > > renewing
> > >>> > > > > > > > > >> delegation
> > >>> > > > > > > > > >> >> > > > tokens?
> > >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether
> > they
> > >>> > > should
> > >>> > > > be
> > >>> > > > > > > > > directed
> > >>> > > > > > > > > >> to the
> > >>> > > > > > > > > >> >> > > > > > > controller.
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > Jun
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM,
> > parth
> > >>> > > > > brahmbhatt <
> > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action
> to
> > add
> > >>> > acls
> > >>> > > > on
> > >>> > > > > > who
> > >>> > > > > > > > can
> > >>> > > > > > > > > >> request
> > >>> > > > > > > > > >> >> > > > > > delegation
> > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case
> > for
> > >>> that
> > >>> > > yet
> > >>> > > > > but
> > >>> > > > > > > > down
> > >>> > > > > > > > > >> the line
> > >>> > > > > > > > > >> >> > > > when
> > >>> > > > > > > > > >> >> > > > > we
> > >>> > > > > > > > > >> >> > > > > > > > start supporting
> > getDelegationTokenAs
> > >>> it
> > >>> > > will
> > >>> > > > > be
> > >>> > > > > > > > > >> necessary.
> > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be
> > only
> > >>> > > > > > > used/distributed
> > >>> > > > > > > > > >> over
> > >>> > > > > > > > > >> >> > secure
> > >>> > > > > > > > > >> >> > > > > > > channels.
> > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end
> > up
> > >>> > > choosing
> > >>> > > > > > > > > >> Invalidation will
> > >>> > > > > > > > > >> >> > > be
> > >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
> > >>> > > controller.
> > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
> > >>> somewhere
> > >>> > > > that
> > >>> > > > > > > > > >> invalidation
> > >>> > > > > > > > > >> >> > will
> > >>> > > > > > > > > >> >> > > > > > directly
> > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is
> > not
> > >>> the
> > >>> > > > > intent.
> > >>> > > > > > > > > >> Invalidation
> > >>> > > > > > > > > >> >> > > will
> > >>> > > > > > > > > >> >> > > > > > either
> > >>> > > > > > > > > >> >> > > > > > > > be request based or due to
> > >>> expiration. No
> > >>> > > > > direct
> > >>> > > > > > > > > >> zookeeper
> > >>> > > > > > > > > >> >> > > > > interaction
> > >>> > > > > > > > > >> >> > > > > > > from
> > >>> > > > > > > > > >> >> > > > > > > > any client.
> > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> > >>> DelegationToken
> > >>> > > > > without
> > >>> > > > > > > the
> > >>> > > > > > > > > >> hmac in
> > >>> > > > > > > > > >> >> > the
> > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
> > >>> confusion.
> > >>> > > The
> > >>> > > > > sole
> > >>> > > > > > > > > >> purpose of
> > >>> > > > > > > > > >> >> > > > > zookeeper
> > >>> > > > > > > > > >> >> > > > > > in
> > >>> > > > > > > > > >> >> > > > > > > > this design is as distribution
> > channel
> > >>> > for
> > >>> > > > > tokens
> > >>> > > > > > > > > >> between all
> > >>> > > > > > > > > >> >> > > > brokers
> > >>> > > > > > > > > >> >> > > > > > > and a
> > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens
> that
> > >>> were
> > >>> > > > > > generated
> > >>> > > > > > > by
> > >>> > > > > > > > > >> making a
> > >>> > > > > > > > > >> >> > > > > request
> > >>> > > > > > > > > >> >> > > > > > > to a
> > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on
> > this
> > >>> in
> > >>> > > > second
> > >>> > > > > > > > > >> paragraph). The
> > >>> > > > > > > > > >> >> > > > token
> > >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
> > >>> renewer,
> > >>> > > > uuid
> > >>> > > > > ,
> > >>> > > > > > > > > >> expiration,
> > >>> > > > > > > > > >> >> > > hmac)
> > >>> > > > > > > > > >> >> > > > ,
> > >>> > > > > > > > > >> >> > > > > > one
> > >>> > > > > > > > > >> >> > > > > > > of
> > >>> > > > > > > > > >> >> > > > > > > > which is the finally generated
> hmac
> > >>> but
> > >>> > > hmac
> > >>> > > > it
> > >>> > > > > > > self
> > >>> > > > > > > > is
> > >>> > > > > > > > > >> >> > derivable
> > >>> > > > > > > > > >> >> > > > if
> > >>> > > > > > > > > >> >> > > > > > you
> > >>> > > > > > > > > >> >> > > > > > > > have all the other elements of
> the
> > >>> token
> > >>> > +
> > >>> > > > > secret
> > >>> > > > > > > key
> > >>> > > > > > > > > to
> > >>> > > > > > > > > >> >> > generate
> > >>> > > > > > > > > >> >> > > > > hmac.
> > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide
> > SSL
> > >>> > > support
> > >>> > > > we
> > >>> > > > > > do
> > >>> > > > > > > > not
> > >>> > > > > > > > > >> want the
> > >>> > > > > > > > > >> >> > > > > entire
> > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
> > >>> zookeeper
> > >>> > > as
> > >>> > > > > that
> > >>> > > > > > > > will
> > >>> > > > > > > > > >> be an
> > >>> > > > > > > > > >> >> > > > insecure
> > >>> > > > > > > > > >> >> > > > > > > wire
> > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store
> all
> > >>> the
> > >>> > > other
> > >>> > > > > > > > elements
> > >>> > > > > > > > > >> of a
> > >>> > > > > > > > > >> >> > > > > delegation
> > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
> > >>> elements
> > >>> > and
> > >>> > > > > > because
> > >>> > > > > > > > > they
> > >>> > > > > > > > > >> also
> > >>> > > > > > > > > >> >> > > have
> > >>> > > > > > > > > >> >> > > > > > access
> > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able
> to
> > >>> > generate
> > >>> > > > > hmac
> > >>> > > > > > on
> > >>> > > > > > > > > >> their end.
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed
> is
> > to
> > >>> > avoid
> > >>> > > > > > > zookeeper
> > >>> > > > > > > > > >> >> > > altogether. A
> > >>> > > > > > > > > >> >> > > > > > > Client
> > >>> > > > > > > > > >> >> > > > > > > > will call broker with required
> > >>> > information
> > >>> > > > > > (owner,
> > >>> > > > > > > > > >> renwer,
> > >>> > > > > > > > > >> >> > > > > expiration)
> > >>> > > > > > > > > >> >> > > > > > > and
> > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid).
> > Broker
> > >>> > won't
> > >>> > > > > store
> > >>> > > > > > > this
> > >>> > > > > > > > > in
> > >>> > > > > > > > > >> >> > > zookeeper.
> > >>> > > > > > > > > >> >> > > > > > From
> > >>> > > > > > > > > >> >> > > > > > > > this point a client can contact
> any
> > >>> > broker
> > >>> > > > with
> > >>> > > > > > all
> > >>> > > > > > > > the
> > >>> > > > > > > > > >> >> > > delegation
> > >>> > > > > > > > > >> >> > > > > > token
> > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration,
> > hmac,
> > >>> > > uuid)
> > >>> > > > > the
> > >>> > > > > > > > borker
> > >>> > > > > > > > > >> will
> > >>> > > > > > > > > >> >> > > > > regenerate
> > >>> > > > > > > > > >> >> > > > > > > the
> > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches
> with
> > >>> hmac
> > >>> > > > > > presented
> > >>> > > > > > > by
> > >>> > > > > > > > > >> client ,
> > >>> > > > > > > > > >> >> > > > broker
> > >>> > > > > > > > > >> >> > > > > > > will
> > >>> > > > > > > > > >> >> > > > > > > > allow the request to
> authenticate.
> > >>> Only
> > >>> > > > > problem
> > >>> > > > > > > with
> > >>> > > > > > > > > >> this
> > >>> > > > > > > > > >> >> > > approach
> > >>> > > > > > > > > >> >> > > > > is
> > >>> > > > > > > > > >> >> > > > > > if
> > >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
> > >>> client
> > >>> > > can
> > >>> > > > > now
> > >>> > > > > > > > > generate
> > >>> > > > > > > > > >> >> > random
> > >>> > > > > > > > > >> >> > > > > tokens
> > >>> > > > > > > > > >> >> > > > > > > and
> > >>> > > > > > > > > >> >> > > > > > > > they will still be able to
> > >>> authenticate
> > >>> > as
> > >>> > > > any
> > >>> > > > > > user
> > >>> > > > > > > > > they
> > >>> > > > > > > > > >> like.
> > >>> > > > > > > > > >> >> > > with
> > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
> > >>> tokens
> > >>> > > > > acquired
> > >>> > > > > > > via
> > >>> > > > > > > > a
> > >>> > > > > > > > > >> broker
> > >>> > > > > > > > > >> >> > > > (using
> > >>> > > > > > > > > >> >> > > > > > some
> > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
> > >>> token)
> > >>> > > will
> > >>> > > > > be
> > >>> > > > > > > > > >> accepted. We
> > >>> > > > > > > > > >> >> > > need
> > >>> > > > > > > > > >> >> > > > to
> > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more
> > >>> sense
> > >>> > and
> > >>> > > > we
> > >>> > > > > > can
> > >>> > > > > > > go
> > >>> > > > > > > > > >> over it
> > >>> > > > > > > > > >> >> > in
> > >>> > > > > > > > > >> >> > > > > > > tomorrow's
> > >>> > > > > > > > > >> >> > > > > > > > meeting.
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite
> to
> > >>> me?
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > Thanks
> > >>> > > > > > > > > >> >> > > > > > > > Parth
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM,
> > Jun
> > >>> > Rao <
> > >>> > > > > > > > > >> jun@confluent.io>
> > >>> > > > > > > > > >> >> > > > wrote:
> > >>> > > > > > > > > >> >> > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
> > comments.
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be
> > useful
> > >>> for
> > >>> > > > Kafka
> > >>> > > > > > > > Connect
> > >>> > > > > > > > > >> and
> > >>> > > > > > > > > >> >> > Kafka
> > >>> > > > > > > > > >> >> > > > > rest
> > >>> > > > > > > > > >> >> > > > > > > > proxy
> > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need
> to
> > >>> run a
> > >>> > > > task
> > >>> > > > > on
> > >>> > > > > > > > > behalf
> > >>> > > > > > > > > >> of a
> > >>> > > > > > > > > >> >> > > > client.
> > >>> > > > > > > > > >> >> > > > > > We
> > >>> > > > > > > > > >> >> > > > > > > > will
> > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how those
> > >>> > services
> > >>> > > > use
> > >>> > > > > > > Kafka
> > >>> > > > > > > > > >> clients
> > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead
> of a
> > >>> > shared
> > >>> > > > > client
> > >>> > > > > > > per
> > >>> > > > > > > > > >> worker,
> > >>> > > > > > > > > >> >> > we
> > >>> > > > > > > > > >> >> > > > will
> > >>> > > > > > > > > >> >> > > > > > > need
> > >>> > > > > > > > > >> >> > > > > > > > a
> > >>> > > > > > > > > >> >> > > > > > > > > client per user task since the
> > >>> > > > authentication
> > >>> > > > > > > > happens
> > >>> > > > > > > > > >> at the
> > >>> > > > > > > > > >> >> > > > > > connection
> > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
> > >>> renewer
> > >>> > > will
> > >>> > > > be
> > >>> > > > > > the
> > >>> > > > > > > > > >> workers.
> > >>> > > > > > > > > >> >> > So,
> > >>> > > > > > > > > >> >> > > we
> > >>> > > > > > > > > >> >> > > > > > > > probably
> > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple
> renewers.
> > For
> > >>> > > Kafka
> > >>> > > > > rest
> > >>> > > > > > > > > proxy,
> > >>> > > > > > > > > >> the
> > >>> > > > > > > > > >> >> > > > renewer
> > >>> > > > > > > > > >> >> > > > > > can
> > >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of
> > the
> > >>> > > token.
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who
> > can
> > >>> > > request
> > >>> > > > > > > > delegation
> > >>> > > > > > > > > >> tokens?
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to
> > send
> > >>> > > > > delegation
> > >>> > > > > > > > tokens
> > >>> > > > > > > > > >> in an
> > >>> > > > > > > > > >> >> > > > > encrypted
> > >>> > > > > > > > > >> >> > > > > > > > > channel?
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for
> > expiring
> > >>> > > > tokens,
> > >>> > > > > > > every
> > >>> > > > > > > > > >> broker?
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens,
> > would
> > >>> it
> > >>> > be
> > >>> > > > > > better
> > >>> > > > > > > to
> > >>> > > > > > > > > do
> > >>> > > > > > > > > >> it in
> > >>> > > > > > > > > >> >> > a
> > >>> > > > > > > > > >> >> > > > > > request
> > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK
> directly?
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client
> in
> > >>> the
> > >>> > > wiki
> > >>> > > > > > > > sometimes
> > >>> > > > > > > > > >> refers
> > >>> > > > > > > > > >> >> > to
> > >>> > > > > > > > > >> >> > > > the
> > >>> > > > > > > > > >> >> > > > > > end
> > >>> > > > > > > > > >> >> > > > > > > > > client and some other times
> > refers
> > >>> to
> > >>> > the
> > >>> > > > > > client
> > >>> > > > > > > > > using
> > >>> > > > > > > > > >> the
> > >>> > > > > > > > > >> >> > > > > delegation
> > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
> > >>> > distinguish
> > >>> > > > > > between
> > >>> > > > > > > > the
> > >>> > > > > > > > > >> two.
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
> > sentence
> > >>> > > "Broker
> > >>> > > > > > also
> > >>> > > > > > > > > >> stores the
> > >>> > > > > > > > > >> >> > > > > > > > DelegationToken
> > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
> > zookeeper."
> > >>> a
> > >>> > bit
> > >>> > > > > > more? I
> > >>> > > > > > > > > >> thought the
> > >>> > > > > > > > > >> >> > > > > > > delegation
> > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > Thanks,
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > Jun
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22
> AM,
> > Jun
> > >>> > Rao
> > >>> > > <
> > >>> > > > > > > > > >> jun@confluent.io>
> > >>> > > > > > > > > >> >> > > > wrote:
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
> > >>> invite.
> > >>> > We
> > >>> > > > can
> > >>> > > > > > > > discuss
> > >>> > > > > > > > > >> this in
> > >>> > > > > > > > > >> >> > > the
> > >>> > > > > > > > > >> >> > > > > > > meeting
> > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > > Jun
> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47
> > AM,
> > >>> > > Harsha <
> > >>> > > > > > > > > >> kafka@harsha.io>
> > >>> > > > > > > > > >> >> > > > wrote:
> > >>> > > > > > > > > >> >> > > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
> > >>> meeting
> > >>> > > > > around
> > >>> > > > > > > > this.
> > >>> > > > > > > > > >> The KIP
> > >>> > > > > > > > > >> >> > is
> > >>> > > > > > > > > >> >> > > > up
> > >>> > > > > > > > > >> >> > > > > > for
> > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if
> > there
> > >>> are
> > >>> > > any
> > >>> > > > > > > > questions
> > >>> > > > > > > > > >> lets
> > >>> > > > > > > > > >> >> > > > quickly
> > >>> > > > > > > > > >> >> > > > > > hash
> > >>> > > > > > > > > >> >> > > > > > > > out
> > >>> > > > > > > > > >> >> > > > > > > > > >> details.
> > >>> > > > > > > > > >> >> > > > > > > > > >>
> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
> > >>> > > > > > > > > >> >> > > > > > > > > >>
> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at
> 08:40
> > >>> AM,
> > >>> > > parth
> > >>> > > > > > > > > brahmbhatt
> > >>> > > > > > > > > >> wrote:
> > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop
> echo
> > >>> > system
> > >>> > > > uses
> > >>> > > > > > so
> > >>> > > > > > > no
> > >>> > > > > > > > > >> good
> > >>> > > > > > > > > >> >> > reason
> > >>> > > > > > > > > >> >> > > > > > really.
> > >>> > > > > > > > > >> >> > > > > > > > We
> > >>> > > > > > > > > >> >> > > > > > > > > >> > could
> > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is
> the
> > >>> > newest
> > >>> > > > > > > > recommended
> > >>> > > > > > > > > >> standard
> > >>> > > > > > > > > >> >> > > is.
> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at
> 3:33
> > >>> AM,
> > >>> > > > Ismael
> > >>> > > > > > > Juma <
> > >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> > >>> > > > > > > > > >> >> > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > wrote:
> > >>> > > > > > > > > >> >> > > > > > > > > >> >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I
> only
> > >>> > started
> > >>> > > > > > > reviewing
> > >>> > > > > > > > > >> this and
> > >>> > > > > > > > > >> >> > > may
> > >>> > > > > > > > > >> >> > > > > have
> > >>> > > > > > > > > >> >> > > > > > > > > >> additional
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
> > >>> immediate
> > >>> > > > > question
> > >>> > > > > > > that
> > >>> > > > > > > > > >> came to
> > >>> > > > > > > > > >> >> > > mind
> > >>> > > > > > > > > >> >> > > > is
> > >>> > > > > > > > > >> >> > > > > > our
> > >>> > > > > > > > > >> >> > > > > > > > > >> choice of
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though
> > it's
> > >>> > > marked
> > >>> > > > > as
> > >>> > > > > > > > > >> OBSOLETE in
> > >>> > > > > > > > > >> >> > the
> > >>> > > > > > > > > >> >> > > > IANA
> > >>> > > > > > > > > >> >> > > > > > > > > Registry
> > >>> > > > > > > > > >> >> > > > > > > > > >> of
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
> > >>> original
> > >>> > > RFC
> > >>> > > > > > > (2831)
> > >>> > > > > > > > > has
> > >>> > > > > > > > > >> been
> > >>> > > > > > > > > >> >> > > moved
> > >>> > > > > > > > > >> >> > > > > to
> > >>> > > > > > > > > >> >> > > > > > > > > Historic
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > https://tools.ietf.org/html/
> > >>> > > rfc6331
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > >
> > >>> > > > > > > > > >> >> > >
> > >>> > > > > > > > > >>
> > >>> > > > > > >
> > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
> > >>> mechanisms.xhtml
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning
> > behind
> > >>> > that
> > >>> > > > > > choice?
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at
> > 11:29
> > >>> > PM,
> > >>> > > > Gwen
> > >>> > > > > > > > > Shapira <
> > >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> > >>> > > > > > > > > >> >> > > > > > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
> > >>> > > > > > > > > >> >> > > > > > > > > >> > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline
> :)
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to
> emphasize
> > >>> that
> > >>> > > even
> > >>> > > > > > though
> > >>> > > > > > > > > >> delegation
> > >>> > > > > > > > > >> >> > > > tokens
> > >>> > > > > > > > > >> >> > > > > > > are a
> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel
> > very
> > >>> > > strongly
> > >>> > > > > > about
> > >>> > > > > > > > not
> > >>> > > > > > > > > >> adding
> > >>> > > > > > > > > >> >> > > > > > dependency
> > >>> > > > > > > > > >> >> > > > > > > > on
> > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
> > >>> delegation
> > >>> > > > > tokens
> > >>> > > > > > > for
> > >>> > > > > > > > > >> Kafka. The
> > >>> > > > > > > > > >> >> > > KIP
> > >>> > > > > > > > > >> >> > > > > > > doesn't
> > >>> > > > > > > > > >> >> > > > > > > > > >> imply
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but
> > if
> > >>> you
> > >>> > > can
> > >>> > > > > > > > clarify...
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
> > dependency.*
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to
> > the
> > >>> KIP
> > >>> > so
> > >>> > > > no
> > >>> > > > > > one
> > >>> > > > > > > > will
> > >>> > > > > > > > > >> read
> > >>> > > > > > > > > >> >> > the
> > >>> > > > > > > > > >> >> > > > KIP
> > >>> > > > > > > > > >> >> > > > > > and
> > >>> > > > > > > > > >> >> > > > > > > > > panic
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the
> > next
> > >>> > > > > release...
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
> > delegation
> > >>> > token
> > >>> > > at
> > >>> > > > > any
> > >>> > > > > > > > time
> > >>> > > > > > > > > >> after
> > >>> > > > > > > > > >> >> > > > > > > > authenticating?
> > >>> > > > > > > > > >> >> > > > > > > > > >> only
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> > >>> > > > authenticated
> > >>> > > > > > you
> > >>> > > > > > > > can
> > >>> > > > > > > > > >> get
> > >>> > > > > > > > > >> >> > > > delegation
> > >>> > > > > > > > > >> >> > > > > > > > tokens.
> > >>> > > > > > > > > >> >> > > > > > > > > >> We
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > need
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> > >>> > > > authenticated
> > >>> > > > > > > using
> > >>> > > > > > > > > >> delegation
> > >>> > > > > > > > > >> >> > > > > token,
> > >>> > > > > > > > > >> >> > > > > > > can
> > >>> > > > > > > > > >> >> > > > > > > > > also
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token
> > again or
> > >>> > not.
> > >>> > > > > Also
> > >>> > > > > > > > there
> > >>> > > > > > > > > >> is the
> > >>> > > > > > > > > >> >> > > > > question
> > >>> > > > > > > > > >> >> > > > > > of
> > >>> > > > > > > > > >> >> > > > > > > > do
> > >>> > > > > > > > > >> >> > > > > > > > > we
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
> > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
> > >>> delegation
> > >>> > > > token
> > >>> > > > > > or
> > >>> > > > > > > we
> > >>> > > > > > > > > >> want
> > >>> > > > > > > > > >> >> > > specific
> > >>> > > > > > > > > >>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ashish Singh <as...@cloudera.com>.
I think we decided to not support secret rotation, I guess this can be
stated clearly on the KIP. Also, more details on how clients will perform
token distribution and how CLI will look like will be helpful.
On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Hi Guys,
>
> This discussion was dead for a while. Are there still contentious
> points? If not, why are there no votes?
>
> On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> > Ashish,
> >
> > Yes, I will send out a KIP invite for next week to discuss KIP-48 and
> other
> > remaining KIPs.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com>
> wrote:
> >
> >> Thanks Harsha!
> >>
> >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
> >> actually make a call on when we should have next KIP call. As there are
> a
> >> few outstanding KIPs that could not be discussed this week, can we have
> a
> >> KIP hangout call next week?
> >>
> >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io>
> >> wrote:
> >>
> >>> Ashish,
> >>> Yes we are working on it. Lets discuss in the next KIP meeting.
> >>> I'll join.
> >>> -Harsha
> >>>
> >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
> >>> wrote:
> >>>
> >>> > Hello Harsha,
> >>> >
> >>> > Are you still working on this? Wondering if we can discuss this in
> next
> >>> KIP
> >>> > meeting, if you can join.
> >>> >
> >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <
> kafka@harsha.io>
> >>> > wrote:
> >>> >
> >>> > > Hi Grant,
> >>> > > We are working on it. Will add the details to KIP about
> the
> >>> > > request protocol.
> >>> > >
> >>> > > Thanks,
> >>> > > Harsha
> >>> > >
> >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com>
> >>> wrote:
> >>> > >
> >>> > > > Hi Parth,
> >>> > > >
> >>> > > > Are you still working on this? If you need any help please don't
> >>> > hesitate
> >>> > > > to ask.
> >>> > > >
> >>> > > > Thanks,
> >>> > > > Grant
> >>> > > >
> >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io>
> wrote:
> >>> > > >
> >>> > > > > Parth,
> >>> > > > >
> >>> > > > > Thanks for the reply.
> >>> > > > >
> >>> > > > > It makes sense to only allow the renewal by users that
> >>> authenticated
> >>> > > > using
> >>> > > > > *non* delegation token mechanism. Then, should we make the
> >>> renewal a
> >>> > > > list?
> >>> > > > > For example, in the case of rest proxy, it will be useful for
> >>> every
> >>> > > > > instance of rest proxy to be able to renew the tokens.
> >>> > > > >
> >>> > > > > It would be clearer if we can document the request protocol
> like
> >>> > > > >
> >>> > > > >
> >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> >>> > > Commandlineandcentralizedadministrativeoperations-
> >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> >>> > > > > .
> >>> > > > >
> >>> > > > > It would also be useful to document the client APIs.
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > >
> >>> > > > > Jun
> >>> > > > >
> >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> >>> > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > >
> >>> > > > > > Hi,
> >>> > > > > >
> >>> > > > > > I am suggesting that we will only allow the renewal by users
> >>> that
> >>> > > > > > authenticated using *non* delegation token mechanism. For
> >>> example,
> >>> > If
> >>> > > > > user
> >>> > > > > > Alice authenticated using kerberos and requested delegation
> >>> tokens,
> >>> > > > only
> >>> > > > > > user Alice authenticated via non delegation token mechanism
> can
> >>> > > renew.
> >>> > > > > > Clients that have access to delegation tokens can not issue
> >>> > renewal
> >>> > > > > > request for renewing their own token and this is primarily
> >>> > important
> >>> > > to
> >>> > > > > > reduce the time window for which a compromised token will be
> >>> valid.
> >>> > > > > >
> >>> > > > > > To clarify, Yes any authenticated user can request delegation
> >>> > tokens
> >>> > > > but
> >>> > > > > > even here I would recommend to avoid creating a chain where a
> >>> > client
> >>> > > > > > authenticated via delegation token request for more
> delegation
> >>> > > tokens.
> >>> > > > > > Basically anyone can request delegation token, as long as
> they
> >>> > > > > authenticate
> >>> > > > > > via a non delegation token mechanism.
> >>> > > > > >
> >>> > > > > > Aren't classes listed here
> >>> > > > > > <
> >>> > > > > >
> >>> > > > >
> >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
> >>> upportforKaf
> >>> > > ka-PublicInterfaces
> >>> > > > > > >
> >>> > > > > > sufficient?
> >>> > > > > >
> >>> > > > > > Thanks
> >>> > > > > > Parth
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io>
> >>> wrote:
> >>> > > > > >
> >>> > > > > > > Parth,
> >>> > > > > > >
> >>> > > > > > > Thanks for the reply. A couple of comments inline below.
> >>> > > > > > >
> >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> >>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > >
> >>> > > > > > > > 1. Who / how are tokens renewed? By original requester
> >>> only? or
> >>> > > > using
> >>> > > > > > > > Kerberos
> >>> > > > > > > > auth only?
> >>> > > > > > > > My recommendation is to do this only using Kerberos auth
> and
> >>> > only
> >>> > > > > threw
> >>> > > > > > > the
> >>> > > > > > > > renewer specified during the acquisition request.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > Hmm, not sure that I follow this. Are you saying that any
> >>> client
> >>> > > > > > > authenticated with the delegation token can renew, i.e.
> there
> >>> is
> >>> > no
> >>> > > > > > renewer
> >>> > > > > > > needed?
> >>> > > > > > >
> >>> > > > > > > Also, just to be clear, any authenticated client (either
> >>> through
> >>> > > SASL
> >>> > > > > or
> >>> > > > > > > SSL) can request a delegation token for the authenticated
> >>> user,
> >>> > > > right?
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> >>> > > > > > > > My recommendation is still to store in ZK or not store
> them
> >>> at
> >>> > > all.
> >>> > > > > The
> >>> > > > > > > > whole controller based distribution is too much overhead
> >>> with
> >>> > not
> >>> > > > > much
> >>> > > > > > to
> >>> > > > > > > > achieve.
> >>> > > > > > > >
> >>> > > > > > > > 3. How are tokens invalidated / expired?
> >>> > > > > > > > Either by expiration time out or through an explicit
> >>> request to
> >>> > > > > > > invalidate.
> >>> > > > > > > >
> >>> > > > > > > > 4. Which encryption algorithm is used?
> >>> > > > > > > > SCRAM
> >>> > > > > > > >
> >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the
> KIP
> >>> but
> >>> > > was
> >>> > > > > > > > discussed
> >>> > > > > > > > in this thread)?
> >>> > > > > > > > There is no imperonation proposal. I tried and explained
> how
> >>> > its
> >>> > > a
> >>> > > > > > > > different problem and why its not really necessary to
> >>> discuss
> >>> > > that
> >>> > > > as
> >>> > > > > > > part
> >>> > > > > > > > of this KIP. This KIP will not support any
> impersonation,
> >>> it
> >>> > > will
> >>> > > > > just
> >>> > > > > > > be
> >>> > > > > > > > another way to authenticate.
> >>> > > > > > > >
> >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> >>> > > > > > > > We do not need new ACLs.
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > Could we document the format of the new request/response
> and
> >>> > their
> >>> > > > > > > associated Resource and Operation for ACL?
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > > 7. How would the delegation token be configured in the
> >>> client?
> >>> > > > > > > > Should be through config. I wasn't planning on supporting
> >>> JAAS
> >>> > > for
> >>> > > > > > > tokens.
> >>> > > > > > > > I don't believe hadoop does this either.
> >>> > > > > > > >
> >>> > > > > > > > Thanks
> >>> > > > > > > > Parth
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <
> jun@confluent.io>
> >>> > > wrote:
> >>> > > > > > > >
> >>> > > > > > > > > Harsha,
> >>> > > > > > > > >
> >>> > > > > > > > > Another question.
> >>> > > > > > > > >
> >>> > > > > > > > > 9. How would the delegation token be configured in the
> >>> > client?
> >>> > > > The
> >>> > > > > > > > standard
> >>> > > > > > > > > way is to do this through JAAS. However, we will need
> to
> >>> > think
> >>> > > > > > through
> >>> > > > > > > if
> >>> > > > > > > > > this is convenient in a shared environment. For
> example,
> >>> > when a
> >>> > > > new
> >>> > > > > > > task
> >>> > > > > > > > is
> >>> > > > > > > > > added to a Storm worker node, do we need to dynamically
> >>> add a
> >>> > > new
> >>> > > > > > > section
> >>> > > > > > > > > in the JAAS file? It may be more convenient if we can
> >>> pass in
> >>> > > the
> >>> > > > > > token
> >>> > > > > > > > > through the config directly w/o going through JAAS.
> >>> > > > > > > > >
> >>> > > > > > > > > Are you or Parth still actively working on this KIP?
> >>> > > > > > > > >
> >>> > > > > > > > > Thanks,
> >>> > > > > > > > >
> >>> > > > > > > > > Jun
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
> >>> jun@confluent.io>
> >>> > > > wrote:
> >>> > > > > > > > >
> >>> > > > > > > > > > Just to add on that list.
> >>> > > > > > > > > >
> >>> > > > > > > > > > 2. It would be good to document the format of the
> data
> >>> > stored
> >>> > > > in
> >>> > > > > > ZK.
> >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the
> tokens
> >>> > > should
> >>> > > > > be
> >>> > > > > > > > > > propagated through ZK like config/acl/quota, or
> through
> >>> the
> >>> > > > > > > controller.
> >>> > > > > > > > > > Currently, the controller is only designed for
> >>> propagating
> >>> > > > topic
> >>> > > > > > > > > metadata,
> >>> > > > > > > > > > but not other data.
> >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
> >>> > > DIGEST-MD5
> >>> > > > > > since
> >>> > > > > > > > it's
> >>> > > > > > > > > > deprecated?
> >>> > > > > > > > > >
> >>> > > > > > > > > > Also, the images in the wiki seem broken.
> >>> > > > > > > > > >
> >>> > > > > > > > > > Thanks,
> >>> > > > > > > > > >
> >>> > > > > > > > > > Jun
> >>> > > > > > > > > >
> >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> >>> > > > > gwen@confluent.io>
> >>> > > > > > > > > wrote:
> >>> > > > > > > > > >
> >>> > > > > > > > > >> From what I can see, remaining questions are:
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original
> requester
> >>> > only?
> >>> > > > or
> >>> > > > > > > using
> >>> > > > > > > > > >> Kerberos auth only?
> >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> >>> > > > > > > > > >> 3. How are tokens invalidated / expired?
> >>> > > > > > > > > >> 4. Which encryption algorithm is used?
> >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in
> the
> >>> > KIP
> >>> > > > but
> >>> > > > > > was
> >>> > > > > > > > > >> discussed in this thread)?
> >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> Gwen
> >>> > > > > > > > > >>
> >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
> >>> kafka@harsha.io>
> >>> > > > wrote:
> >>> > > > > > > > > >> > Jun & Ismael,
> >>> > > > > > > > > >> > Unfortunately I couldn't
> >>> attend
> >>> > > the
> >>> > > > > KIP
> >>> > > > > > > > > meeting
> >>> > > > > > > > > >> > when delegation tokens
> >>> > discussed.
> >>> > > > > > > > Appreciate
> >>> > > > > > > > > if
> >>> > > > > > > > > >> > you can update the
> thread if
> >>> > you
> >>> > > > have
> >>> > > > > > any
> >>> > > > > > > > > >> > further questions.
> >>> > > > > > > > > >> > Thanks,
> >>> > > > > > > > > >> > Harsha
> >>> > > > > > > > > >> >
> >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei
> wrote:
> >>> > > > > > > > > >> >> It seems that the links to images in the KIP are
> >>> > broken.
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> Liquan
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth
> brahmbhatt <
> >>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > > > > >> >>
> >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> >>> > > > > > > > > >> >> > In the current proposal we only allow a user to
> >>> get
> >>> > > > > > delegation
> >>> > > > > > > > > token
> >>> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > the identity that it authenticated as using
> >>> another
> >>> > > > > > mechanism,
> >>> > > > > > > > i.e.
> >>> > > > > > > > > >> A user
> >>> > > > > > > > > >> >> > that authenticate using a keytab for principal
> >>> > > > > > > user1@EXAMPLE.COM
> >>> > > > > > > > > >> will get
> >>> > > > > > > > > >> >> > delegation tokens for that user only. In
> future I
> >>> > think
> >>> > > > we
> >>> > > > > > will
> >>> > > > > > > > > have
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > extend support such that we allow some set of
> >>> users (
> >>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> >>> > storm-nimbus@EXAMPLE.COM)
> >>> > > > to
> >>> > > > > > > > acquire
> >>> > > > > > > > > >> >> > delegation tokens on behalf of other users
> whose
> >>> > > identity
> >>> > > > > > they
> >>> > > > > > > > have
> >>> > > > > > > > > >> >> > verified independently. Kafka brokers will
> have
> >>> ACLs
> >>> > > to
> >>> > > > > > > control
> >>> > > > > > > > > >> which
> >>> > > > > > > > > >> >> > users are allowed to impersonate other users
> and
> >>> get
> >>> > > > tokens
> >>> > > > > > on
> >>> > > > > > > > > >> behalf of
> >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole
> different
> >>> > > problem
> >>> > > > in
> >>> > > > > > my
> >>> > > > > > > > > >> opinion and
> >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
> >>> renewing
> >>> > > > > > delegation
> >>> > > > > > > > > >> tokens?
> >>> > > > > > > > > >> >> > Typically this should be very very low, 1
> request
> >>> per
> >>> > > > > minute
> >>> > > > > > > is a
> >>> > > > > > > > > >> >> > relatively high estimate. However it depends on
> >>> the
> >>> > > token
> >>> > > > > > > > > >> expiration. I am
> >>> > > > > > > > > >> >> > less worried about the extra load it puts on
> >>> > controller
> >>> > > > vs
> >>> > > > > > the
> >>> > > > > > > > > added
> >>> > > > > > > > > >> >> > complexity and the value it offers.
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > Thanks
> >>> > > > > > > > > >> >> > Parth
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> >>> > > > > > > ismael@juma.me.uk>
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> >
> >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
> >>> separate
> >>> > > KIP
> >>> > > > > as
> >>> > > > > > it
> >>> > > > > > > > > will
> >>> > > > > > > > > >> >> > > introduce user visible changes. We could also
> >>> > update
> >>> > > > > KIP-48
> >>> > > > > > > to
> >>> > > > > > > > > >> have this
> >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
> >>> > > separately.
> >>> > > > We
> >>> > > > > > can
> >>> > > > > > > > > >> discuss
> >>> > > > > > > > > >> >> > that
> >>> > > > > > > > > >> >> > > in the KIP call today.
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > Ismael
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini
> Sivaram
> >>> <
> >>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >> >> > > > Ismael,
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > I have created a JIRA (
> >>> > > > > > > > > >> >> > https://issues.apache.org/
> jira/browse/KAFKA-3751)
> >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would
> >>> that
> >>> > > need
> >>> > > > > > > another
> >>> > > > > > > > > >> KIP? If
> >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this
> just
> >>> be
> >>> > a
> >>> > > > JIRA
> >>> > > > > > > that
> >>> > > > > > > > > gets
> >>> > > > > > > > > >> >> > > reviewed
> >>> > > > > > > > > >> >> > > > when the PR is ready?
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > Thank you,
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > Rajini
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael
> Juma <
> >>> > > > > > > > > ismael@juma.me.uk>
> >>> > > > > > > > > >> >> > wrote:
> >>> > > > > > > > > >> >> > > >
> >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
> >>> > candidate.
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as
> a
> >>> SASL
> >>> > > > > > mechanism
> >>> > > > > > > > > that
> >>> > > > > > > > > >> might
> >>> > > > > > > > > >> >> > be
> >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning
> to
> >>> > check
> >>> > > > it
> >>> > > > > in
> >>> > > > > > > > more
> >>> > > > > > > > > >> detail.
> >>> > > > > > > > > >> >> > > Good
> >>> > > > > > > > > >> >> > > > > to know that you are willing to
> contribute
> >>> an
> >>> > > > > > > > implementation.
> >>> > > > > > > > > >> Maybe
> >>> > > > > > > > > >> >> > we
> >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > Ismael
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
> >>> > Sivaram <
> >>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> >>> > > > > > > > > >> >> > > > >
> >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> >>> > Authentication
> >>> > > > > > > > Mechanism)
> >>> > > > > > > > > >> is a
> >>> > > > > > > > > >> >> > > better
> >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't
> >>> come
> >>> > > > with a
> >>> > > > > > > > > built-in
> >>> > > > > > > > > >> SCRAM
> >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be
> >>> happy
> >>> > > to
> >>> > > > > add
> >>> > > > > > > > > support
> >>> > > > > > > > > >> in
> >>> > > > > > > > > >> >> > Kafka
> >>> > > > > > > > > >> >> > > > > since
> >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to
> support
> >>> > > anyway.
> >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
> >>> > describes
> >>> > > > the
> >>> > > > > > > > protocol
> >>> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun
> Rao <
> >>> > > > > > > > jun@confluent.io
> >>> > > > > > > > > >
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > > > > > > Parth,
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple
> of
> >>> > more
> >>> > > > > > > questions.
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs
> >>> mean?
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of
> getting
> >>> and
> >>> > > > > > renewing
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > tokens?
> >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether
> they
> >>> > > should
> >>> > > > be
> >>> > > > > > > > > directed
> >>> > > > > > > > > >> to the
> >>> > > > > > > > > >> >> > > > > > > controller.
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM,
> parth
> >>> > > > > brahmbhatt <
> >>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Hi Jun,
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to
> add
> >>> > acls
> >>> > > > on
> >>> > > > > > who
> >>> > > > > > > > can
> >>> > > > > > > > > >> request
> >>> > > > > > > > > >> >> > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case
> for
> >>> that
> >>> > > yet
> >>> > > > > but
> >>> > > > > > > > down
> >>> > > > > > > > > >> the line
> >>> > > > > > > > > >> >> > > > when
> >>> > > > > > > > > >> >> > > > > we
> >>> > > > > > > > > >> >> > > > > > > > start supporting
> getDelegationTokenAs
> >>> it
> >>> > > will
> >>> > > > > be
> >>> > > > > > > > > >> necessary.
> >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be
> only
> >>> > > > > > > used/distributed
> >>> > > > > > > > > >> over
> >>> > > > > > > > > >> >> > secure
> >>> > > > > > > > > >> >> > > > > > > channels.
> >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end
> up
> >>> > > choosing
> >>> > > > > > > > > >> Invalidation will
> >>> > > > > > > > > >> >> > > be
> >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
> >>> > > controller.
> >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
> >>> somewhere
> >>> > > > that
> >>> > > > > > > > > >> invalidation
> >>> > > > > > > > > >> >> > will
> >>> > > > > > > > > >> >> > > > > > directly
> >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is
> not
> >>> the
> >>> > > > > intent.
> >>> > > > > > > > > >> Invalidation
> >>> > > > > > > > > >> >> > > will
> >>> > > > > > > > > >> >> > > > > > either
> >>> > > > > > > > > >> >> > > > > > > > be request based or due to
> >>> expiration. No
> >>> > > > > direct
> >>> > > > > > > > > >> zookeeper
> >>> > > > > > > > > >> >> > > > > interaction
> >>> > > > > > > > > >> >> > > > > > > from
> >>> > > > > > > > > >> >> > > > > > > > any client.
> >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> >>> DelegationToken
> >>> > > > > without
> >>> > > > > > > the
> >>> > > > > > > > > >> hmac in
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
> >>> confusion.
> >>> > > The
> >>> > > > > sole
> >>> > > > > > > > > >> purpose of
> >>> > > > > > > > > >> >> > > > > zookeeper
> >>> > > > > > > > > >> >> > > > > > in
> >>> > > > > > > > > >> >> > > > > > > > this design is as distribution
> channel
> >>> > for
> >>> > > > > tokens
> >>> > > > > > > > > >> between all
> >>> > > > > > > > > >> >> > > > brokers
> >>> > > > > > > > > >> >> > > > > > > and a
> >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that
> >>> were
> >>> > > > > > generated
> >>> > > > > > > by
> >>> > > > > > > > > >> making a
> >>> > > > > > > > > >> >> > > > > request
> >>> > > > > > > > > >> >> > > > > > > to a
> >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on
> this
> >>> in
> >>> > > > second
> >>> > > > > > > > > >> paragraph). The
> >>> > > > > > > > > >> >> > > > token
> >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
> >>> renewer,
> >>> > > > uuid
> >>> > > > > ,
> >>> > > > > > > > > >> expiration,
> >>> > > > > > > > > >> >> > > hmac)
> >>> > > > > > > > > >> >> > > > ,
> >>> > > > > > > > > >> >> > > > > > one
> >>> > > > > > > > > >> >> > > > > > > of
> >>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac
> >>> but
> >>> > > hmac
> >>> > > > it
> >>> > > > > > > self
> >>> > > > > > > > is
> >>> > > > > > > > > >> >> > derivable
> >>> > > > > > > > > >> >> > > > if
> >>> > > > > > > > > >> >> > > > > > you
> >>> > > > > > > > > >> >> > > > > > > > have all the other elements of the
> >>> token
> >>> > +
> >>> > > > > secret
> >>> > > > > > > key
> >>> > > > > > > > > to
> >>> > > > > > > > > >> >> > generate
> >>> > > > > > > > > >> >> > > > > hmac.
> >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide
> SSL
> >>> > > support
> >>> > > > we
> >>> > > > > > do
> >>> > > > > > > > not
> >>> > > > > > > > > >> want the
> >>> > > > > > > > > >> >> > > > > entire
> >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
> >>> zookeeper
> >>> > > as
> >>> > > > > that
> >>> > > > > > > > will
> >>> > > > > > > > > >> be an
> >>> > > > > > > > > >> >> > > > insecure
> >>> > > > > > > > > >> >> > > > > > > wire
> >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all
> >>> the
> >>> > > other
> >>> > > > > > > > elements
> >>> > > > > > > > > >> of a
> >>> > > > > > > > > >> >> > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
> >>> elements
> >>> > and
> >>> > > > > > because
> >>> > > > > > > > > they
> >>> > > > > > > > > >> also
> >>> > > > > > > > > >> >> > > have
> >>> > > > > > > > > >> >> > > > > > access
> >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to
> >>> > generate
> >>> > > > > hmac
> >>> > > > > > on
> >>> > > > > > > > > >> their end.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is
> to
> >>> > avoid
> >>> > > > > > > zookeeper
> >>> > > > > > > > > >> >> > > altogether. A
> >>> > > > > > > > > >> >> > > > > > > Client
> >>> > > > > > > > > >> >> > > > > > > > will call broker with required
> >>> > information
> >>> > > > > > (owner,
> >>> > > > > > > > > >> renwer,
> >>> > > > > > > > > >> >> > > > > expiration)
> >>> > > > > > > > > >> >> > > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid).
> Broker
> >>> > won't
> >>> > > > > store
> >>> > > > > > > this
> >>> > > > > > > > > in
> >>> > > > > > > > > >> >> > > zookeeper.
> >>> > > > > > > > > >> >> > > > > > From
> >>> > > > > > > > > >> >> > > > > > > > this point a client can contact any
> >>> > broker
> >>> > > > with
> >>> > > > > > all
> >>> > > > > > > > the
> >>> > > > > > > > > >> >> > > delegation
> >>> > > > > > > > > >> >> > > > > > token
> >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration,
> hmac,
> >>> > > uuid)
> >>> > > > > the
> >>> > > > > > > > borker
> >>> > > > > > > > > >> will
> >>> > > > > > > > > >> >> > > > > regenerate
> >>> > > > > > > > > >> >> > > > > > > the
> >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with
> >>> hmac
> >>> > > > > > presented
> >>> > > > > > > by
> >>> > > > > > > > > >> client ,
> >>> > > > > > > > > >> >> > > > broker
> >>> > > > > > > > > >> >> > > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate.
> >>> Only
> >>> > > > > problem
> >>> > > > > > > with
> >>> > > > > > > > > >> this
> >>> > > > > > > > > >> >> > > approach
> >>> > > > > > > > > >> >> > > > > is
> >>> > > > > > > > > >> >> > > > > > if
> >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
> >>> client
> >>> > > can
> >>> > > > > now
> >>> > > > > > > > > generate
> >>> > > > > > > > > >> >> > random
> >>> > > > > > > > > >> >> > > > > tokens
> >>> > > > > > > > > >> >> > > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > they will still be able to
> >>> authenticate
> >>> > as
> >>> > > > any
> >>> > > > > > user
> >>> > > > > > > > > they
> >>> > > > > > > > > >> like.
> >>> > > > > > > > > >> >> > > with
> >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
> >>> tokens
> >>> > > > > acquired
> >>> > > > > > > via
> >>> > > > > > > > a
> >>> > > > > > > > > >> broker
> >>> > > > > > > > > >> >> > > > (using
> >>> > > > > > > > > >> >> > > > > > some
> >>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
> >>> token)
> >>> > > will
> >>> > > > > be
> >>> > > > > > > > > >> accepted. We
> >>> > > > > > > > > >> >> > > need
> >>> > > > > > > > > >> >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more
> >>> sense
> >>> > and
> >>> > > > we
> >>> > > > > > can
> >>> > > > > > > go
> >>> > > > > > > > > >> over it
> >>> > > > > > > > > >> >> > in
> >>> > > > > > > > > >> >> > > > > > > tomorrow's
> >>> > > > > > > > > >> >> > > > > > > > meeting.
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to
> >>> me?
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > Thanks
> >>> > > > > > > > > >> >> > > > > > > > Parth
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM,
> Jun
> >>> > Rao <
> >>> > > > > > > > > >> jun@confluent.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few
> comments.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be
> useful
> >>> for
> >>> > > > Kafka
> >>> > > > > > > > Connect
> >>> > > > > > > > > >> and
> >>> > > > > > > > > >> >> > Kafka
> >>> > > > > > > > > >> >> > > > > rest
> >>> > > > > > > > > >> >> > > > > > > > proxy
> >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to
> >>> run a
> >>> > > > task
> >>> > > > > on
> >>> > > > > > > > > behalf
> >>> > > > > > > > > >> of a
> >>> > > > > > > > > >> >> > > > client.
> >>> > > > > > > > > >> >> > > > > > We
> >>> > > > > > > > > >> >> > > > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > > likely need to change how those
> >>> > services
> >>> > > > use
> >>> > > > > > > Kafka
> >>> > > > > > > > > >> clients
> >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
> >>> > shared
> >>> > > > > client
> >>> > > > > > > per
> >>> > > > > > > > > >> worker,
> >>> > > > > > > > > >> >> > we
> >>> > > > > > > > > >> >> > > > will
> >>> > > > > > > > > >> >> > > > > > > need
> >>> > > > > > > > > >> >> > > > > > > > a
> >>> > > > > > > > > >> >> > > > > > > > > client per user task since the
> >>> > > > authentication
> >>> > > > > > > > happens
> >>> > > > > > > > > >> at the
> >>> > > > > > > > > >> >> > > > > > connection
> >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
> >>> renewer
> >>> > > will
> >>> > > > be
> >>> > > > > > the
> >>> > > > > > > > > >> workers.
> >>> > > > > > > > > >> >> > So,
> >>> > > > > > > > > >> >> > > we
> >>> > > > > > > > > >> >> > > > > > > > probably
> >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers.
> For
> >>> > > Kafka
> >>> > > > > rest
> >>> > > > > > > > > proxy,
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > renewer
> >>> > > > > > > > > >> >> > > > > > can
> >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of
> the
> >>> > > token.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who
> can
> >>> > > request
> >>> > > > > > > > delegation
> >>> > > > > > > > > >> tokens?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to
> send
> >>> > > > > delegation
> >>> > > > > > > > tokens
> >>> > > > > > > > > >> in an
> >>> > > > > > > > > >> >> > > > > encrypted
> >>> > > > > > > > > >> >> > > > > > > > > channel?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for
> expiring
> >>> > > > tokens,
> >>> > > > > > > every
> >>> > > > > > > > > >> broker?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens,
> would
> >>> it
> >>> > be
> >>> > > > > > better
> >>> > > > > > > to
> >>> > > > > > > > > do
> >>> > > > > > > > > >> it in
> >>> > > > > > > > > >> >> > a
> >>> > > > > > > > > >> >> > > > > > request
> >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in
> >>> the
> >>> > > wiki
> >>> > > > > > > > sometimes
> >>> > > > > > > > > >> refers
> >>> > > > > > > > > >> >> > to
> >>> > > > > > > > > >> >> > > > the
> >>> > > > > > > > > >> >> > > > > > end
> >>> > > > > > > > > >> >> > > > > > > > > client and some other times
> refers
> >>> to
> >>> > the
> >>> > > > > > client
> >>> > > > > > > > > using
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
> >>> > distinguish
> >>> > > > > > between
> >>> > > > > > > > the
> >>> > > > > > > > > >> two.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the
> sentence
> >>> > > "Broker
> >>> > > > > > also
> >>> > > > > > > > > >> stores the
> >>> > > > > > > > > >> >> > > > > > > > DelegationToken
> >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the
> zookeeper."
> >>> a
> >>> > bit
> >>> > > > > > more? I
> >>> > > > > > > > > >> thought the
> >>> > > > > > > > > >> >> > > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM,
> Jun
> >>> > Rao
> >>> > > <
> >>> > > > > > > > > >> jun@confluent.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
> >>> invite.
> >>> > We
> >>> > > > can
> >>> > > > > > > > discuss
> >>> > > > > > > > > >> this in
> >>> > > > > > > > > >> >> > > the
> >>> > > > > > > > > >> >> > > > > > > meeting
> >>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > Jun
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47
> AM,
> >>> > > Harsha <
> >>> > > > > > > > > >> kafka@harsha.io>
> >>> > > > > > > > > >> >> > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
> >>> meeting
> >>> > > > > around
> >>> > > > > > > > this.
> >>> > > > > > > > > >> The KIP
> >>> > > > > > > > > >> >> > is
> >>> > > > > > > > > >> >> > > > up
> >>> > > > > > > > > >> >> > > > > > for
> >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if
> there
> >>> are
> >>> > > any
> >>> > > > > > > > questions
> >>> > > > > > > > > >> lets
> >>> > > > > > > > > >> >> > > > quickly
> >>> > > > > > > > > >> >> > > > > > hash
> >>> > > > > > > > > >> >> > > > > > > > out
> >>> > > > > > > > > >> >> > > > > > > > > >> details.
> >>> > > > > > > > > >> >> > > > > > > > > >>
> >>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >> Harsha
> >>> > > > > > > > > >> >> > > > > > > > > >>
> >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40
> >>> AM,
> >>> > > parth
> >>> > > > > > > > > brahmbhatt
> >>> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
> >>> > system
> >>> > > > uses
> >>> > > > > > so
> >>> > > > > > > no
> >>> > > > > > > > > >> good
> >>> > > > > > > > > >> >> > reason
> >>> > > > > > > > > >> >> > > > > > really.
> >>> > > > > > > > > >> >> > > > > > > > We
> >>> > > > > > > > > >> >> > > > > > > > > >> > could
> >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
> >>> > newest
> >>> > > > > > > > recommended
> >>> > > > > > > > > >> standard
> >>> > > > > > > > > >> >> > > is.
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> >>> > > > > > > > > >> >> > > > > > > > > >> > Parth
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33
> >>> AM,
> >>> > > > Ismael
> >>> > > > > > > Juma <
> >>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> >>> > > > > > > > > >> >> > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
> >>> > started
> >>> > > > > > > reviewing
> >>> > > > > > > > > >> this and
> >>> > > > > > > > > >> >> > > may
> >>> > > > > > > > > >> >> > > > > have
> >>> > > > > > > > > >> >> > > > > > > > > >> additional
> >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
> >>> immediate
> >>> > > > > question
> >>> > > > > > > that
> >>> > > > > > > > > >> came to
> >>> > > > > > > > > >> >> > > mind
> >>> > > > > > > > > >> >> > > > is
> >>> > > > > > > > > >> >> > > > > > our
> >>> > > > > > > > > >> >> > > > > > > > > >> choice of
> >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though
> it's
> >>> > > marked
> >>> > > > > as
> >>> > > > > > > > > >> OBSOLETE in
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > IANA
> >>> > > > > > > > > >> >> > > > > > > > > Registry
> >>> > > > > > > > > >> >> > > > > > > > > >> of
> >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
> >>> original
> >>> > > RFC
> >>> > > > > > > (2831)
> >>> > > > > > > > > has
> >>> > > > > > > > > >> been
> >>> > > > > > > > > >> >> > > moved
> >>> > > > > > > > > >> >> > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > Historic
> >>> > > > > > > > > >> >> > > > > > > > > >> > > status:
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> https://tools.ietf.org/html/
> >>> > > rfc6331
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > >
> >>> > > > > > > > > >> >> > >
> >>> > > > > > > > > >>
> >>> > > > > > >
> >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
> >>> mechanisms.xhtml
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning
> behind
> >>> > that
> >>> > > > > > choice?
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at
> 11:29
> >>> > PM,
> >>> > > > Gwen
> >>> > > > > > > > > Shapira <
> >>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> >>> > > > > > > > > >> >> > > > > > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> wrote:
> >>> > > > > > > > > >> >> > > > > > > > > >> > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize
> >>> that
> >>> > > even
> >>> > > > > > though
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > tokens
> >>> > > > > > > > > >> >> > > > > > > are a
> >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel
> very
> >>> > > strongly
> >>> > > > > > about
> >>> > > > > > > > not
> >>> > > > > > > > > >> adding
> >>> > > > > > > > > >> >> > > > > > dependency
> >>> > > > > > > > > >> >> > > > > > > > on
> >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
> >>> delegation
> >>> > > > > tokens
> >>> > > > > > > for
> >>> > > > > > > > > >> Kafka. The
> >>> > > > > > > > > >> >> > > KIP
> >>> > > > > > > > > >> >> > > > > > > doesn't
> >>> > > > > > > > > >> >> > > > > > > > > >> imply
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but
> if
> >>> you
> >>> > > can
> >>> > > > > > > > clarify...
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop
> dependency.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to
> the
> >>> KIP
> >>> > so
> >>> > > > no
> >>> > > > > > one
> >>> > > > > > > > will
> >>> > > > > > > > > >> read
> >>> > > > > > > > > >> >> > the
> >>> > > > > > > > > >> >> > > > KIP
> >>> > > > > > > > > >> >> > > > > > and
> >>> > > > > > > > > >> >> > > > > > > > > panic
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the
> next
> >>> > > > > release...
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get
> delegation
> >>> > token
> >>> > > at
> >>> > > > > any
> >>> > > > > > > > time
> >>> > > > > > > > > >> after
> >>> > > > > > > > > >> >> > > > > > > > authenticating?
> >>> > > > > > > > > >> >> > > > > > > > > >> only
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> >>> > > > authenticated
> >>> > > > > > you
> >>> > > > > > > > can
> >>> > > > > > > > > >> get
> >>> > > > > > > > > >> >> > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > tokens.
> >>> > > > > > > > > >> >> > > > > > > > > >> We
> >>> > > > > > > > > >> >> > > > > > > > > >> > > need
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> >>> > > > authenticated
> >>> > > > > > > using
> >>> > > > > > > > > >> delegation
> >>> > > > > > > > > >> >> > > > > token,
> >>> > > > > > > > > >> >> > > > > > > can
> >>> > > > > > > > > >> >> > > > > > > > > also
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token
> again or
> >>> > not.
> >>> > > > > Also
> >>> > > > > > > > there
> >>> > > > > > > > > >> is the
> >>> > > > > > > > > >> >> > > > > question
> >>> > > > > > > > > >> >> > > > > > of
> >>> > > > > > > > > >> >> > > > > > > > do
> >>> > > > > > > > > >> >> > > > > > > > > we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > allow
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
> >>> delegation
> >>> > > > token
> >>> > > > > > or
> >>> > > > > > > we
> >>> > > > > > > > > >> want
> >>> > > > > > > > > >> >> > > specific
> >>> > > > > > > > > >> >> > > > > > ACLs
> >>> > > > > > > > > >> >> > > > > > > (I
> >>> > > > > > > > > >> >> > > > > > > > > >> think
> >>> > > > > > > > > >> >> > > > > > > > > >> > > its
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > an
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
> >>> > > overkill.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating
> two
> >>> > > > options:
> >>> > > > > > > > Either
> >>> > > > > > > > > >> require
> >>> > > > > > > > > >> >> > > > > Kerberos
> >>> > > > > > > > > >> >> > > > > > > > auth
> >>> > > > > > > > > >> >> > > > > > > > > >> for
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require
> >>> non-owners
> >>> > to
> >>> > > > > > renew.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
> >>> > simpler
> >>> > > > (it
> >>> > > > > > > > > basically
> >>> > > > > > > > > >> >> > require
> >>> > > > > > > > > >> >> > > a
> >>> > > > > > > > > >> >> > > > > "job
> >>> > > > > > > > > >> >> > > > > > > > > master"
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility
> for
> >>> the
> >>> > > > > > renewal,
> >>> > > > > > > it
> >>> > > > > > > > > >> will have
> >>> > > > > > > > > >> >> > > its
> >>> > > > > > > > > >> >> > > > > own
> >>> > > > > > > > > >> >> > > > > > > > > >> identity
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this
> is
> >>> the
> >>> > > > > correct
> >>> > > > > > > > > design
> >>> > > > > > > > > >> >> > pattern
> >>> > > > > > > > > >> >> > > > > > anyway.
> >>> > > > > > > > > >> >> > > > > > > > For
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect
> Nimbus to
> >>> > > > > coordinate
> >>> > > > > > > > > >> renewals?),
> >>> > > > > > > > > >> >> > but
> >>> > > > > > > > > >> >> > > > it
> >>> > > > > > > > > >> >> > > > > is
> >>> > > > > > > > > >> >> > > > > > > > hard
> >>> > > > > > > > > >> >> > > > > > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity
> without
> >>> > > looking
> >>> > > > at
> >>> > > > > > the
> >>> > > > > > > > > code
> >>> > > > > > > > > >> >> > changes
> >>> > > > > > > > > >> >> > > > > > > required.
> >>> > > > > > > > > >> >> > > > > > > > If
> >>> > > > > > > > > >> >> > > > > > > > > >> you
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
> >>> > "require
> >>> > > > > > > Kerberos"
> >>> > > > > > > > > >> will look
> >>> > > > > > > > > >> >> > > in
> >>> > > > > > > > > >> >> > > > > > Kafka
> >>> > > > > > > > > >> >> > > > > > > > > code,
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a
> >>> look.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is
> >>> that
> >>> > > > tokens
> >>> > > > > > will
> >>> > > > > > > > > >> propagate
> >>> > > > > > > > > >> >> > via
> >>> > > > > > > > > >> >> > > > ZK
> >>> > > > > > > > > >> >> > > > > > but
> >>> > > > > > > > > >> >> > > > > > > > > >> without
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> >>> > > > > UpdateMetadata
> >>> > > > > > > > > >> protocol,
> >>> > > > > > > > > >> >> > > > correct?
> >>> > > > > > > > > >> >> > > > > > > > Clients
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry
> on
> >>> > SASL
> >>> > > > auth
> >>> > > > > > > > failure
> >>> > > > > > > > > >> (IIRC),
> >>> > > > > > > > > >> >> > > but
> >>> > > > > > > > > >> >> > > > > > since
> >>> > > > > > > > > >> >> > > > > > > > the
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate
> between
> >>> > > brokers
> >>> > > > > > > asynch,
> >>> > > > > > > > > we
> >>> > > > > > > > > >> will
> >>> > > > > > > > > >> >> > > need
> >>> > > > > > > > > >> >> > > > to
> >>> > > > > > > > > >> >> > > > > > > > retry a
> >>> > > > > > > > > >> >> > > > > > > > > >> bit
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients
> failing
> >>> > auth
> >>> > > > due
> >>> > > > > to
> >>> > > > > > > > > timing
> >>> > > > > > > > > >> >> > issues.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
> >>> > > alternatives
> >>> > > > > > right
> >>> > > > > > > > > now.
> >>> > > > > > > > > >> The
> >>> > > > > > > > > >> >> > > > current
> >>> > > > > > > > > >> >> > > > > > > > > documented
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based
> and it
> >>> > does
> >>> > > > not
> >>> > > > > > > > require
> >>> > > > > > > > > >> any
> >>> > > > > > > > > >> >> > > changes
> >>> > > > > > > > > >> >> > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An
> alternative
> >>> > > > approach
> >>> > > > > > can
> >>> > > > > > > > > remove
> >>> > > > > > > > > >> >> > > zookeeper
> >>> > > > > > > > > >> >> > > > > > > > > dependency
> >>> > > > > > > > > >> >> > > > > > > > > >> as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > well
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss
> that
> >>> in
> >>> > KIP
> >>> > > > > > > > discussion
> >>> > > > > > > > > >> call.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds
> interesting.
> >>> Do
> >>> > you
> >>> > > > > want
> >>> > > > > > to
> >>> > > > > > > > > ping
> >>> > > > > > > > > >> Jun to
> >>> > > > > > > > > >> >> > > > > > arrange a
> >>> > > > > > > > > >> >> > > > > > > > > call?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
> >>> > suggestion
> >>> > > of
> >>> > > > > > > having
> >>> > > > > > > > > >> just the
> >>> > > > > > > > > >> >> > > > > > controller
> >>> > > > > > > > > >> >> > > > > > > > > issue
> >>> > > > > > > > > >> >> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to
> >>> avoid
> >>> > > > > syncing
> >>> > > > > > a
> >>> > > > > > > > > shared
> >>> > > > > > > > > >> >> > secret.
> >>> > > > > > > > > >> >> > > > Not
> >>> > > > > > > > > >> >> > > > > > > sure
> >>> > > > > > > > > >> >> > > > > > > > if
> >>> > > > > > > > > >> >> > > > > > > > > >> we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
> >>> > > discussion
> >>> > > > > here
> >>> > > > > > > or
> >>> > > > > > > > on
> >>> > > > > > > > > >> the
> >>> > > > > > > > > >> >> > > wiki. I
> >>> > > > > > > > > >> >> > > > > > think
> >>> > > > > > > > > >> >> > > > > > > > > that
> >>> > > > > > > > > >> >> > > > > > > > > >> we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the
> problem
> >>> of
> >>> > > > "token
> >>> > > > > > > > > >> distribution"
> >>> > > > > > > > > >> >> > > from
> >>> > > > > > > > > >> >> > > > > > > "shared
> >>> > > > > > > > > >> >> > > > > > > > > >> secret
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use
> the
> >>> > > > > controller
> >>> > > > > > > as
> >>> > > > > > > > > the
> >>> > > > > > > > > >> only
> >>> > > > > > > > > >> >> > > token
> >>> > > > > > > > > >> >> > > > > > > > generator
> >>> > > > > > > > > >> >> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second
> issue,
> >>> > while
> >>> > > > > still
> >>> > > > > > > > using
> >>> > > > > > > > > >> ZK async
> >>> > > > > > > > > >> >> > > to
> >>> > > > > > > > > >> >> > > > > > > > distribute
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
> >>> > previous
> >>> > > > > Email
> >>> > > > > > I
> >>> > > > > > > am
> >>> > > > > > > > > >> fine with
> >>> > > > > > > > > >> >> > > > that
> >>> > > > > > > > > >> >> > > > > > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > long
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > as
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the
> extra
> >>> > > > complexity
> >>> > > > > > of
> >>> > > > > > > > > >> >> > > adding/updating
> >>> > > > > > > > > >> >> > > > > APIS
> >>> > > > > > > > > >> >> > > > > > > > adds
> >>> > > > > > > > > >> >> > > > > > > > > >> enough
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage
> with
> >>> > the
> >>> > > > > > > controller
> >>> > > > > > > > > >> approach
> >>> > > > > > > > > >> >> > is
> >>> > > > > > > > > >> >> > > > > > secret
> >>> > > > > > > > > >> >> > > > > > > > > >> rotation
> >>> > > > > > > > > >> >> > > > > > > > > >> > > can
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > be
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and
> >>> would
> >>> > > not
> >>> > > > > > > require
> >>> > > > > > > > > >> >> > > deployment. *
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> >>> > > > complexity
> >>> > > > > > (or
> >>> > > > > > > > > point
> >>> > > > > > > > > >> me to
> >>> > > > > > > > > >> >> > > the
> >>> > > > > > > > > >> >> > > > > > email
> >>> > > > > > > > > >> >> > > > > > > I
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis
> are
> >>> > > > required?
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell,
> >>> clients
> >>> > > can
> >>> > > > > > > already
> >>> > > > > > > > > >> find the
> >>> > > > > > > > > >> >> > > > > > controller
> >>> > > > > > > > > >> >> > > > > > > > from
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
> >>> > > concerned
> >>> > > > > > about
> >>> > > > > > > > > >> controller
> >>> > > > > > > > > >> >> > > > load.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the
> idea
> >>> of
> >>> > > > forcing
> >>> > > > > > > > > kerberos
> >>> > > > > > > > > >> auth
> >>> > > > > > > > > >> >> > for
> >>> > > > > > > > > >> >> > > > > > > renwal, I
> >>> > > > > > > > > >> >> > > > > > > > > >> think
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport
> >>> layer
> >>> > > the
> >>> > > > > the
> >>> > > > > > > > > request
> >>> > > > > > > > > >> >> > content
> >>> > > > > > > > > >> >> > > > in
> >>> > > > > > > > > >> >> > > > > a
> >>> > > > > > > > > >> >> > > > > > > > pretty
> >>> > > > > > > > > >> >> > > > > > > > > >> ugly
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
> >>> > renewer
> >>> > > to
> >>> > > > > > > > non-owner
> >>> > > > > > > > > >> is
> >>> > > > > > > > > >> >> > > better.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a
> >>> necessary
> >>> > > > evil.
> >>> > > > > > > While
> >>> > > > > > > > > >> this will
> >>> > > > > > > > > >> >> > > make
> >>> > > > > > > > > >> >> > > > > the
> >>> > > > > > > > > >> >> > > > > > > > kafka
> >>> > > > > > > > > >> >> > > > > > > > > >> code
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight
> forward ,
> >>> > > > forcing
> >>> > > > > > > > renewer
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > non-owner
> >>> > > > > > > > > >> >> > > > > > > pushes
> >>> > > > > > > > > >> >> > > > > > > > > >> the code
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and
> >>> makes
> >>> > > it
> >>> > > > > even
> >>> > > > > > > > > harder
> >>> > > > > > > > > >> to
> >>> > > > > > > > > >> >> > > > > > integrate. *
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I
> >>> don't
> >>> > > think
> >>> > > > > the
> >>> > > > > > > > > >> "renewal by
> >>> > > > > > > > > >> >> > > > other"
> >>> > > > > > > > > >> >> > > > > > > > approach
> >>> > > > > > > > > >> >> > > > > > > > > >> is
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the
> clients
> >>> we
> >>> > > > expect
> >>> > > > > to
> >>> > > > > > > use
> >>> > > > > > > > > >> >> > delegation
> >>> > > > > > > > > >> >> > > > > tokens
> >>> > > > > > > > > >> >> > > > > > > > since
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an
> >>> app-master
> >>> > of
> >>> > > > some
> >>> > > > > > > sort
> >>> > > > > > > > > who
> >>> > > > > > > > > >> >> > > requested
> >>> > > > > > > > > >> >> > > > > the
> >>> > > > > > > > > >> >> > > > > > > > token
> >>> > > > > > > > > >> >> > > > > > > > > to
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my
> >>> question
> >>> > > on
> >>> > > > > how
> >>> > > > > > > > > multiple
> >>> > > > > > > > > >> >> > > > identities
> >>> > > > > > > > > >> >> > > > > > will
> >>> > > > > > > > > >> >> > > > > > > > be
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super
> >>> clear
> >>> > to
> >>> > > > me -
> >>> > > > > > > > AFAIK,
> >>> > > > > > > > > >> we don't
> >>> > > > > > > > > >> >> > > > > > > > authenticate
> >>> > > > > > > > > >> >> > > > > > > > > >> each
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we
> authenticate
> >>> > > > > > connections.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
> >>> > connections,
> >>> > > > and
> >>> > > > > > only
> >>> > > > > > > > > when
> >>> > > > > > > > > >> they
> >>> > > > > > > > > >> >> > are
> >>> > > > > > > > > >> >> > > > > being
> >>> > > > > > > > > >> >> > > > > > > > > >> established.
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Let
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this
> as
> >>> a
> >>> > > > > question,
> >>> > > > > > > in
> >>> > > > > > > > > >> absence of
> >>> > > > > > > > > >> >> > > > > > > delegation
> >>> > > > > > > > > >> >> > > > > > > > > >> tokens if
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > we
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use
> >>> case
> >>> > > > using
> >>> > > > > > user
> >>> > > > > > > > > >> TGT's how
> >>> > > > > > > > > >> >> > > would
> >>> > > > > > > > > >> >> > > > > we
> >>> > > > > > > > > >> >> > > > > > do
> >>> > > > > > > > > >> >> > > > > > > > it?
> >>> > > > > > > > > >> >> > > > > > > > > >> My
> >>> > > > > > > > > >> >> > > > > > > > > >> > > point
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no
> >>> different
> >>> > > > with
> >>> > > > > > > > > delegation
> >>> > > > > > > > > >> >> > tokens.
> >>> > > > > > > > > >> >> > > > The
> >>> > > > > > > > > >> >> > > > > > use
> >>> > > > > > > > > >> >> > > > > > > > > case
> >>> > > > > > > > > >> >> > > > > > > > > >> you
> >>> > > > > > > > > >> >> > > > > > > > > >> > > are
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more
> >>> like
> >>> > > > > > > > impersonation.*
> >>> > > > > > > > > >> >> > > > > > > > > >> > > >
> >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that
> one of
> >>> > the
> >>> > > > > things
> >>> > > > > > > > that
> >>> > > > > > > > > >> >> > del
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Regards,
> >> Ashish
> >>
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
--
Regards,
Ashish
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Gwen Shapira <gw...@confluent.io>.
Hi Guys,
This discussion was dead for a while. Are there still contentious
points? If not, why are there no votes?
On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <ju...@confluent.io> wrote:
> Ashish,
>
> Yes, I will send out a KIP invite for next week to discuss KIP-48 and other
> remaining KIPs.
>
> Thanks,
>
> Jun
>
> On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com> wrote:
>
>> Thanks Harsha!
>>
>> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
>> actually make a call on when we should have next KIP call. As there are a
>> few outstanding KIPs that could not be discussed this week, can we have a
>> KIP hangout call next week?
>>
>> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io>
>> wrote:
>>
>>> Ashish,
>>> Yes we are working on it. Lets discuss in the next KIP meeting.
>>> I'll join.
>>> -Harsha
>>>
>>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
>>> wrote:
>>>
>>> > Hello Harsha,
>>> >
>>> > Are you still working on this? Wondering if we can discuss this in next
>>> KIP
>>> > meeting, if you can join.
>>> >
>>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io>
>>> > wrote:
>>> >
>>> > > Hi Grant,
>>> > > We are working on it. Will add the details to KIP about the
>>> > > request protocol.
>>> > >
>>> > > Thanks,
>>> > > Harsha
>>> > >
>>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com>
>>> wrote:
>>> > >
>>> > > > Hi Parth,
>>> > > >
>>> > > > Are you still working on this? If you need any help please don't
>>> > hesitate
>>> > > > to ask.
>>> > > >
>>> > > > Thanks,
>>> > > > Grant
>>> > > >
>>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
>>> > > >
>>> > > > > Parth,
>>> > > > >
>>> > > > > Thanks for the reply.
>>> > > > >
>>> > > > > It makes sense to only allow the renewal by users that
>>> authenticated
>>> > > > using
>>> > > > > *non* delegation token mechanism. Then, should we make the
>>> renewal a
>>> > > > list?
>>> > > > > For example, in the case of rest proxy, it will be useful for
>>> every
>>> > > > > instance of rest proxy to be able to renew the tokens.
>>> > > > >
>>> > > > > It would be clearer if we can document the request protocol like
>>> > > > >
>>> > > > >
>>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
>>> > > Commandlineandcentralizedadministrativeoperations-
>>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
>>> > > > > .
>>> > > > >
>>> > > > > It would also be useful to document the client APIs.
>>> > > > >
>>> > > > > Thanks,
>>> > > > >
>>> > > > > Jun
>>> > > > >
>>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
>>> > > > > brahmbhatt.parth@gmail.com> wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > I am suggesting that we will only allow the renewal by users
>>> that
>>> > > > > > authenticated using *non* delegation token mechanism. For
>>> example,
>>> > If
>>> > > > > user
>>> > > > > > Alice authenticated using kerberos and requested delegation
>>> tokens,
>>> > > > only
>>> > > > > > user Alice authenticated via non delegation token mechanism can
>>> > > renew.
>>> > > > > > Clients that have access to delegation tokens can not issue
>>> > renewal
>>> > > > > > request for renewing their own token and this is primarily
>>> > important
>>> > > to
>>> > > > > > reduce the time window for which a compromised token will be
>>> valid.
>>> > > > > >
>>> > > > > > To clarify, Yes any authenticated user can request delegation
>>> > tokens
>>> > > > but
>>> > > > > > even here I would recommend to avoid creating a chain where a
>>> > client
>>> > > > > > authenticated via delegation token request for more delegation
>>> > > tokens.
>>> > > > > > Basically anyone can request delegation token, as long as they
>>> > > > > authenticate
>>> > > > > > via a non delegation token mechanism.
>>> > > > > >
>>> > > > > > Aren't classes listed here
>>> > > > > > <
>>> > > > > >
>>> > > > >
>>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
>>> upportforKaf
>>> > > ka-PublicInterfaces
>>> > > > > > >
>>> > > > > > sufficient?
>>> > > > > >
>>> > > > > > Thanks
>>> > > > > > Parth
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io>
>>> wrote:
>>> > > > > >
>>> > > > > > > Parth,
>>> > > > > > >
>>> > > > > > > Thanks for the reply. A couple of comments inline below.
>>> > > > > > >
>>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
>>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > 1. Who / how are tokens renewed? By original requester
>>> only? or
>>> > > > using
>>> > > > > > > > Kerberos
>>> > > > > > > > auth only?
>>> > > > > > > > My recommendation is to do this only using Kerberos auth and
>>> > only
>>> > > > > threw
>>> > > > > > > the
>>> > > > > > > > renewer specified during the acquisition request.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > Hmm, not sure that I follow this. Are you saying that any
>>> client
>>> > > > > > > authenticated with the delegation token can renew, i.e. there
>>> is
>>> > no
>>> > > > > > renewer
>>> > > > > > > needed?
>>> > > > > > >
>>> > > > > > > Also, just to be clear, any authenticated client (either
>>> through
>>> > > SASL
>>> > > > > or
>>> > > > > > > SSL) can request a delegation token for the authenticated
>>> user,
>>> > > > right?
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
>>> > > > > > > > My recommendation is still to store in ZK or not store them
>>> at
>>> > > all.
>>> > > > > The
>>> > > > > > > > whole controller based distribution is too much overhead
>>> with
>>> > not
>>> > > > > much
>>> > > > > > to
>>> > > > > > > > achieve.
>>> > > > > > > >
>>> > > > > > > > 3. How are tokens invalidated / expired?
>>> > > > > > > > Either by expiration time out or through an explicit
>>> request to
>>> > > > > > > invalidate.
>>> > > > > > > >
>>> > > > > > > > 4. Which encryption algorithm is used?
>>> > > > > > > > SCRAM
>>> > > > > > > >
>>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP
>>> but
>>> > > was
>>> > > > > > > > discussed
>>> > > > > > > > in this thread)?
>>> > > > > > > > There is no imperonation proposal. I tried and explained how
>>> > its
>>> > > a
>>> > > > > > > > different problem and why its not really necessary to
>>> discuss
>>> > > that
>>> > > > as
>>> > > > > > > part
>>> > > > > > > > of this KIP. This KIP will not support any impersonation,
>>> it
>>> > > will
>>> > > > > just
>>> > > > > > > be
>>> > > > > > > > another way to authenticate.
>>> > > > > > > >
>>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
>>> > > > > > > > We do not need new ACLs.
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > Could we document the format of the new request/response and
>>> > their
>>> > > > > > > associated Resource and Operation for ACL?
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > > 7. How would the delegation token be configured in the
>>> client?
>>> > > > > > > > Should be through config. I wasn't planning on supporting
>>> JAAS
>>> > > for
>>> > > > > > > tokens.
>>> > > > > > > > I don't believe hadoop does this either.
>>> > > > > > > >
>>> > > > > > > > Thanks
>>> > > > > > > > Parth
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io>
>>> > > wrote:
>>> > > > > > > >
>>> > > > > > > > > Harsha,
>>> > > > > > > > >
>>> > > > > > > > > Another question.
>>> > > > > > > > >
>>> > > > > > > > > 9. How would the delegation token be configured in the
>>> > client?
>>> > > > The
>>> > > > > > > > standard
>>> > > > > > > > > way is to do this through JAAS. However, we will need to
>>> > think
>>> > > > > > through
>>> > > > > > > if
>>> > > > > > > > > this is convenient in a shared environment. For example,
>>> > when a
>>> > > > new
>>> > > > > > > task
>>> > > > > > > > is
>>> > > > > > > > > added to a Storm worker node, do we need to dynamically
>>> add a
>>> > > new
>>> > > > > > > section
>>> > > > > > > > > in the JAAS file? It may be more convenient if we can
>>> pass in
>>> > > the
>>> > > > > > token
>>> > > > > > > > > through the config directly w/o going through JAAS.
>>> > > > > > > > >
>>> > > > > > > > > Are you or Parth still actively working on this KIP?
>>> > > > > > > > >
>>> > > > > > > > > Thanks,
>>> > > > > > > > >
>>> > > > > > > > > Jun
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
>>> jun@confluent.io>
>>> > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > > Just to add on that list.
>>> > > > > > > > > >
>>> > > > > > > > > > 2. It would be good to document the format of the data
>>> > stored
>>> > > > in
>>> > > > > > ZK.
>>> > > > > > > > > > 7. Earlier, there was a discussion on whether the tokens
>>> > > should
>>> > > > > be
>>> > > > > > > > > > propagated through ZK like config/acl/quota, or through
>>> the
>>> > > > > > > controller.
>>> > > > > > > > > > Currently, the controller is only designed for
>>> propagating
>>> > > > topic
>>> > > > > > > > > metadata,
>>> > > > > > > > > > but not other data.
>>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
>>> > > DIGEST-MD5
>>> > > > > > since
>>> > > > > > > > it's
>>> > > > > > > > > > deprecated?
>>> > > > > > > > > >
>>> > > > > > > > > > Also, the images in the wiki seem broken.
>>> > > > > > > > > >
>>> > > > > > > > > > Thanks,
>>> > > > > > > > > >
>>> > > > > > > > > > Jun
>>> > > > > > > > > >
>>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
>>> > > > > gwen@confluent.io>
>>> > > > > > > > > wrote:
>>> > > > > > > > > >
>>> > > > > > > > > >> From what I can see, remaining questions are:
>>> > > > > > > > > >>
>>> > > > > > > > > >> 1. Who / how are tokens renewed? By original requester
>>> > only?
>>> > > > or
>>> > > > > > > using
>>> > > > > > > > > >> Kerberos auth only?
>>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
>>> > > > > > > > > >> 3. How are tokens invalidated / expired?
>>> > > > > > > > > >> 4. Which encryption algorithm is used?
>>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the
>>> > KIP
>>> > > > but
>>> > > > > > was
>>> > > > > > > > > >> discussed in this thread)?
>>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
>>> > > > > > > > > >>
>>> > > > > > > > > >> Gwen
>>> > > > > > > > > >>
>>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
>>> kafka@harsha.io>
>>> > > > wrote:
>>> > > > > > > > > >> > Jun & Ismael,
>>> > > > > > > > > >> > Unfortunately I couldn't
>>> attend
>>> > > the
>>> > > > > KIP
>>> > > > > > > > > meeting
>>> > > > > > > > > >> > when delegation tokens
>>> > discussed.
>>> > > > > > > > Appreciate
>>> > > > > > > > > if
>>> > > > > > > > > >> > you can update the thread if
>>> > you
>>> > > > have
>>> > > > > > any
>>> > > > > > > > > >> > further questions.
>>> > > > > > > > > >> > Thanks,
>>> > > > > > > > > >> > Harsha
>>> > > > > > > > > >> >
>>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
>>> > > > > > > > > >> >> It seems that the links to images in the KIP are
>>> > broken.
>>> > > > > > > > > >> >>
>>> > > > > > > > > >> >> Liquan
>>> > > > > > > > > >> >>
>>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
>>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
>>> > > > > > > > > >> >>
>>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
>>> > > > > > > > > >> >> > In the current proposal we only allow a user to
>>> get
>>> > > > > > delegation
>>> > > > > > > > > token
>>> > > > > > > > > >> for
>>> > > > > > > > > >> >> > the identity that it authenticated as using
>>> another
>>> > > > > > mechanism,
>>> > > > > > > > i.e.
>>> > > > > > > > > >> A user
>>> > > > > > > > > >> >> > that authenticate using a keytab for principal
>>> > > > > > > user1@EXAMPLE.COM
>>> > > > > > > > > >> will get
>>> > > > > > > > > >> >> > delegation tokens for that user only. In future I
>>> > think
>>> > > > we
>>> > > > > > will
>>> > > > > > > > > have
>>> > > > > > > > > >> to
>>> > > > > > > > > >> >> > extend support such that we allow some set of
>>> users (
>>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
>>> > storm-nimbus@EXAMPLE.COM)
>>> > > > to
>>> > > > > > > > acquire
>>> > > > > > > > > >> >> > delegation tokens on behalf of other users whose
>>> > > identity
>>> > > > > > they
>>> > > > > > > > have
>>> > > > > > > > > >> >> > verified independently. Kafka brokers will have
>>> ACLs
>>> > > to
>>> > > > > > > control
>>> > > > > > > > > >> which
>>> > > > > > > > > >> >> > users are allowed to impersonate other users and
>>> get
>>> > > > tokens
>>> > > > > > on
>>> > > > > > > > > >> behalf of
>>> > > > > > > > > >> >> > them. Overall Impersonation is a whole different
>>> > > problem
>>> > > > in
>>> > > > > > my
>>> > > > > > > > > >> opinion and
>>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
>>> renewing
>>> > > > > > delegation
>>> > > > > > > > > >> tokens?
>>> > > > > > > > > >> >> > Typically this should be very very low, 1 request
>>> per
>>> > > > > minute
>>> > > > > > > is a
>>> > > > > > > > > >> >> > relatively high estimate. However it depends on
>>> the
>>> > > token
>>> > > > > > > > > >> expiration. I am
>>> > > > > > > > > >> >> > less worried about the extra load it puts on
>>> > controller
>>> > > > vs
>>> > > > > > the
>>> > > > > > > > > added
>>> > > > > > > > > >> >> > complexity and the value it offers.
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> > Thanks
>>> > > > > > > > > >> >> > Parth
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
>>> > > > > > > ismael@juma.me.uk>
>>> > > > > > > > > >> wrote:
>>> > > > > > > > > >> >> >
>>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
>>> separate
>>> > > KIP
>>> > > > > as
>>> > > > > > it
>>> > > > > > > > > will
>>> > > > > > > > > >> >> > > introduce user visible changes. We could also
>>> > update
>>> > > > > KIP-48
>>> > > > > > > to
>>> > > > > > > > > >> have this
>>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
>>> > > separately.
>>> > > > We
>>> > > > > > can
>>> > > > > > > > > >> discuss
>>> > > > > > > > > >> >> > that
>>> > > > > > > > > >> >> > > in the KIP call today.
>>> > > > > > > > > >> >> > >
>>> > > > > > > > > >> >> > > Ismael
>>> > > > > > > > > >> >> > >
>>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram
>>> <
>>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
>>> > > > > > > > > >> >> > >
>>> > > > > > > > > >> >> > > > Ismael,
>>> > > > > > > > > >> >> > > >
>>> > > > > > > > > >> >> > > > I have created a JIRA (
>>> > > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
>>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would
>>> that
>>> > > need
>>> > > > > > > another
>>> > > > > > > > > >> KIP? If
>>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just
>>> be
>>> > a
>>> > > > JIRA
>>> > > > > > > that
>>> > > > > > > > > gets
>>> > > > > > > > > >> >> > > reviewed
>>> > > > > > > > > >> >> > > > when the PR is ready?
>>> > > > > > > > > >> >> > > >
>>> > > > > > > > > >> >> > > > Thank you,
>>> > > > > > > > > >> >> > > >
>>> > > > > > > > > >> >> > > > Rajini
>>> > > > > > > > > >> >> > > >
>>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
>>> > > > > > > > > ismael@juma.me.uk>
>>> > > > > > > > > >> >> > wrote:
>>> > > > > > > > > >> >> > > >
>>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
>>> > candidate.
>>> > > > > > > > > >> >> > > > >
>>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as a
>>> SASL
>>> > > > > > mechanism
>>> > > > > > > > > that
>>> > > > > > > > > >> might
>>> > > > > > > > > >> >> > be
>>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to
>>> > check
>>> > > > it
>>> > > > > in
>>> > > > > > > > more
>>> > > > > > > > > >> detail.
>>> > > > > > > > > >> >> > > Good
>>> > > > > > > > > >> >> > > > > to know that you are willing to contribute
>>> an
>>> > > > > > > > implementation.
>>> > > > > > > > > >> Maybe
>>> > > > > > > > > >> >> > we
>>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
>>> > > > > > > > > >> >> > > > >
>>> > > > > > > > > >> >> > > > > Ismael
>>> > > > > > > > > >> >> > > > >
>>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
>>> > Sivaram <
>>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
>>> > > > > > > > > >> >> > > > >
>>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
>>> > Authentication
>>> > > > > > > > Mechanism)
>>> > > > > > > > > >> is a
>>> > > > > > > > > >> >> > > better
>>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't
>>> come
>>> > > > with a
>>> > > > > > > > > built-in
>>> > > > > > > > > >> SCRAM
>>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be
>>> happy
>>> > > to
>>> > > > > add
>>> > > > > > > > > support
>>> > > > > > > > > >> in
>>> > > > > > > > > >> >> > Kafka
>>> > > > > > > > > >> >> > > > > since
>>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to support
>>> > > anyway.
>>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
>>> > describes
>>> > > > the
>>> > > > > > > > protocol
>>> > > > > > > > > >> for
>>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
>>> > > > > > > > > >> >> > > > > >
>>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
>>> > > > > > > > jun@confluent.io
>>> > > > > > > > > >
>>> > > > > > > > > >> wrote:
>>> > > > > > > > > >> >> > > > > >
>>> > > > > > > > > >> >> > > > > > > Parth,
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of
>>> > more
>>> > > > > > > questions.
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs
>>> mean?
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting
>>> and
>>> > > > > > renewing
>>> > > > > > > > > >> delegation
>>> > > > > > > > > >> >> > > > tokens?
>>> > > > > > > > > >> >> > > > > > > That may have an impact on whether they
>>> > > should
>>> > > > be
>>> > > > > > > > > directed
>>> > > > > > > > > >> to the
>>> > > > > > > > > >> >> > > > > > > controller.
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > Jun
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
>>> > > > > brahmbhatt <
>>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > > Hi Jun,
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add
>>> > acls
>>> > > > on
>>> > > > > > who
>>> > > > > > > > can
>>> > > > > > > > > >> request
>>> > > > > > > > > >> >> > > > > > delegation
>>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for
>>> that
>>> > > yet
>>> > > > > but
>>> > > > > > > > down
>>> > > > > > > > > >> the line
>>> > > > > > > > > >> >> > > > when
>>> > > > > > > > > >> >> > > > > we
>>> > > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs
>>> it
>>> > > will
>>> > > > > be
>>> > > > > > > > > >> necessary.
>>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
>>> > > > > > > used/distributed
>>> > > > > > > > > >> over
>>> > > > > > > > > >> >> > secure
>>> > > > > > > > > >> >> > > > > > > channels.
>>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end up
>>> > > choosing
>>> > > > > > > > > >> Invalidation will
>>> > > > > > > > > >> >> > > be
>>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
>>> > > controller.
>>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
>>> somewhere
>>> > > > that
>>> > > > > > > > > >> invalidation
>>> > > > > > > > > >> >> > will
>>> > > > > > > > > >> >> > > > > > directly
>>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is not
>>> the
>>> > > > > intent.
>>> > > > > > > > > >> Invalidation
>>> > > > > > > > > >> >> > > will
>>> > > > > > > > > >> >> > > > > > either
>>> > > > > > > > > >> >> > > > > > > > be request based or due to
>>> expiration. No
>>> > > > > direct
>>> > > > > > > > > >> zookeeper
>>> > > > > > > > > >> >> > > > > interaction
>>> > > > > > > > > >> >> > > > > > > from
>>> > > > > > > > > >> >> > > > > > > > any client.
>>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
>>> DelegationToken
>>> > > > > without
>>> > > > > > > the
>>> > > > > > > > > >> hmac in
>>> > > > > > > > > >> >> > the
>>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
>>> confusion.
>>> > > The
>>> > > > > sole
>>> > > > > > > > > >> purpose of
>>> > > > > > > > > >> >> > > > > zookeeper
>>> > > > > > > > > >> >> > > > > > in
>>> > > > > > > > > >> >> > > > > > > > this design is as distribution channel
>>> > for
>>> > > > > tokens
>>> > > > > > > > > >> between all
>>> > > > > > > > > >> >> > > > brokers
>>> > > > > > > > > >> >> > > > > > > and a
>>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that
>>> were
>>> > > > > > generated
>>> > > > > > > by
>>> > > > > > > > > >> making a
>>> > > > > > > > > >> >> > > > > request
>>> > > > > > > > > >> >> > > > > > > to a
>>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on this
>>> in
>>> > > > second
>>> > > > > > > > > >> paragraph). The
>>> > > > > > > > > >> >> > > > token
>>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
>>> renewer,
>>> > > > uuid
>>> > > > > ,
>>> > > > > > > > > >> expiration,
>>> > > > > > > > > >> >> > > hmac)
>>> > > > > > > > > >> >> > > > ,
>>> > > > > > > > > >> >> > > > > > one
>>> > > > > > > > > >> >> > > > > > > of
>>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac
>>> but
>>> > > hmac
>>> > > > it
>>> > > > > > > self
>>> > > > > > > > is
>>> > > > > > > > > >> >> > derivable
>>> > > > > > > > > >> >> > > > if
>>> > > > > > > > > >> >> > > > > > you
>>> > > > > > > > > >> >> > > > > > > > have all the other elements of the
>>> token
>>> > +
>>> > > > > secret
>>> > > > > > > key
>>> > > > > > > > > to
>>> > > > > > > > > >> >> > generate
>>> > > > > > > > > >> >> > > > > hmac.
>>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL
>>> > > support
>>> > > > we
>>> > > > > > do
>>> > > > > > > > not
>>> > > > > > > > > >> want the
>>> > > > > > > > > >> >> > > > > entire
>>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
>>> zookeeper
>>> > > as
>>> > > > > that
>>> > > > > > > > will
>>> > > > > > > > > >> be an
>>> > > > > > > > > >> >> > > > insecure
>>> > > > > > > > > >> >> > > > > > > wire
>>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all
>>> the
>>> > > other
>>> > > > > > > > elements
>>> > > > > > > > > >> of a
>>> > > > > > > > > >> >> > > > > delegation
>>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
>>> elements
>>> > and
>>> > > > > > because
>>> > > > > > > > > they
>>> > > > > > > > > >> also
>>> > > > > > > > > >> >> > > have
>>> > > > > > > > > >> >> > > > > > access
>>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to
>>> > generate
>>> > > > > hmac
>>> > > > > > on
>>> > > > > > > > > >> their end.
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is to
>>> > avoid
>>> > > > > > > zookeeper
>>> > > > > > > > > >> >> > > altogether. A
>>> > > > > > > > > >> >> > > > > > > Client
>>> > > > > > > > > >> >> > > > > > > > will call broker with required
>>> > information
>>> > > > > > (owner,
>>> > > > > > > > > >> renwer,
>>> > > > > > > > > >> >> > > > > expiration)
>>> > > > > > > > > >> >> > > > > > > and
>>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker
>>> > won't
>>> > > > > store
>>> > > > > > > this
>>> > > > > > > > > in
>>> > > > > > > > > >> >> > > zookeeper.
>>> > > > > > > > > >> >> > > > > > From
>>> > > > > > > > > >> >> > > > > > > > this point a client can contact any
>>> > broker
>>> > > > with
>>> > > > > > all
>>> > > > > > > > the
>>> > > > > > > > > >> >> > > delegation
>>> > > > > > > > > >> >> > > > > > token
>>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac,
>>> > > uuid)
>>> > > > > the
>>> > > > > > > > borker
>>> > > > > > > > > >> will
>>> > > > > > > > > >> >> > > > > regenerate
>>> > > > > > > > > >> >> > > > > > > the
>>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with
>>> hmac
>>> > > > > > presented
>>> > > > > > > by
>>> > > > > > > > > >> client ,
>>> > > > > > > > > >> >> > > > broker
>>> > > > > > > > > >> >> > > > > > > will
>>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate.
>>> Only
>>> > > > > problem
>>> > > > > > > with
>>> > > > > > > > > >> this
>>> > > > > > > > > >> >> > > approach
>>> > > > > > > > > >> >> > > > > is
>>> > > > > > > > > >> >> > > > > > if
>>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
>>> client
>>> > > can
>>> > > > > now
>>> > > > > > > > > generate
>>> > > > > > > > > >> >> > random
>>> > > > > > > > > >> >> > > > > tokens
>>> > > > > > > > > >> >> > > > > > > and
>>> > > > > > > > > >> >> > > > > > > > they will still be able to
>>> authenticate
>>> > as
>>> > > > any
>>> > > > > > user
>>> > > > > > > > > they
>>> > > > > > > > > >> like.
>>> > > > > > > > > >> >> > > with
>>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
>>> tokens
>>> > > > > acquired
>>> > > > > > > via
>>> > > > > > > > a
>>> > > > > > > > > >> broker
>>> > > > > > > > > >> >> > > > (using
>>> > > > > > > > > >> >> > > > > > some
>>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
>>> token)
>>> > > will
>>> > > > > be
>>> > > > > > > > > >> accepted. We
>>> > > > > > > > > >> >> > > need
>>> > > > > > > > > >> >> > > > to
>>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more
>>> sense
>>> > and
>>> > > > we
>>> > > > > > can
>>> > > > > > > go
>>> > > > > > > > > >> over it
>>> > > > > > > > > >> >> > in
>>> > > > > > > > > >> >> > > > > > > tomorrow's
>>> > > > > > > > > >> >> > > > > > > > meeting.
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to
>>> me?
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > Thanks
>>> > > > > > > > > >> >> > > > > > > > Parth
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun
>>> > Rao <
>>> > > > > > > > > >> jun@confluent.io>
>>> > > > > > > > > >> >> > > > wrote:
>>> > > > > > > > > >> >> > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful
>>> for
>>> > > > Kafka
>>> > > > > > > > Connect
>>> > > > > > > > > >> and
>>> > > > > > > > > >> >> > Kafka
>>> > > > > > > > > >> >> > > > > rest
>>> > > > > > > > > >> >> > > > > > > > proxy
>>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to
>>> run a
>>> > > > task
>>> > > > > on
>>> > > > > > > > > behalf
>>> > > > > > > > > >> of a
>>> > > > > > > > > >> >> > > > client.
>>> > > > > > > > > >> >> > > > > > We
>>> > > > > > > > > >> >> > > > > > > > will
>>> > > > > > > > > >> >> > > > > > > > > likely need to change how those
>>> > services
>>> > > > use
>>> > > > > > > Kafka
>>> > > > > > > > > >> clients
>>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
>>> > shared
>>> > > > > client
>>> > > > > > > per
>>> > > > > > > > > >> worker,
>>> > > > > > > > > >> >> > we
>>> > > > > > > > > >> >> > > > will
>>> > > > > > > > > >> >> > > > > > > need
>>> > > > > > > > > >> >> > > > > > > > a
>>> > > > > > > > > >> >> > > > > > > > > client per user task since the
>>> > > > authentication
>>> > > > > > > > happens
>>> > > > > > > > > >> at the
>>> > > > > > > > > >> >> > > > > > connection
>>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
>>> renewer
>>> > > will
>>> > > > be
>>> > > > > > the
>>> > > > > > > > > >> workers.
>>> > > > > > > > > >> >> > So,
>>> > > > > > > > > >> >> > > we
>>> > > > > > > > > >> >> > > > > > > > probably
>>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For
>>> > > Kafka
>>> > > > > rest
>>> > > > > > > > > proxy,
>>> > > > > > > > > >> the
>>> > > > > > > > > >> >> > > > renewer
>>> > > > > > > > > >> >> > > > > > can
>>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of the
>>> > > token.
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can
>>> > > request
>>> > > > > > > > delegation
>>> > > > > > > > > >> tokens?
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
>>> > > > > delegation
>>> > > > > > > > tokens
>>> > > > > > > > > >> in an
>>> > > > > > > > > >> >> > > > > encrypted
>>> > > > > > > > > >> >> > > > > > > > > channel?
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
>>> > > > tokens,
>>> > > > > > > every
>>> > > > > > > > > >> broker?
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would
>>> it
>>> > be
>>> > > > > > better
>>> > > > > > > to
>>> > > > > > > > > do
>>> > > > > > > > > >> it in
>>> > > > > > > > > >> >> > a
>>> > > > > > > > > >> >> > > > > > request
>>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in
>>> the
>>> > > wiki
>>> > > > > > > > sometimes
>>> > > > > > > > > >> refers
>>> > > > > > > > > >> >> > to
>>> > > > > > > > > >> >> > > > the
>>> > > > > > > > > >> >> > > > > > end
>>> > > > > > > > > >> >> > > > > > > > > client and some other times refers
>>> to
>>> > the
>>> > > > > > client
>>> > > > > > > > > using
>>> > > > > > > > > >> the
>>> > > > > > > > > >> >> > > > > delegation
>>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
>>> > distinguish
>>> > > > > > between
>>> > > > > > > > the
>>> > > > > > > > > >> two.
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence
>>> > > "Broker
>>> > > > > > also
>>> > > > > > > > > >> stores the
>>> > > > > > > > > >> >> > > > > > > > DelegationToken
>>> > > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper."
>>> a
>>> > bit
>>> > > > > > more? I
>>> > > > > > > > > >> thought the
>>> > > > > > > > > >> >> > > > > > > delegation
>>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > Thanks,
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > Jun
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun
>>> > Rao
>>> > > <
>>> > > > > > > > > >> jun@confluent.io>
>>> > > > > > > > > >> >> > > > wrote:
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
>>> > > > > > > > > >> >> > > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
>>> invite.
>>> > We
>>> > > > can
>>> > > > > > > > discuss
>>> > > > > > > > > >> this in
>>> > > > > > > > > >> >> > > the
>>> > > > > > > > > >> >> > > > > > > meeting
>>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
>>> > > > > > > > > >> >> > > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > > Thanks,
>>> > > > > > > > > >> >> > > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > > Jun
>>> > > > > > > > > >> >> > > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM,
>>> > > Harsha <
>>> > > > > > > > > >> kafka@harsha.io>
>>> > > > > > > > > >> >> > > > wrote:
>>> > > > > > > > > >> >> > > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
>>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
>>> meeting
>>> > > > > around
>>> > > > > > > > this.
>>> > > > > > > > > >> The KIP
>>> > > > > > > > > >> >> > is
>>> > > > > > > > > >> >> > > > up
>>> > > > > > > > > >> >> > > > > > for
>>> > > > > > > > > >> >> > > > > > > > > >> sometime and if there
>>> are
>>> > > any
>>> > > > > > > > questions
>>> > > > > > > > > >> lets
>>> > > > > > > > > >> >> > > > quickly
>>> > > > > > > > > >> >> > > > > > hash
>>> > > > > > > > > >> >> > > > > > > > out
>>> > > > > > > > > >> >> > > > > > > > > >> details.
>>> > > > > > > > > >> >> > > > > > > > > >>
>>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
>>> > > > > > > > > >> >> > > > > > > > > >> Harsha
>>> > > > > > > > > >> >> > > > > > > > > >>
>>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40
>>> AM,
>>> > > parth
>>> > > > > > > > > brahmbhatt
>>> > > > > > > > > >> wrote:
>>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
>>> > system
>>> > > > uses
>>> > > > > > so
>>> > > > > > > no
>>> > > > > > > > > >> good
>>> > > > > > > > > >> >> > reason
>>> > > > > > > > > >> >> > > > > > really.
>>> > > > > > > > > >> >> > > > > > > > We
>>> > > > > > > > > >> >> > > > > > > > > >> > could
>>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
>>> > newest
>>> > > > > > > > recommended
>>> > > > > > > > > >> standard
>>> > > > > > > > > >> >> > > is.
>>> > > > > > > > > >> >> > > > > > > > > >> >
>>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
>>> > > > > > > > > >> >> > > > > > > > > >> > Parth
>>> > > > > > > > > >> >> > > > > > > > > >> >
>>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33
>>> AM,
>>> > > > Ismael
>>> > > > > > > Juma <
>>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
>>> > > > > > > > > >> >> > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > wrote:
>>> > > > > > > > > >> >> > > > > > > > > >> >
>>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
>>> > started
>>> > > > > > > reviewing
>>> > > > > > > > > >> this and
>>> > > > > > > > > >> >> > > may
>>> > > > > > > > > >> >> > > > > have
>>> > > > > > > > > >> >> > > > > > > > > >> additional
>>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
>>> immediate
>>> > > > > question
>>> > > > > > > that
>>> > > > > > > > > >> came to
>>> > > > > > > > > >> >> > > mind
>>> > > > > > > > > >> >> > > > is
>>> > > > > > > > > >> >> > > > > > our
>>> > > > > > > > > >> >> > > > > > > > > >> choice of
>>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's
>>> > > marked
>>> > > > > as
>>> > > > > > > > > >> OBSOLETE in
>>> > > > > > > > > >> >> > the
>>> > > > > > > > > >> >> > > > IANA
>>> > > > > > > > > >> >> > > > > > > > > Registry
>>> > > > > > > > > >> >> > > > > > > > > >> of
>>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
>>> original
>>> > > RFC
>>> > > > > > > (2831)
>>> > > > > > > > > has
>>> > > > > > > > > >> been
>>> > > > > > > > > >> >> > > moved
>>> > > > > > > > > >> >> > > > > to
>>> > > > > > > > > >> >> > > > > > > > > Historic
>>> > > > > > > > > >> >> > > > > > > > > >> > > status:
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/
>>> > > rfc6331
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > >
>>> > > > > > > > > >> >> > >
>>> > > > > > > > > >>
>>> > > > > > >
>>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
>>> mechanisms.xhtml
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind
>>> > that
>>> > > > > > choice?
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
>>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29
>>> > PM,
>>> > > > Gwen
>>> > > > > > > > > Shapira <
>>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
>>> > > > > > > > > >> >> > > > > > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> wrote:
>>> > > > > > > > > >> >> > > > > > > > > >> > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize
>>> that
>>> > > even
>>> > > > > > though
>>> > > > > > > > > >> delegation
>>> > > > > > > > > >> >> > > > tokens
>>> > > > > > > > > >> >> > > > > > > are a
>>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very
>>> > > strongly
>>> > > > > > about
>>> > > > > > > > not
>>> > > > > > > > > >> adding
>>> > > > > > > > > >> >> > > > > > dependency
>>> > > > > > > > > >> >> > > > > > > > on
>>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
>>> delegation
>>> > > > > tokens
>>> > > > > > > for
>>> > > > > > > > > >> Kafka. The
>>> > > > > > > > > >> >> > > KIP
>>> > > > > > > > > >> >> > > > > > > doesn't
>>> > > > > > > > > >> >> > > > > > > > > >> imply
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if
>>> you
>>> > > can
>>> > > > > > > > clarify...
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the
>>> KIP
>>> > so
>>> > > > no
>>> > > > > > one
>>> > > > > > > > will
>>> > > > > > > > > >> read
>>> > > > > > > > > >> >> > the
>>> > > > > > > > > >> >> > > > KIP
>>> > > > > > > > > >> >> > > > > > and
>>> > > > > > > > > >> >> > > > > > > > > panic
>>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
>>> > > > > release...
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation
>>> > token
>>> > > at
>>> > > > > any
>>> > > > > > > > time
>>> > > > > > > > > >> after
>>> > > > > > > > > >> >> > > > > > > > authenticating?
>>> > > > > > > > > >> >> > > > > > > > > >> only
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
>>> > > > authenticated
>>> > > > > > you
>>> > > > > > > > can
>>> > > > > > > > > >> get
>>> > > > > > > > > >> >> > > > delegation
>>> > > > > > > > > >> >> > > > > > > > tokens.
>>> > > > > > > > > >> >> > > > > > > > > >> We
>>> > > > > > > > > >> >> > > > > > > > > >> > > need
>>> > > > > > > > > >> >> > > > > > > > > >> > > > to
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
>>> > > > authenticated
>>> > > > > > > using
>>> > > > > > > > > >> delegation
>>> > > > > > > > > >> >> > > > > token,
>>> > > > > > > > > >> >> > > > > > > can
>>> > > > > > > > > >> >> > > > > > > > > also
>>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or
>>> > not.
>>> > > > > Also
>>> > > > > > > > there
>>> > > > > > > > > >> is the
>>> > > > > > > > > >> >> > > > > question
>>> > > > > > > > > >> >> > > > > > of
>>> > > > > > > > > >> >> > > > > > > > do
>>> > > > > > > > > >> >> > > > > > > > > we
>>> > > > > > > > > >> >> > > > > > > > > >> > > allow
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
>>> delegation
>>> > > > token
>>> > > > > > or
>>> > > > > > > we
>>> > > > > > > > > >> want
>>> > > > > > > > > >> >> > > specific
>>> > > > > > > > > >> >> > > > > > ACLs
>>> > > > > > > > > >> >> > > > > > > (I
>>> > > > > > > > > >> >> > > > > > > > > >> think
>>> > > > > > > > > >> >> > > > > > > > > >> > > its
>>> > > > > > > > > >> >> > > > > > > > > >> > > > an
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
>>> > > overkill.
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
>>> > > > options:
>>> > > > > > > > Either
>>> > > > > > > > > >> require
>>> > > > > > > > > >> >> > > > > Kerberos
>>> > > > > > > > > >> >> > > > > > > > auth
>>> > > > > > > > > >> >> > > > > > > > > >> for
>>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require
>>> non-owners
>>> > to
>>> > > > > > renew.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
>>> > simpler
>>> > > > (it
>>> > > > > > > > > basically
>>> > > > > > > > > >> >> > require
>>> > > > > > > > > >> >> > > a
>>> > > > > > > > > >> >> > > > > "job
>>> > > > > > > > > >> >> > > > > > > > > master"
>>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for
>>> the
>>> > > > > > renewal,
>>> > > > > > > it
>>> > > > > > > > > >> will have
>>> > > > > > > > > >> >> > > its
>>> > > > > > > > > >> >> > > > > own
>>> > > > > > > > > >> >> > > > > > > > > >> identity
>>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is
>>> the
>>> > > > > correct
>>> > > > > > > > > design
>>> > > > > > > > > >> >> > pattern
>>> > > > > > > > > >> >> > > > > > anyway.
>>> > > > > > > > > >> >> > > > > > > > For
>>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
>>> > > > > coordinate
>>> > > > > > > > > >> renewals?),
>>> > > > > > > > > >> >> > but
>>> > > > > > > > > >> >> > > > it
>>> > > > > > > > > >> >> > > > > is
>>> > > > > > > > > >> >> > > > > > > > hard
>>> > > > > > > > > >> >> > > > > > > > > to
>>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without
>>> > > looking
>>> > > > at
>>> > > > > > the
>>> > > > > > > > > code
>>> > > > > > > > > >> >> > changes
>>> > > > > > > > > >> >> > > > > > > required.
>>> > > > > > > > > >> >> > > > > > > > If
>>> > > > > > > > > >> >> > > > > > > > > >> you
>>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
>>> > "require
>>> > > > > > > Kerberos"
>>> > > > > > > > > >> will look
>>> > > > > > > > > >> >> > > in
>>> > > > > > > > > >> >> > > > > > Kafka
>>> > > > > > > > > >> >> > > > > > > > > code,
>>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a
>>> look.
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is
>>> that
>>> > > > tokens
>>> > > > > > will
>>> > > > > > > > > >> propagate
>>> > > > > > > > > >> >> > via
>>> > > > > > > > > >> >> > > > ZK
>>> > > > > > > > > >> >> > > > > > but
>>> > > > > > > > > >> >> > > > > > > > > >> without
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
>>> > > > > UpdateMetadata
>>> > > > > > > > > >> protocol,
>>> > > > > > > > > >> >> > > > correct?
>>> > > > > > > > > >> >> > > > > > > > Clients
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on
>>> > SASL
>>> > > > auth
>>> > > > > > > > failure
>>> > > > > > > > > >> (IIRC),
>>> > > > > > > > > >> >> > > but
>>> > > > > > > > > >> >> > > > > > since
>>> > > > > > > > > >> >> > > > > > > > the
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between
>>> > > brokers
>>> > > > > > > asynch,
>>> > > > > > > > > we
>>> > > > > > > > > >> will
>>> > > > > > > > > >> >> > > need
>>> > > > > > > > > >> >> > > > to
>>> > > > > > > > > >> >> > > > > > > > retry a
>>> > > > > > > > > >> >> > > > > > > > > >> bit
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing
>>> > auth
>>> > > > due
>>> > > > > to
>>> > > > > > > > > timing
>>> > > > > > > > > >> >> > issues.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
>>> > > alternatives
>>> > > > > > right
>>> > > > > > > > > now.
>>> > > > > > > > > >> The
>>> > > > > > > > > >> >> > > > current
>>> > > > > > > > > >> >> > > > > > > > > documented
>>> > > > > > > > > >> >> > > > > > > > > >> > > > approach
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it
>>> > does
>>> > > > not
>>> > > > > > > > require
>>> > > > > > > > > >> any
>>> > > > > > > > > >> >> > > changes
>>> > > > > > > > > >> >> > > > > to
>>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
>>> > > > approach
>>> > > > > > can
>>> > > > > > > > > remove
>>> > > > > > > > > >> >> > > zookeeper
>>> > > > > > > > > >> >> > > > > > > > > dependency
>>> > > > > > > > > >> >> > > > > > > > > >> as
>>> > > > > > > > > >> >> > > > > > > > > >> > > well
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that
>>> in
>>> > KIP
>>> > > > > > > > discussion
>>> > > > > > > > > >> call.*
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting.
>>> Do
>>> > you
>>> > > > > want
>>> > > > > > to
>>> > > > > > > > > ping
>>> > > > > > > > > >> Jun to
>>> > > > > > > > > >> >> > > > > > arrange a
>>> > > > > > > > > >> >> > > > > > > > > call?
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
>>> > suggestion
>>> > > of
>>> > > > > > > having
>>> > > > > > > > > >> just the
>>> > > > > > > > > >> >> > > > > > controller
>>> > > > > > > > > >> >> > > > > > > > > issue
>>> > > > > > > > > >> >> > > > > > > > > >> the
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to
>>> avoid
>>> > > > > syncing
>>> > > > > > a
>>> > > > > > > > > shared
>>> > > > > > > > > >> >> > secret.
>>> > > > > > > > > >> >> > > > Not
>>> > > > > > > > > >> >> > > > > > > sure
>>> > > > > > > > > >> >> > > > > > > > if
>>> > > > > > > > > >> >> > > > > > > > > >> we
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
>>> > > discussion
>>> > > > > here
>>> > > > > > > or
>>> > > > > > > > on
>>> > > > > > > > > >> the
>>> > > > > > > > > >> >> > > wiki. I
>>> > > > > > > > > >> >> > > > > > think
>>> > > > > > > > > >> >> > > > > > > > > that
>>> > > > > > > > > >> >> > > > > > > > > >> we
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem
>>> of
>>> > > > "token
>>> > > > > > > > > >> distribution"
>>> > > > > > > > > >> >> > > from
>>> > > > > > > > > >> >> > > > > > > "shared
>>> > > > > > > > > >> >> > > > > > > > > >> secret
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
>>> > > > > controller
>>> > > > > > > as
>>> > > > > > > > > the
>>> > > > > > > > > >> only
>>> > > > > > > > > >> >> > > token
>>> > > > > > > > > >> >> > > > > > > > generator
>>> > > > > > > > > >> >> > > > > > > > > >> to
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue,
>>> > while
>>> > > > > still
>>> > > > > > > > using
>>> > > > > > > > > >> ZK async
>>> > > > > > > > > >> >> > > to
>>> > > > > > > > > >> >> > > > > > > > distribute
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
>>> > previous
>>> > > > > Email
>>> > > > > > I
>>> > > > > > > am
>>> > > > > > > > > >> fine with
>>> > > > > > > > > >> >> > > > that
>>> > > > > > > > > >> >> > > > > > > > approach
>>> > > > > > > > > >> >> > > > > > > > > >> as
>>> > > > > > > > > >> >> > > > > > > > > >> > > long
>>> > > > > > > > > >> >> > > > > > > > > >> > > > as
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
>>> > > > complexity
>>> > > > > > of
>>> > > > > > > > > >> >> > > adding/updating
>>> > > > > > > > > >> >> > > > > APIS
>>> > > > > > > > > >> >> > > > > > > > adds
>>> > > > > > > > > >> >> > > > > > > > > >> enough
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with
>>> > the
>>> > > > > > > controller
>>> > > > > > > > > >> approach
>>> > > > > > > > > >> >> > is
>>> > > > > > > > > >> >> > > > > > secret
>>> > > > > > > > > >> >> > > > > > > > > >> rotation
>>> > > > > > > > > >> >> > > > > > > > > >> > > can
>>> > > > > > > > > >> >> > > > > > > > > >> > > > be
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and
>>> would
>>> > > not
>>> > > > > > > require
>>> > > > > > > > > >> >> > > deployment. *
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
>>> > > > complexity
>>> > > > > > (or
>>> > > > > > > > > point
>>> > > > > > > > > >> me to
>>> > > > > > > > > >> >> > > the
>>> > > > > > > > > >> >> > > > > > email
>>> > > > > > > > > >> >> > > > > > > I
>>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
>>> > > > required?
>>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell,
>>> clients
>>> > > can
>>> > > > > > > already
>>> > > > > > > > > >> find the
>>> > > > > > > > > >> >> > > > > > controller
>>> > > > > > > > > >> >> > > > > > > > from
>>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
>>> > > concerned
>>> > > > > > about
>>> > > > > > > > > >> controller
>>> > > > > > > > > >> >> > > > load.
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea
>>> of
>>> > > > forcing
>>> > > > > > > > > kerberos
>>> > > > > > > > > >> auth
>>> > > > > > > > > >> >> > for
>>> > > > > > > > > >> >> > > > > > > renwal, I
>>> > > > > > > > > >> >> > > > > > > > > >> think
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport
>>> layer
>>> > > the
>>> > > > > the
>>> > > > > > > > > request
>>> > > > > > > > > >> >> > content
>>> > > > > > > > > >> >> > > > in
>>> > > > > > > > > >> >> > > > > a
>>> > > > > > > > > >> >> > > > > > > > pretty
>>> > > > > > > > > >> >> > > > > > > > > >> ugly
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
>>> > renewer
>>> > > to
>>> > > > > > > > non-owner
>>> > > > > > > > > >> is
>>> > > > > > > > > >> >> > > better.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a
>>> necessary
>>> > > > evil.
>>> > > > > > > While
>>> > > > > > > > > >> this will
>>> > > > > > > > > >> >> > > make
>>> > > > > > > > > >> >> > > > > the
>>> > > > > > > > > >> >> > > > > > > > kafka
>>> > > > > > > > > >> >> > > > > > > > > >> code
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
>>> > > > forcing
>>> > > > > > > > renewer
>>> > > > > > > > > >> to
>>> > > > > > > > > >> >> > > > non-owner
>>> > > > > > > > > >> >> > > > > > > pushes
>>> > > > > > > > > >> >> > > > > > > > > >> the code
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and
>>> makes
>>> > > it
>>> > > > > even
>>> > > > > > > > > harder
>>> > > > > > > > > >> to
>>> > > > > > > > > >> >> > > > > > integrate. *
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I
>>> don't
>>> > > think
>>> > > > > the
>>> > > > > > > > > >> "renewal by
>>> > > > > > > > > >> >> > > > other"
>>> > > > > > > > > >> >> > > > > > > > approach
>>> > > > > > > > > >> >> > > > > > > > > >> is
>>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients
>>> we
>>> > > > expect
>>> > > > > to
>>> > > > > > > use
>>> > > > > > > > > >> >> > delegation
>>> > > > > > > > > >> >> > > > > tokens
>>> > > > > > > > > >> >> > > > > > > > since
>>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an
>>> app-master
>>> > of
>>> > > > some
>>> > > > > > > sort
>>> > > > > > > > > who
>>> > > > > > > > > >> >> > > requested
>>> > > > > > > > > >> >> > > > > the
>>> > > > > > > > > >> >> > > > > > > > token
>>> > > > > > > > > >> >> > > > > > > > > to
>>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with.
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my
>>> question
>>> > > on
>>> > > > > how
>>> > > > > > > > > multiple
>>> > > > > > > > > >> >> > > > identities
>>> > > > > > > > > >> >> > > > > > will
>>> > > > > > > > > >> >> > > > > > > > be
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super
>>> clear
>>> > to
>>> > > > me -
>>> > > > > > > > AFAIK,
>>> > > > > > > > > >> we don't
>>> > > > > > > > > >> >> > > > > > > > authenticate
>>> > > > > > > > > >> >> > > > > > > > > >> each
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
>>> > > > > > connections.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
>>> > connections,
>>> > > > and
>>> > > > > > only
>>> > > > > > > > > when
>>> > > > > > > > > >> they
>>> > > > > > > > > >> >> > are
>>> > > > > > > > > >> >> > > > > being
>>> > > > > > > > > >> >> > > > > > > > > >> established.
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Let
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as
>>> a
>>> > > > > question,
>>> > > > > > > in
>>> > > > > > > > > >> absence of
>>> > > > > > > > > >> >> > > > > > > delegation
>>> > > > > > > > > >> >> > > > > > > > > >> tokens if
>>> > > > > > > > > >> >> > > > > > > > > >> > > > we
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use
>>> case
>>> > > > using
>>> > > > > > user
>>> > > > > > > > > >> TGT's how
>>> > > > > > > > > >> >> > > would
>>> > > > > > > > > >> >> > > > > we
>>> > > > > > > > > >> >> > > > > > do
>>> > > > > > > > > >> >> > > > > > > > it?
>>> > > > > > > > > >> >> > > > > > > > > >> My
>>> > > > > > > > > >> >> > > > > > > > > >> > > point
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no
>>> different
>>> > > > with
>>> > > > > > > > > delegation
>>> > > > > > > > > >> >> > tokens.
>>> > > > > > > > > >> >> > > > The
>>> > > > > > > > > >> >> > > > > > use
>>> > > > > > > > > >> >> > > > > > > > > case
>>> > > > > > > > > >> >> > > > > > > > > >> you
>>> > > > > > > > > >> >> > > > > > > > > >> > > are
>>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more
>>> like
>>> > > > > > > > impersonation.*
>>> > > > > > > > > >> >> > > > > > > > > >> > > >
>>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of
>>> > the
>>> > > > > things
>>> > > > > > > > that
>>> > > > > > > > > >> >> > del
>>>
>>
>>
>>
>> --
>>
>> Regards,
>> Ashish
>>
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Ashish,
Yes, I will send out a KIP invite for next week to discuss KIP-48 and other
remaining KIPs.
Thanks,
Jun
On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <as...@cloudera.com> wrote:
> Thanks Harsha!
>
> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
> actually make a call on when we should have next KIP call. As there are a
> few outstanding KIPs that could not be discussed this week, can we have a
> KIP hangout call next week?
>
> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Ashish,
>> Yes we are working on it. Lets discuss in the next KIP meeting.
>> I'll join.
>> -Harsha
>>
>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com>
>> wrote:
>>
>> > Hello Harsha,
>> >
>> > Are you still working on this? Wondering if we can discuss this in next
>> KIP
>> > meeting, if you can join.
>> >
>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io>
>> > wrote:
>> >
>> > > Hi Grant,
>> > > We are working on it. Will add the details to KIP about the
>> > > request protocol.
>> > >
>> > > Thanks,
>> > > Harsha
>> > >
>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com>
>> wrote:
>> > >
>> > > > Hi Parth,
>> > > >
>> > > > Are you still working on this? If you need any help please don't
>> > hesitate
>> > > > to ask.
>> > > >
>> > > > Thanks,
>> > > > Grant
>> > > >
>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
>> > > >
>> > > > > Parth,
>> > > > >
>> > > > > Thanks for the reply.
>> > > > >
>> > > > > It makes sense to only allow the renewal by users that
>> authenticated
>> > > > using
>> > > > > *non* delegation token mechanism. Then, should we make the
>> renewal a
>> > > > list?
>> > > > > For example, in the case of rest proxy, it will be useful for
>> every
>> > > > > instance of rest proxy to be able to renew the tokens.
>> > > > >
>> > > > > It would be clearer if we can document the request protocol like
>> > > > >
>> > > > >
>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
>> > > Commandlineandcentralizedadministrativeoperations-
>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
>> > > > > .
>> > > > >
>> > > > > It would also be useful to document the client APIs.
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Jun
>> > > > >
>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
>> > > > > brahmbhatt.parth@gmail.com> wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I am suggesting that we will only allow the renewal by users
>> that
>> > > > > > authenticated using *non* delegation token mechanism. For
>> example,
>> > If
>> > > > > user
>> > > > > > Alice authenticated using kerberos and requested delegation
>> tokens,
>> > > > only
>> > > > > > user Alice authenticated via non delegation token mechanism can
>> > > renew.
>> > > > > > Clients that have access to delegation tokens can not issue
>> > renewal
>> > > > > > request for renewing their own token and this is primarily
>> > important
>> > > to
>> > > > > > reduce the time window for which a compromised token will be
>> valid.
>> > > > > >
>> > > > > > To clarify, Yes any authenticated user can request delegation
>> > tokens
>> > > > but
>> > > > > > even here I would recommend to avoid creating a chain where a
>> > client
>> > > > > > authenticated via delegation token request for more delegation
>> > > tokens.
>> > > > > > Basically anyone can request delegation token, as long as they
>> > > > > authenticate
>> > > > > > via a non delegation token mechanism.
>> > > > > >
>> > > > > > Aren't classes listed here
>> > > > > > <
>> > > > > >
>> > > > >
>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens
>> upportforKaf
>> > > ka-PublicInterfaces
>> > > > > > >
>> > > > > > sufficient?
>> > > > > >
>> > > > > > Thanks
>> > > > > > Parth
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io>
>> wrote:
>> > > > > >
>> > > > > > > Parth,
>> > > > > > >
>> > > > > > > Thanks for the reply. A couple of comments inline below.
>> > > > > > >
>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > 1. Who / how are tokens renewed? By original requester
>> only? or
>> > > > using
>> > > > > > > > Kerberos
>> > > > > > > > auth only?
>> > > > > > > > My recommendation is to do this only using Kerberos auth and
>> > only
>> > > > > threw
>> > > > > > > the
>> > > > > > > > renewer specified during the acquisition request.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > Hmm, not sure that I follow this. Are you saying that any
>> client
>> > > > > > > authenticated with the delegation token can renew, i.e. there
>> is
>> > no
>> > > > > > renewer
>> > > > > > > needed?
>> > > > > > >
>> > > > > > > Also, just to be clear, any authenticated client (either
>> through
>> > > SASL
>> > > > > or
>> > > > > > > SSL) can request a delegation token for the authenticated
>> user,
>> > > > right?
>> > > > > > >
>> > > > > > >
>> > > > > > > > 2. Are tokens stored on each broker or in ZK?
>> > > > > > > > My recommendation is still to store in ZK or not store them
>> at
>> > > all.
>> > > > > The
>> > > > > > > > whole controller based distribution is too much overhead
>> with
>> > not
>> > > > > much
>> > > > > > to
>> > > > > > > > achieve.
>> > > > > > > >
>> > > > > > > > 3. How are tokens invalidated / expired?
>> > > > > > > > Either by expiration time out or through an explicit
>> request to
>> > > > > > > invalidate.
>> > > > > > > >
>> > > > > > > > 4. Which encryption algorithm is used?
>> > > > > > > > SCRAM
>> > > > > > > >
>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP
>> but
>> > > was
>> > > > > > > > discussed
>> > > > > > > > in this thread)?
>> > > > > > > > There is no imperonation proposal. I tried and explained how
>> > its
>> > > a
>> > > > > > > > different problem and why its not really necessary to
>> discuss
>> > > that
>> > > > as
>> > > > > > > part
>> > > > > > > > of this KIP. This KIP will not support any impersonation,
>> it
>> > > will
>> > > > > just
>> > > > > > > be
>> > > > > > > > another way to authenticate.
>> > > > > > > >
>> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
>> > > > > > > > We do not need new ACLs.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > Could we document the format of the new request/response and
>> > their
>> > > > > > > associated Resource and Operation for ACL?
>> > > > > > >
>> > > > > > >
>> > > > > > > > 7. How would the delegation token be configured in the
>> client?
>> > > > > > > > Should be through config. I wasn't planning on supporting
>> JAAS
>> > > for
>> > > > > > > tokens.
>> > > > > > > > I don't believe hadoop does this either.
>> > > > > > > >
>> > > > > > > > Thanks
>> > > > > > > > Parth
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io>
>> > > wrote:
>> > > > > > > >
>> > > > > > > > > Harsha,
>> > > > > > > > >
>> > > > > > > > > Another question.
>> > > > > > > > >
>> > > > > > > > > 9. How would the delegation token be configured in the
>> > client?
>> > > > The
>> > > > > > > > standard
>> > > > > > > > > way is to do this through JAAS. However, we will need to
>> > think
>> > > > > > through
>> > > > > > > if
>> > > > > > > > > this is convenient in a shared environment. For example,
>> > when a
>> > > > new
>> > > > > > > task
>> > > > > > > > is
>> > > > > > > > > added to a Storm worker node, do we need to dynamically
>> add a
>> > > new
>> > > > > > > section
>> > > > > > > > > in the JAAS file? It may be more convenient if we can
>> pass in
>> > > the
>> > > > > > token
>> > > > > > > > > through the config directly w/o going through JAAS.
>> > > > > > > > >
>> > > > > > > > > Are you or Parth still actively working on this KIP?
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > >
>> > > > > > > > > Jun
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <
>> jun@confluent.io>
>> > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Just to add on that list.
>> > > > > > > > > >
>> > > > > > > > > > 2. It would be good to document the format of the data
>> > stored
>> > > > in
>> > > > > > ZK.
>> > > > > > > > > > 7. Earlier, there was a discussion on whether the tokens
>> > > should
>> > > > > be
>> > > > > > > > > > propagated through ZK like config/acl/quota, or through
>> the
>> > > > > > > controller.
>> > > > > > > > > > Currently, the controller is only designed for
>> propagating
>> > > > topic
>> > > > > > > > > metadata,
>> > > > > > > > > > but not other data.
>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
>> > > DIGEST-MD5
>> > > > > > since
>> > > > > > > > it's
>> > > > > > > > > > deprecated?
>> > > > > > > > > >
>> > > > > > > > > > Also, the images in the wiki seem broken.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
>> > > > > gwen@confluent.io>
>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > >> From what I can see, remaining questions are:
>> > > > > > > > > >>
>> > > > > > > > > >> 1. Who / how are tokens renewed? By original requester
>> > only?
>> > > > or
>> > > > > > > using
>> > > > > > > > > >> Kerberos auth only?
>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
>> > > > > > > > > >> 3. How are tokens invalidated / expired?
>> > > > > > > > > >> 4. Which encryption algorithm is used?
>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the
>> > KIP
>> > > > but
>> > > > > > was
>> > > > > > > > > >> discussed in this thread)?
>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
>> > > > > > > > > >>
>> > > > > > > > > >> Gwen
>> > > > > > > > > >>
>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <
>> kafka@harsha.io>
>> > > > wrote:
>> > > > > > > > > >> > Jun & Ismael,
>> > > > > > > > > >> > Unfortunately I couldn't
>> attend
>> > > the
>> > > > > KIP
>> > > > > > > > > meeting
>> > > > > > > > > >> > when delegation tokens
>> > discussed.
>> > > > > > > > Appreciate
>> > > > > > > > > if
>> > > > > > > > > >> > you can update the thread if
>> > you
>> > > > have
>> > > > > > any
>> > > > > > > > > >> > further questions.
>> > > > > > > > > >> > Thanks,
>> > > > > > > > > >> > Harsha
>> > > > > > > > > >> >
>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
>> > > > > > > > > >> >> It seems that the links to images in the KIP are
>> > broken.
>> > > > > > > > > >> >>
>> > > > > > > > > >> >> Liquan
>> > > > > > > > > >> >>
>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
>> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
>> > > > > > > > > >> >>
>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
>> > > > > > > > > >> >> > In the current proposal we only allow a user to
>> get
>> > > > > > delegation
>> > > > > > > > > token
>> > > > > > > > > >> for
>> > > > > > > > > >> >> > the identity that it authenticated as using
>> another
>> > > > > > mechanism,
>> > > > > > > > i.e.
>> > > > > > > > > >> A user
>> > > > > > > > > >> >> > that authenticate using a keytab for principal
>> > > > > > > user1@EXAMPLE.COM
>> > > > > > > > > >> will get
>> > > > > > > > > >> >> > delegation tokens for that user only. In future I
>> > think
>> > > > we
>> > > > > > will
>> > > > > > > > > have
>> > > > > > > > > >> to
>> > > > > > > > > >> >> > extend support such that we allow some set of
>> users (
>> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
>> > storm-nimbus@EXAMPLE.COM)
>> > > > to
>> > > > > > > > acquire
>> > > > > > > > > >> >> > delegation tokens on behalf of other users whose
>> > > identity
>> > > > > > they
>> > > > > > > > have
>> > > > > > > > > >> >> > verified independently. Kafka brokers will have
>> ACLs
>> > > to
>> > > > > > > control
>> > > > > > > > > >> which
>> > > > > > > > > >> >> > users are allowed to impersonate other users and
>> get
>> > > > tokens
>> > > > > > on
>> > > > > > > > > >> behalf of
>> > > > > > > > > >> >> > them. Overall Impersonation is a whole different
>> > > problem
>> > > > in
>> > > > > > my
>> > > > > > > > > >> opinion and
>> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> > 111. What's the typical rate of getting and
>> renewing
>> > > > > > delegation
>> > > > > > > > > >> tokens?
>> > > > > > > > > >> >> > Typically this should be very very low, 1 request
>> per
>> > > > > minute
>> > > > > > > is a
>> > > > > > > > > >> >> > relatively high estimate. However it depends on
>> the
>> > > token
>> > > > > > > > > >> expiration. I am
>> > > > > > > > > >> >> > less worried about the extra load it puts on
>> > controller
>> > > > vs
>> > > > > > the
>> > > > > > > > > added
>> > > > > > > > > >> >> > complexity and the value it offers.
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> > Thanks
>> > > > > > > > > >> >> > Parth
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
>> > > > > > > ismael@juma.me.uk>
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> >> >
>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
>> separate
>> > > KIP
>> > > > > as
>> > > > > > it
>> > > > > > > > > will
>> > > > > > > > > >> >> > > introduce user visible changes. We could also
>> > update
>> > > > > KIP-48
>> > > > > > > to
>> > > > > > > > > >> have this
>> > > > > > > > > >> >> > > information, but it seems cleaner to do it
>> > > separately.
>> > > > We
>> > > > > > can
>> > > > > > > > > >> discuss
>> > > > > > > > > >> >> > that
>> > > > > > > > > >> >> > > in the KIP call today.
>> > > > > > > > > >> >> > >
>> > > > > > > > > >> >> > > Ismael
>> > > > > > > > > >> >> > >
>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram
>> <
>> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
>> > > > > > > > > >> >> > >
>> > > > > > > > > >> >> > > > Ismael,
>> > > > > > > > > >> >> > > >
>> > > > > > > > > >> >> > > > I have created a JIRA (
>> > > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would
>> that
>> > > need
>> > > > > > > another
>> > > > > > > > > >> KIP? If
>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just
>> be
>> > a
>> > > > JIRA
>> > > > > > > that
>> > > > > > > > > gets
>> > > > > > > > > >> >> > > reviewed
>> > > > > > > > > >> >> > > > when the PR is ready?
>> > > > > > > > > >> >> > > >
>> > > > > > > > > >> >> > > > Thank you,
>> > > > > > > > > >> >> > > >
>> > > > > > > > > >> >> > > > Rajini
>> > > > > > > > > >> >> > > >
>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
>> > > > > > > > > ismael@juma.me.uk>
>> > > > > > > > > >> >> > wrote:
>> > > > > > > > > >> >> > > >
>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
>> > candidate.
>> > > > > > > > > >> >> > > > >
>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as a
>> SASL
>> > > > > > mechanism
>> > > > > > > > > that
>> > > > > > > > > >> might
>> > > > > > > > > >> >> > be
>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to
>> > check
>> > > > it
>> > > > > in
>> > > > > > > > more
>> > > > > > > > > >> detail.
>> > > > > > > > > >> >> > > Good
>> > > > > > > > > >> >> > > > > to know that you are willing to contribute
>> an
>> > > > > > > > implementation.
>> > > > > > > > > >> Maybe
>> > > > > > > > > >> >> > we
>> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
>> > > > > > > > > >> >> > > > >
>> > > > > > > > > >> >> > > > > Ismael
>> > > > > > > > > >> >> > > > >
>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
>> > Sivaram <
>> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
>> > > > > > > > > >> >> > > > >
>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
>> > Authentication
>> > > > > > > > Mechanism)
>> > > > > > > > > >> is a
>> > > > > > > > > >> >> > > better
>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't
>> come
>> > > > with a
>> > > > > > > > > built-in
>> > > > > > > > > >> SCRAM
>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be
>> happy
>> > > to
>> > > > > add
>> > > > > > > > > support
>> > > > > > > > > >> in
>> > > > > > > > > >> >> > Kafka
>> > > > > > > > > >> >> > > > > since
>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to support
>> > > anyway.
>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
>> > describes
>> > > > the
>> > > > > > > > protocol
>> > > > > > > > > >> for
>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
>> > > > > > > > > >> >> > > > > >
>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
>> > > > > > > > jun@confluent.io
>> > > > > > > > > >
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> >> > > > > >
>> > > > > > > > > >> >> > > > > > > Parth,
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of
>> > more
>> > > > > > > questions.
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs
>> mean?
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting
>> and
>> > > > > > renewing
>> > > > > > > > > >> delegation
>> > > > > > > > > >> >> > > > tokens?
>> > > > > > > > > >> >> > > > > > > That may have an impact on whether they
>> > > should
>> > > > be
>> > > > > > > > > directed
>> > > > > > > > > >> to the
>> > > > > > > > > >> >> > > > > > > controller.
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > Jun
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
>> > > > > brahmbhatt <
>> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > > Hi Jun,
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add
>> > acls
>> > > > on
>> > > > > > who
>> > > > > > > > can
>> > > > > > > > > >> request
>> > > > > > > > > >> >> > > > > > delegation
>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for
>> that
>> > > yet
>> > > > > but
>> > > > > > > > down
>> > > > > > > > > >> the line
>> > > > > > > > > >> >> > > > when
>> > > > > > > > > >> >> > > > > we
>> > > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs
>> it
>> > > will
>> > > > > be
>> > > > > > > > > >> necessary.
>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
>> > > > > > > used/distributed
>> > > > > > > > > >> over
>> > > > > > > > > >> >> > secure
>> > > > > > > > > >> >> > > > > > > channels.
>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end up
>> > > choosing
>> > > > > > > > > >> Invalidation will
>> > > > > > > > > >> >> > > be
>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
>> > > controller.
>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
>> somewhere
>> > > > that
>> > > > > > > > > >> invalidation
>> > > > > > > > > >> >> > will
>> > > > > > > > > >> >> > > > > > directly
>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is not
>> the
>> > > > > intent.
>> > > > > > > > > >> Invalidation
>> > > > > > > > > >> >> > > will
>> > > > > > > > > >> >> > > > > > either
>> > > > > > > > > >> >> > > > > > > > be request based or due to
>> expiration. No
>> > > > > direct
>> > > > > > > > > >> zookeeper
>> > > > > > > > > >> >> > > > > interaction
>> > > > > > > > > >> >> > > > > > > from
>> > > > > > > > > >> >> > > > > > > > any client.
>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
>> DelegationToken
>> > > > > without
>> > > > > > > the
>> > > > > > > > > >> hmac in
>> > > > > > > > > >> >> > the
>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
>> confusion.
>> > > The
>> > > > > sole
>> > > > > > > > > >> purpose of
>> > > > > > > > > >> >> > > > > zookeeper
>> > > > > > > > > >> >> > > > > > in
>> > > > > > > > > >> >> > > > > > > > this design is as distribution channel
>> > for
>> > > > > tokens
>> > > > > > > > > >> between all
>> > > > > > > > > >> >> > > > brokers
>> > > > > > > > > >> >> > > > > > > and a
>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that
>> were
>> > > > > > generated
>> > > > > > > by
>> > > > > > > > > >> making a
>> > > > > > > > > >> >> > > > > request
>> > > > > > > > > >> >> > > > > > > to a
>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on this
>> in
>> > > > second
>> > > > > > > > > >> paragraph). The
>> > > > > > > > > >> >> > > > token
>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
>> renewer,
>> > > > uuid
>> > > > > ,
>> > > > > > > > > >> expiration,
>> > > > > > > > > >> >> > > hmac)
>> > > > > > > > > >> >> > > > ,
>> > > > > > > > > >> >> > > > > > one
>> > > > > > > > > >> >> > > > > > > of
>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac
>> but
>> > > hmac
>> > > > it
>> > > > > > > self
>> > > > > > > > is
>> > > > > > > > > >> >> > derivable
>> > > > > > > > > >> >> > > > if
>> > > > > > > > > >> >> > > > > > you
>> > > > > > > > > >> >> > > > > > > > have all the other elements of the
>> token
>> > +
>> > > > > secret
>> > > > > > > key
>> > > > > > > > > to
>> > > > > > > > > >> >> > generate
>> > > > > > > > > >> >> > > > > hmac.
>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL
>> > > support
>> > > > we
>> > > > > > do
>> > > > > > > > not
>> > > > > > > > > >> want the
>> > > > > > > > > >> >> > > > > entire
>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
>> zookeeper
>> > > as
>> > > > > that
>> > > > > > > > will
>> > > > > > > > > >> be an
>> > > > > > > > > >> >> > > > insecure
>> > > > > > > > > >> >> > > > > > > wire
>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all
>> the
>> > > other
>> > > > > > > > elements
>> > > > > > > > > >> of a
>> > > > > > > > > >> >> > > > > delegation
>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these
>> elements
>> > and
>> > > > > > because
>> > > > > > > > > they
>> > > > > > > > > >> also
>> > > > > > > > > >> >> > > have
>> > > > > > > > > >> >> > > > > > access
>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to
>> > generate
>> > > > > hmac
>> > > > > > on
>> > > > > > > > > >> their end.
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is to
>> > avoid
>> > > > > > > zookeeper
>> > > > > > > > > >> >> > > altogether. A
>> > > > > > > > > >> >> > > > > > > Client
>> > > > > > > > > >> >> > > > > > > > will call broker with required
>> > information
>> > > > > > (owner,
>> > > > > > > > > >> renwer,
>> > > > > > > > > >> >> > > > > expiration)
>> > > > > > > > > >> >> > > > > > > and
>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker
>> > won't
>> > > > > store
>> > > > > > > this
>> > > > > > > > > in
>> > > > > > > > > >> >> > > zookeeper.
>> > > > > > > > > >> >> > > > > > From
>> > > > > > > > > >> >> > > > > > > > this point a client can contact any
>> > broker
>> > > > with
>> > > > > > all
>> > > > > > > > the
>> > > > > > > > > >> >> > > delegation
>> > > > > > > > > >> >> > > > > > token
>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac,
>> > > uuid)
>> > > > > the
>> > > > > > > > borker
>> > > > > > > > > >> will
>> > > > > > > > > >> >> > > > > regenerate
>> > > > > > > > > >> >> > > > > > > the
>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with
>> hmac
>> > > > > > presented
>> > > > > > > by
>> > > > > > > > > >> client ,
>> > > > > > > > > >> >> > > > broker
>> > > > > > > > > >> >> > > > > > > will
>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate.
>> Only
>> > > > > problem
>> > > > > > > with
>> > > > > > > > > >> this
>> > > > > > > > > >> >> > > approach
>> > > > > > > > > >> >> > > > > is
>> > > > > > > > > >> >> > > > > > if
>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
>> client
>> > > can
>> > > > > now
>> > > > > > > > > generate
>> > > > > > > > > >> >> > random
>> > > > > > > > > >> >> > > > > tokens
>> > > > > > > > > >> >> > > > > > > and
>> > > > > > > > > >> >> > > > > > > > they will still be able to
>> authenticate
>> > as
>> > > > any
>> > > > > > user
>> > > > > > > > > they
>> > > > > > > > > >> like.
>> > > > > > > > > >> >> > > with
>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only
>> tokens
>> > > > > acquired
>> > > > > > > via
>> > > > > > > > a
>> > > > > > > > > >> broker
>> > > > > > > > > >> >> > > > (using
>> > > > > > > > > >> >> > > > > > some
>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
>> token)
>> > > will
>> > > > > be
>> > > > > > > > > >> accepted. We
>> > > > > > > > > >> >> > > need
>> > > > > > > > > >> >> > > > to
>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more
>> sense
>> > and
>> > > > we
>> > > > > > can
>> > > > > > > go
>> > > > > > > > > >> over it
>> > > > > > > > > >> >> > in
>> > > > > > > > > >> >> > > > > > > tomorrow's
>> > > > > > > > > >> >> > > > > > > > meeting.
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to
>> me?
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > Thanks
>> > > > > > > > > >> >> > > > > > > > Parth
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun
>> > Rao <
>> > > > > > > > > >> jun@confluent.io>
>> > > > > > > > > >> >> > > > wrote:
>> > > > > > > > > >> >> > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful
>> for
>> > > > Kafka
>> > > > > > > > Connect
>> > > > > > > > > >> and
>> > > > > > > > > >> >> > Kafka
>> > > > > > > > > >> >> > > > > rest
>> > > > > > > > > >> >> > > > > > > > proxy
>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to
>> run a
>> > > > task
>> > > > > on
>> > > > > > > > > behalf
>> > > > > > > > > >> of a
>> > > > > > > > > >> >> > > > client.
>> > > > > > > > > >> >> > > > > > We
>> > > > > > > > > >> >> > > > > > > > will
>> > > > > > > > > >> >> > > > > > > > > likely need to change how those
>> > services
>> > > > use
>> > > > > > > Kafka
>> > > > > > > > > >> clients
>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
>> > shared
>> > > > > client
>> > > > > > > per
>> > > > > > > > > >> worker,
>> > > > > > > > > >> >> > we
>> > > > > > > > > >> >> > > > will
>> > > > > > > > > >> >> > > > > > > need
>> > > > > > > > > >> >> > > > > > > > a
>> > > > > > > > > >> >> > > > > > > > > client per user task since the
>> > > > authentication
>> > > > > > > > happens
>> > > > > > > > > >> at the
>> > > > > > > > > >> >> > > > > > connection
>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the
>> renewer
>> > > will
>> > > > be
>> > > > > > the
>> > > > > > > > > >> workers.
>> > > > > > > > > >> >> > So,
>> > > > > > > > > >> >> > > we
>> > > > > > > > > >> >> > > > > > > > probably
>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For
>> > > Kafka
>> > > > > rest
>> > > > > > > > > proxy,
>> > > > > > > > > >> the
>> > > > > > > > > >> >> > > > renewer
>> > > > > > > > > >> >> > > > > > can
>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of the
>> > > token.
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can
>> > > request
>> > > > > > > > delegation
>> > > > > > > > > >> tokens?
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
>> > > > > delegation
>> > > > > > > > tokens
>> > > > > > > > > >> in an
>> > > > > > > > > >> >> > > > > encrypted
>> > > > > > > > > >> >> > > > > > > > > channel?
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
>> > > > tokens,
>> > > > > > > every
>> > > > > > > > > >> broker?
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would
>> it
>> > be
>> > > > > > better
>> > > > > > > to
>> > > > > > > > > do
>> > > > > > > > > >> it in
>> > > > > > > > > >> >> > a
>> > > > > > > > > >> >> > > > > > request
>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in
>> the
>> > > wiki
>> > > > > > > > sometimes
>> > > > > > > > > >> refers
>> > > > > > > > > >> >> > to
>> > > > > > > > > >> >> > > > the
>> > > > > > > > > >> >> > > > > > end
>> > > > > > > > > >> >> > > > > > > > > client and some other times refers
>> to
>> > the
>> > > > > > client
>> > > > > > > > > using
>> > > > > > > > > >> the
>> > > > > > > > > >> >> > > > > delegation
>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
>> > distinguish
>> > > > > > between
>> > > > > > > > the
>> > > > > > > > > >> two.
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence
>> > > "Broker
>> > > > > > also
>> > > > > > > > > >> stores the
>> > > > > > > > > >> >> > > > > > > > DelegationToken
>> > > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper."
>> a
>> > bit
>> > > > > > more? I
>> > > > > > > > > >> thought the
>> > > > > > > > > >> >> > > > > > > delegation
>> > > > > > > > > >> >> > > > > > > > > token is the hmac.
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > Thanks,
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > Jun
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun
>> > Rao
>> > > <
>> > > > > > > > > >> jun@confluent.io>
>> > > > > > > > > >> >> > > > wrote:
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
>> > > > > > > > > >> >> > > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting
>> invite.
>> > We
>> > > > can
>> > > > > > > > discuss
>> > > > > > > > > >> this in
>> > > > > > > > > >> >> > > the
>> > > > > > > > > >> >> > > > > > > meeting
>> > > > > > > > > >> >> > > > > > > > > > tomorrow.
>> > > > > > > > > >> >> > > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > > Thanks,
>> > > > > > > > > >> >> > > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > > Jun
>> > > > > > > > > >> >> > > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM,
>> > > Harsha <
>> > > > > > > > > >> kafka@harsha.io>
>> > > > > > > > > >> >> > > > wrote:
>> > > > > > > > > >> >> > > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > >> Hi All,
>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
>> meeting
>> > > > > around
>> > > > > > > > this.
>> > > > > > > > > >> The KIP
>> > > > > > > > > >> >> > is
>> > > > > > > > > >> >> > > > up
>> > > > > > > > > >> >> > > > > > for
>> > > > > > > > > >> >> > > > > > > > > >> sometime and if there
>> are
>> > > any
>> > > > > > > > questions
>> > > > > > > > > >> lets
>> > > > > > > > > >> >> > > > quickly
>> > > > > > > > > >> >> > > > > > hash
>> > > > > > > > > >> >> > > > > > > > out
>> > > > > > > > > >> >> > > > > > > > > >> details.
>> > > > > > > > > >> >> > > > > > > > > >>
>> > > > > > > > > >> >> > > > > > > > > >> Thanks,
>> > > > > > > > > >> >> > > > > > > > > >> Harsha
>> > > > > > > > > >> >> > > > > > > > > >>
>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40
>> AM,
>> > > parth
>> > > > > > > > > brahmbhatt
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
>> > system
>> > > > uses
>> > > > > > so
>> > > > > > > no
>> > > > > > > > > >> good
>> > > > > > > > > >> >> > reason
>> > > > > > > > > >> >> > > > > > really.
>> > > > > > > > > >> >> > > > > > > > We
>> > > > > > > > > >> >> > > > > > > > > >> > could
>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
>> > newest
>> > > > > > > > recommended
>> > > > > > > > > >> standard
>> > > > > > > > > >> >> > > is.
>> > > > > > > > > >> >> > > > > > > > > >> >
>> > > > > > > > > >> >> > > > > > > > > >> > Thanks
>> > > > > > > > > >> >> > > > > > > > > >> > Parth
>> > > > > > > > > >> >> > > > > > > > > >> >
>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33
>> AM,
>> > > > Ismael
>> > > > > > > Juma <
>> > > > > > > > > >> >> > > > > ismael@juma.me.uk
>> > > > > > > > > >> >> > > > > > >
>> > > > > > > > > >> >> > > > > > > > > wrote:
>> > > > > > > > > >> >> > > > > > > > > >> >
>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
>> > started
>> > > > > > > reviewing
>> > > > > > > > > >> this and
>> > > > > > > > > >> >> > > may
>> > > > > > > > > >> >> > > > > have
>> > > > > > > > > >> >> > > > > > > > > >> additional
>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The
>> immediate
>> > > > > question
>> > > > > > > that
>> > > > > > > > > >> came to
>> > > > > > > > > >> >> > > mind
>> > > > > > > > > >> >> > > > is
>> > > > > > > > > >> >> > > > > > our
>> > > > > > > > > >> >> > > > > > > > > >> choice of
>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's
>> > > marked
>> > > > > as
>> > > > > > > > > >> OBSOLETE in
>> > > > > > > > > >> >> > the
>> > > > > > > > > >> >> > > > IANA
>> > > > > > > > > >> >> > > > > > > > > Registry
>> > > > > > > > > >> >> > > > > > > > > >> of
>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
>> original
>> > > RFC
>> > > > > > > (2831)
>> > > > > > > > > has
>> > > > > > > > > >> been
>> > > > > > > > > >> >> > > moved
>> > > > > > > > > >> >> > > > > to
>> > > > > > > > > >> >> > > > > > > > > Historic
>> > > > > > > > > >> >> > > > > > > > > >> > > status:
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/
>> > > rfc6331
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > >
>> > > > > > > > > >> >> > >
>> > > > > > > > > >>
>> > > > > > >
>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-
>> mechanisms.xhtml
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind
>> > that
>> > > > > > choice?
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29
>> > PM,
>> > > > Gwen
>> > > > > > > > > Shapira <
>> > > > > > > > > >> >> > > > > > > gwen@confluent.io
>> > > > > > > > > >> >> > > > > > > > >
>> > > > > > > > > >> >> > > > > > > > > >> wrote:
>> > > > > > > > > >> >> > > > > > > > > >> > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize
>> that
>> > > even
>> > > > > > though
>> > > > > > > > > >> delegation
>> > > > > > > > > >> >> > > > tokens
>> > > > > > > > > >> >> > > > > > > are a
>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very
>> > > strongly
>> > > > > > about
>> > > > > > > > not
>> > > > > > > > > >> adding
>> > > > > > > > > >> >> > > > > > dependency
>> > > > > > > > > >> >> > > > > > > > on
>> > > > > > > > > >> >> > > > > > > > > >> Hadoop
>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
>> delegation
>> > > > > tokens
>> > > > > > > for
>> > > > > > > > > >> Kafka. The
>> > > > > > > > > >> >> > > KIP
>> > > > > > > > > >> >> > > > > > > doesn't
>> > > > > > > > > >> >> > > > > > > > > >> imply
>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if
>> you
>> > > can
>> > > > > > > > clarify...
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the
>> KIP
>> > so
>> > > > no
>> > > > > > one
>> > > > > > > > will
>> > > > > > > > > >> read
>> > > > > > > > > >> >> > the
>> > > > > > > > > >> >> > > > KIP
>> > > > > > > > > >> >> > > > > > and
>> > > > > > > > > >> >> > > > > > > > > panic
>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
>> > > > > release...
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation
>> > token
>> > > at
>> > > > > any
>> > > > > > > > time
>> > > > > > > > > >> after
>> > > > > > > > > >> >> > > > > > > > authenticating?
>> > > > > > > > > >> >> > > > > > > > > >> only
>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
>> > > > authenticated
>> > > > > > you
>> > > > > > > > can
>> > > > > > > > > >> get
>> > > > > > > > > >> >> > > > delegation
>> > > > > > > > > >> >> > > > > > > > tokens.
>> > > > > > > > > >> >> > > > > > > > > >> We
>> > > > > > > > > >> >> > > > > > > > > >> > > need
>> > > > > > > > > >> >> > > > > > > > > >> > > > to
>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
>> > > > authenticated
>> > > > > > > using
>> > > > > > > > > >> delegation
>> > > > > > > > > >> >> > > > > token,
>> > > > > > > > > >> >> > > > > > > can
>> > > > > > > > > >> >> > > > > > > > > also
>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or
>> > not.
>> > > > > Also
>> > > > > > > > there
>> > > > > > > > > >> is the
>> > > > > > > > > >> >> > > > > question
>> > > > > > > > > >> >> > > > > > of
>> > > > > > > > > >> >> > > > > > > > do
>> > > > > > > > > >> >> > > > > > > > > we
>> > > > > > > > > >> >> > > > > > > > > >> > > allow
>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
>> delegation
>> > > > token
>> > > > > > or
>> > > > > > > we
>> > > > > > > > > >> want
>> > > > > > > > > >> >> > > specific
>> > > > > > > > > >> >> > > > > > ACLs
>> > > > > > > > > >> >> > > > > > > (I
>> > > > > > > > > >> >> > > > > > > > > >> think
>> > > > > > > > > >> >> > > > > > > > > >> > > its
>> > > > > > > > > >> >> > > > > > > > > >> > > > an
>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
>> > > overkill.
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
>> > > > options:
>> > > > > > > > Either
>> > > > > > > > > >> require
>> > > > > > > > > >> >> > > > > Kerberos
>> > > > > > > > > >> >> > > > > > > > auth
>> > > > > > > > > >> >> > > > > > > > > >> for
>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require
>> non-owners
>> > to
>> > > > > > renew.
>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
>> > simpler
>> > > > (it
>> > > > > > > > > basically
>> > > > > > > > > >> >> > require
>> > > > > > > > > >> >> > > a
>> > > > > > > > > >> >> > > > > "job
>> > > > > > > > > >> >> > > > > > > > > master"
>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for
>> the
>> > > > > > renewal,
>> > > > > > > it
>> > > > > > > > > >> will have
>> > > > > > > > > >> >> > > its
>> > > > > > > > > >> >> > > > > own
>> > > > > > > > > >> >> > > > > > > > > >> identity
>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is
>> the
>> > > > > correct
>> > > > > > > > > design
>> > > > > > > > > >> >> > pattern
>> > > > > > > > > >> >> > > > > > anyway.
>> > > > > > > > > >> >> > > > > > > > For
>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
>> > > > > coordinate
>> > > > > > > > > >> renewals?),
>> > > > > > > > > >> >> > but
>> > > > > > > > > >> >> > > > it
>> > > > > > > > > >> >> > > > > is
>> > > > > > > > > >> >> > > > > > > > hard
>> > > > > > > > > >> >> > > > > > > > > to
>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without
>> > > looking
>> > > > at
>> > > > > > the
>> > > > > > > > > code
>> > > > > > > > > >> >> > changes
>> > > > > > > > > >> >> > > > > > > required.
>> > > > > > > > > >> >> > > > > > > > If
>> > > > > > > > > >> >> > > > > > > > > >> you
>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
>> > "require
>> > > > > > > Kerberos"
>> > > > > > > > > >> will look
>> > > > > > > > > >> >> > > in
>> > > > > > > > > >> >> > > > > > Kafka
>> > > > > > > > > >> >> > > > > > > > > code,
>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a
>> look.
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is
>> that
>> > > > tokens
>> > > > > > will
>> > > > > > > > > >> propagate
>> > > > > > > > > >> >> > via
>> > > > > > > > > >> >> > > > ZK
>> > > > > > > > > >> >> > > > > > but
>> > > > > > > > > >> >> > > > > > > > > >> without
>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
>> > > > > UpdateMetadata
>> > > > > > > > > >> protocol,
>> > > > > > > > > >> >> > > > correct?
>> > > > > > > > > >> >> > > > > > > > Clients
>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on
>> > SASL
>> > > > auth
>> > > > > > > > failure
>> > > > > > > > > >> (IIRC),
>> > > > > > > > > >> >> > > but
>> > > > > > > > > >> >> > > > > > since
>> > > > > > > > > >> >> > > > > > > > the
>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between
>> > > brokers
>> > > > > > > asynch,
>> > > > > > > > > we
>> > > > > > > > > >> will
>> > > > > > > > > >> >> > > need
>> > > > > > > > > >> >> > > > to
>> > > > > > > > > >> >> > > > > > > > retry a
>> > > > > > > > > >> >> > > > > > > > > >> bit
>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing
>> > auth
>> > > > due
>> > > > > to
>> > > > > > > > > timing
>> > > > > > > > > >> >> > issues.
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
>> > > alternatives
>> > > > > > right
>> > > > > > > > > now.
>> > > > > > > > > >> The
>> > > > > > > > > >> >> > > > current
>> > > > > > > > > >> >> > > > > > > > > documented
>> > > > > > > > > >> >> > > > > > > > > >> > > > approach
>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it
>> > does
>> > > > not
>> > > > > > > > require
>> > > > > > > > > >> any
>> > > > > > > > > >> >> > > changes
>> > > > > > > > > >> >> > > > > to
>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
>> > > > approach
>> > > > > > can
>> > > > > > > > > remove
>> > > > > > > > > >> >> > > zookeeper
>> > > > > > > > > >> >> > > > > > > > > dependency
>> > > > > > > > > >> >> > > > > > > > > >> as
>> > > > > > > > > >> >> > > > > > > > > >> > > well
>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that
>> in
>> > KIP
>> > > > > > > > discussion
>> > > > > > > > > >> call.*
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting.
>> Do
>> > you
>> > > > > want
>> > > > > > to
>> > > > > > > > > ping
>> > > > > > > > > >> Jun to
>> > > > > > > > > >> >> > > > > > arrange a
>> > > > > > > > > >> >> > > > > > > > > call?
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
>> > suggestion
>> > > of
>> > > > > > > having
>> > > > > > > > > >> just the
>> > > > > > > > > >> >> > > > > > controller
>> > > > > > > > > >> >> > > > > > > > > issue
>> > > > > > > > > >> >> > > > > > > > > >> the
>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to
>> avoid
>> > > > > syncing
>> > > > > > a
>> > > > > > > > > shared
>> > > > > > > > > >> >> > secret.
>> > > > > > > > > >> >> > > > Not
>> > > > > > > > > >> >> > > > > > > sure
>> > > > > > > > > >> >> > > > > > > > if
>> > > > > > > > > >> >> > > > > > > > > >> we
>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
>> > > discussion
>> > > > > here
>> > > > > > > or
>> > > > > > > > on
>> > > > > > > > > >> the
>> > > > > > > > > >> >> > > wiki. I
>> > > > > > > > > >> >> > > > > > think
>> > > > > > > > > >> >> > > > > > > > > that
>> > > > > > > > > >> >> > > > > > > > > >> we
>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem
>> of
>> > > > "token
>> > > > > > > > > >> distribution"
>> > > > > > > > > >> >> > > from
>> > > > > > > > > >> >> > > > > > > "shared
>> > > > > > > > > >> >> > > > > > > > > >> secret
>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
>> > > > > controller
>> > > > > > > as
>> > > > > > > > > the
>> > > > > > > > > >> only
>> > > > > > > > > >> >> > > token
>> > > > > > > > > >> >> > > > > > > > generator
>> > > > > > > > > >> >> > > > > > > > > >> to
>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue,
>> > while
>> > > > > still
>> > > > > > > > using
>> > > > > > > > > >> ZK async
>> > > > > > > > > >> >> > > to
>> > > > > > > > > >> >> > > > > > > > distribute
>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
>> > previous
>> > > > > Email
>> > > > > > I
>> > > > > > > am
>> > > > > > > > > >> fine with
>> > > > > > > > > >> >> > > > that
>> > > > > > > > > >> >> > > > > > > > approach
>> > > > > > > > > >> >> > > > > > > > > >> as
>> > > > > > > > > >> >> > > > > > > > > >> > > long
>> > > > > > > > > >> >> > > > > > > > > >> > > > as
>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
>> > > > complexity
>> > > > > > of
>> > > > > > > > > >> >> > > adding/updating
>> > > > > > > > > >> >> > > > > APIS
>> > > > > > > > > >> >> > > > > > > > adds
>> > > > > > > > > >> >> > > > > > > > > >> enough
>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with
>> > the
>> > > > > > > controller
>> > > > > > > > > >> approach
>> > > > > > > > > >> >> > is
>> > > > > > > > > >> >> > > > > > secret
>> > > > > > > > > >> >> > > > > > > > > >> rotation
>> > > > > > > > > >> >> > > > > > > > > >> > > can
>> > > > > > > > > >> >> > > > > > > > > >> > > > be
>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and
>> would
>> > > not
>> > > > > > > require
>> > > > > > > > > >> >> > > deployment. *
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
>> > > > complexity
>> > > > > > (or
>> > > > > > > > > point
>> > > > > > > > > >> me to
>> > > > > > > > > >> >> > > the
>> > > > > > > > > >> >> > > > > > email
>> > > > > > > > > >> >> > > > > > > I
>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
>> > > > required?
>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell,
>> clients
>> > > can
>> > > > > > > already
>> > > > > > > > > >> find the
>> > > > > > > > > >> >> > > > > > controller
>> > > > > > > > > >> >> > > > > > > > from
>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
>> > > concerned
>> > > > > > about
>> > > > > > > > > >> controller
>> > > > > > > > > >> >> > > > load.
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea
>> of
>> > > > forcing
>> > > > > > > > > kerberos
>> > > > > > > > > >> auth
>> > > > > > > > > >> >> > for
>> > > > > > > > > >> >> > > > > > > renwal, I
>> > > > > > > > > >> >> > > > > > > > > >> think
>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport
>> layer
>> > > the
>> > > > > the
>> > > > > > > > > request
>> > > > > > > > > >> >> > content
>> > > > > > > > > >> >> > > > in
>> > > > > > > > > >> >> > > > > a
>> > > > > > > > > >> >> > > > > > > > pretty
>> > > > > > > > > >> >> > > > > > > > > >> ugly
>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
>> > renewer
>> > > to
>> > > > > > > > non-owner
>> > > > > > > > > >> is
>> > > > > > > > > >> >> > > better.
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a
>> necessary
>> > > > evil.
>> > > > > > > While
>> > > > > > > > > >> this will
>> > > > > > > > > >> >> > > make
>> > > > > > > > > >> >> > > > > the
>> > > > > > > > > >> >> > > > > > > > kafka
>> > > > > > > > > >> >> > > > > > > > > >> code
>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
>> > > > forcing
>> > > > > > > > renewer
>> > > > > > > > > >> to
>> > > > > > > > > >> >> > > > non-owner
>> > > > > > > > > >> >> > > > > > > pushes
>> > > > > > > > > >> >> > > > > > > > > >> the code
>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and
>> makes
>> > > it
>> > > > > even
>> > > > > > > > > harder
>> > > > > > > > > >> to
>> > > > > > > > > >> >> > > > > > integrate. *
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I
>> don't
>> > > think
>> > > > > the
>> > > > > > > > > >> "renewal by
>> > > > > > > > > >> >> > > > other"
>> > > > > > > > > >> >> > > > > > > > approach
>> > > > > > > > > >> >> > > > > > > > > >> is
>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients
>> we
>> > > > expect
>> > > > > to
>> > > > > > > use
>> > > > > > > > > >> >> > delegation
>> > > > > > > > > >> >> > > > > tokens
>> > > > > > > > > >> >> > > > > > > > since
>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an
>> app-master
>> > of
>> > > > some
>> > > > > > > sort
>> > > > > > > > > who
>> > > > > > > > > >> >> > > requested
>> > > > > > > > > >> >> > > > > the
>> > > > > > > > > >> >> > > > > > > > token
>> > > > > > > > > >> >> > > > > > > > > to
>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with.
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my
>> question
>> > > on
>> > > > > how
>> > > > > > > > > multiple
>> > > > > > > > > >> >> > > > identities
>> > > > > > > > > >> >> > > > > > will
>> > > > > > > > > >> >> > > > > > > > be
>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super
>> clear
>> > to
>> > > > me -
>> > > > > > > > AFAIK,
>> > > > > > > > > >> we don't
>> > > > > > > > > >> >> > > > > > > > authenticate
>> > > > > > > > > >> >> > > > > > > > > >> each
>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
>> > > > > > connections.
>> > > > > > > > > >> >> > > > > > > > > >> > > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
>> > connections,
>> > > > and
>> > > > > > only
>> > > > > > > > > when
>> > > > > > > > > >> they
>> > > > > > > > > >> >> > are
>> > > > > > > > > >> >> > > > > being
>> > > > > > > > > >> >> > > > > > > > > >> established.
>> > > > > > > > > >> >> > > > > > > > > >> > > > Let
>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as
>> a
>> > > > > question,
>> > > > > > > in
>> > > > > > > > > >> absence of
>> > > > > > > > > >> >> > > > > > > delegation
>> > > > > > > > > >> >> > > > > > > > > >> tokens if
>> > > > > > > > > >> >> > > > > > > > > >> > > > we
>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use
>> case
>> > > > using
>> > > > > > user
>> > > > > > > > > >> TGT's how
>> > > > > > > > > >> >> > > would
>> > > > > > > > > >> >> > > > > we
>> > > > > > > > > >> >> > > > > > do
>> > > > > > > > > >> >> > > > > > > > it?
>> > > > > > > > > >> >> > > > > > > > > >> My
>> > > > > > > > > >> >> > > > > > > > > >> > > point
>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no
>> different
>> > > > with
>> > > > > > > > > delegation
>> > > > > > > > > >> >> > tokens.
>> > > > > > > > > >> >> > > > The
>> > > > > > > > > >> >> > > > > > use
>> > > > > > > > > >> >> > > > > > > > > case
>> > > > > > > > > >> >> > > > > > > > > >> you
>> > > > > > > > > >> >> > > > > > > > > >> > > are
>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more
>> like
>> > > > > > > > impersonation.*
>> > > > > > > > > >> >> > > > > > > > > >> > > >
>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of
>> > the
>> > > > > things
>> > > > > > > > that
>> > > > > > > > > >> >> > del
>>
>
>
>
> --
>
> Regards,
> Ashish
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ashish Singh <as...@cloudera.com>.
Thanks Harsha!
Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not
actually make a call on when we should have next KIP call. As there are a
few outstanding KIPs that could not be discussed this week, can we have a
KIP hangout call next week?
On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Ashish,
> Yes we are working on it. Lets discuss in the next KIP meeting.
> I'll join.
> -Harsha
>
> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com> wrote:
>
> > Hello Harsha,
> >
> > Are you still working on this? Wondering if we can discuss this in next
> KIP
> > meeting, if you can join.
> >
> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> > > Hi Grant,
> > > We are working on it. Will add the details to KIP about the
> > > request protocol.
> > >
> > > Thanks,
> > > Harsha
> > >
> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com>
> wrote:
> > >
> > > > Hi Parth,
> > > >
> > > > Are you still working on this? If you need any help please don't
> > hesitate
> > > > to ask.
> > > >
> > > > Thanks,
> > > > Grant
> > > >
> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Parth,
> > > > >
> > > > > Thanks for the reply.
> > > > >
> > > > > It makes sense to only allow the renewal by users that
> authenticated
> > > > using
> > > > > *non* delegation token mechanism. Then, should we make the renewal
> a
> > > > list?
> > > > > For example, in the case of rest proxy, it will be useful for every
> > > > > instance of rest proxy to be able to renew the tokens.
> > > > >
> > > > > It would be clearer if we can document the request protocol like
> > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> > > Commandlineandcentralizedadministrativeoperations-
> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> > > > > .
> > > > >
> > > > > It would also be useful to document the client APIs.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am suggesting that we will only allow the renewal by users that
> > > > > > authenticated using *non* delegation token mechanism. For
> example,
> > If
> > > > > user
> > > > > > Alice authenticated using kerberos and requested delegation
> tokens,
> > > > only
> > > > > > user Alice authenticated via non delegation token mechanism can
> > > renew.
> > > > > > Clients that have access to delegation tokens can not issue
> > renewal
> > > > > > request for renewing their own token and this is primarily
> > important
> > > to
> > > > > > reduce the time window for which a compromised token will be
> valid.
> > > > > >
> > > > > > To clarify, Yes any authenticated user can request delegation
> > tokens
> > > > but
> > > > > > even here I would recommend to avoid creating a chain where a
> > client
> > > > > > authenticated via delegation token request for more delegation
> > > tokens.
> > > > > > Basically anyone can request delegation token, as long as they
> > > > > authenticate
> > > > > > via a non delegation token mechanism.
> > > > > >
> > > > > > Aren't classes listed here
> > > > > > <
> > > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 48+Delegation+token+support+for+Kafka#KIP-
> 48DelegationtokensupportforKaf
> > > ka-PublicInterfaces
> > > > > > >
> > > > > > sufficient?
> > > > > >
> > > > > > Thanks
> > > > > > Parth
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Parth,
> > > > > > >
> > > > > > > Thanks for the reply. A couple of comments inline below.
> > > > > > >
> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > >
> > > > > > > > 1. Who / how are tokens renewed? By original requester only?
> or
> > > > using
> > > > > > > > Kerberos
> > > > > > > > auth only?
> > > > > > > > My recommendation is to do this only using Kerberos auth and
> > only
> > > > > threw
> > > > > > > the
> > > > > > > > renewer specified during the acquisition request.
> > > > > > > >
> > > > > > > >
> > > > > > > Hmm, not sure that I follow this. Are you saying that any
> client
> > > > > > > authenticated with the delegation token can renew, i.e. there
> is
> > no
> > > > > > renewer
> > > > > > > needed?
> > > > > > >
> > > > > > > Also, just to be clear, any authenticated client (either
> through
> > > SASL
> > > > > or
> > > > > > > SSL) can request a delegation token for the authenticated user,
> > > > right?
> > > > > > >
> > > > > > >
> > > > > > > > 2. Are tokens stored on each broker or in ZK?
> > > > > > > > My recommendation is still to store in ZK or not store them
> at
> > > all.
> > > > > The
> > > > > > > > whole controller based distribution is too much overhead with
> > not
> > > > > much
> > > > > > to
> > > > > > > > achieve.
> > > > > > > >
> > > > > > > > 3. How are tokens invalidated / expired?
> > > > > > > > Either by expiration time out or through an explicit request
> to
> > > > > > > invalidate.
> > > > > > > >
> > > > > > > > 4. Which encryption algorithm is used?
> > > > > > > > SCRAM
> > > > > > > >
> > > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP
> but
> > > was
> > > > > > > > discussed
> > > > > > > > in this thread)?
> > > > > > > > There is no imperonation proposal. I tried and explained how
> > its
> > > a
> > > > > > > > different problem and why its not really necessary to discuss
> > > that
> > > > as
> > > > > > > part
> > > > > > > > of this KIP. This KIP will not support any impersonation, it
> > > will
> > > > > just
> > > > > > > be
> > > > > > > > another way to authenticate.
> > > > > > > >
> > > > > > > > 6. Do we need new ACLs, if so - for what actions?
> > > > > > > > We do not need new ACLs.
> > > > > > > >
> > > > > > > >
> > > > > > > Could we document the format of the new request/response and
> > their
> > > > > > > associated Resource and Operation for ACL?
> > > > > > >
> > > > > > >
> > > > > > > > 7. How would the delegation token be configured in the
> client?
> > > > > > > > Should be through config. I wasn't planning on supporting
> JAAS
> > > for
> > > > > > > tokens.
> > > > > > > > I don't believe hadoop does this either.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Parth
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > > > >
> > > > > > > > > Harsha,
> > > > > > > > >
> > > > > > > > > Another question.
> > > > > > > > >
> > > > > > > > > 9. How would the delegation token be configured in the
> > client?
> > > > The
> > > > > > > > standard
> > > > > > > > > way is to do this through JAAS. However, we will need to
> > think
> > > > > > through
> > > > > > > if
> > > > > > > > > this is convenient in a shared environment. For example,
> > when a
> > > > new
> > > > > > > task
> > > > > > > > is
> > > > > > > > > added to a Storm worker node, do we need to dynamically
> add a
> > > new
> > > > > > > section
> > > > > > > > > in the JAAS file? It may be more convenient if we can pass
> in
> > > the
> > > > > > token
> > > > > > > > > through the config directly w/o going through JAAS.
> > > > > > > > >
> > > > > > > > > Are you or Parth still actively working on this KIP?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <jun@confluent.io
> >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Just to add on that list.
> > > > > > > > > >
> > > > > > > > > > 2. It would be good to document the format of the data
> > stored
> > > > in
> > > > > > ZK.
> > > > > > > > > > 7. Earlier, there was a discussion on whether the tokens
> > > should
> > > > > be
> > > > > > > > > > propagated through ZK like config/acl/quota, or through
> the
> > > > > > > controller.
> > > > > > > > > > Currently, the controller is only designed for
> propagating
> > > > topic
> > > > > > > > > metadata,
> > > > > > > > > > but not other data.
> > > > > > > > > > 8. Should we use SCRAM to send the token instead of
> > > DIGEST-MD5
> > > > > > since
> > > > > > > > it's
> > > > > > > > > > deprecated?
> > > > > > > > > >
> > > > > > > > > > Also, the images in the wiki seem broken.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> > > > > gwen@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> From what I can see, remaining questions are:
> > > > > > > > > >>
> > > > > > > > > >> 1. Who / how are tokens renewed? By original requester
> > only?
> > > > or
> > > > > > > using
> > > > > > > > > >> Kerberos auth only?
> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > > > > > > >> 3. How are tokens invalidated / expired?
> > > > > > > > > >> 4. Which encryption algorithm is used?
> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the
> > KIP
> > > > but
> > > > > > was
> > > > > > > > > >> discussed in this thread)?
> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > > > > > > >>
> > > > > > > > > >> Gwen
> > > > > > > > > >>
> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <kafka@harsha.io
> >
> > > > wrote:
> > > > > > > > > >> > Jun & Ismael,
> > > > > > > > > >> > Unfortunately I couldn't
> attend
> > > the
> > > > > KIP
> > > > > > > > > meeting
> > > > > > > > > >> > when delegation tokens
> > discussed.
> > > > > > > > Appreciate
> > > > > > > > > if
> > > > > > > > > >> > you can update the thread if
> > you
> > > > have
> > > > > > any
> > > > > > > > > >> > further questions.
> > > > > > > > > >> > Thanks,
> > > > > > > > > >> > Harsha
> > > > > > > > > >> >
> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > > > > > > >> >> It seems that the links to images in the KIP are
> > broken.
> > > > > > > > > >> >>
> > > > > > > > > >> >> Liquan
> > > > > > > > > >> >>
> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > > >> >>
> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > > > > > > >> >> > In the current proposal we only allow a user to get
> > > > > > delegation
> > > > > > > > > token
> > > > > > > > > >> for
> > > > > > > > > >> >> > the identity that it authenticated as using another
> > > > > > mechanism,
> > > > > > > > i.e.
> > > > > > > > > >> A user
> > > > > > > > > >> >> > that authenticate using a keytab for principal
> > > > > > > user1@EXAMPLE.COM
> > > > > > > > > >> will get
> > > > > > > > > >> >> > delegation tokens for that user only. In future I
> > think
> > > > we
> > > > > > will
> > > > > > > > > have
> > > > > > > > > >> to
> > > > > > > > > >> >> > extend support such that we allow some set of
> users (
> > > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> > storm-nimbus@EXAMPLE.COM)
> > > > to
> > > > > > > > acquire
> > > > > > > > > >> >> > delegation tokens on behalf of other users whose
> > > identity
> > > > > > they
> > > > > > > > have
> > > > > > > > > >> >> > verified independently. Kafka brokers will have
> ACLs
> > > to
> > > > > > > control
> > > > > > > > > >> which
> > > > > > > > > >> >> > users are allowed to impersonate other users and
> get
> > > > tokens
> > > > > > on
> > > > > > > > > >> behalf of
> > > > > > > > > >> >> > them. Overall Impersonation is a whole different
> > > problem
> > > > in
> > > > > > my
> > > > > > > > > >> opinion and
> > > > > > > > > >> >> > I think we can tackle it in separate KIP.
> > > > > > > > > >> >> >
> > > > > > > > > >> >> > 111. What's the typical rate of getting and
> renewing
> > > > > > delegation
> > > > > > > > > >> tokens?
> > > > > > > > > >> >> > Typically this should be very very low, 1 request
> per
> > > > > minute
> > > > > > > is a
> > > > > > > > > >> >> > relatively high estimate. However it depends on the
> > > token
> > > > > > > > > >> expiration. I am
> > > > > > > > > >> >> > less worried about the extra load it puts on
> > controller
> > > > vs
> > > > > > the
> > > > > > > > > added
> > > > > > > > > >> >> > complexity and the value it offers.
> > > > > > > > > >> >> >
> > > > > > > > > >> >> > Thanks
> > > > > > > > > >> >> > Parth
> > > > > > > > > >> >> >
> > > > > > > > > >> >> >
> > > > > > > > > >> >> >
> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > > > > > > ismael@juma.me.uk>
> > > > > > > > > >> wrote:
> > > > > > > > > >> >> >
> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a
> separate
> > > KIP
> > > > > as
> > > > > > it
> > > > > > > > > will
> > > > > > > > > >> >> > > introduce user visible changes. We could also
> > update
> > > > > KIP-48
> > > > > > > to
> > > > > > > > > >> have this
> > > > > > > > > >> >> > > information, but it seems cleaner to do it
> > > separately.
> > > > We
> > > > > > can
> > > > > > > > > >> discuss
> > > > > > > > > >> >> > that
> > > > > > > > > >> >> > > in the KIP call today.
> > > > > > > > > >> >> > >
> > > > > > > > > >> >> > > Ismael
> > > > > > > > > >> >> > >
> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > > > > > > >> >> > >
> > > > > > > > > >> >> > > > Ismael,
> > > > > > > > > >> >> > > >
> > > > > > > > > >> >> > > > I have created a JIRA (
> > > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would
> that
> > > need
> > > > > > > another
> > > > > > > > > >> KIP? If
> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just
> be
> > a
> > > > JIRA
> > > > > > > that
> > > > > > > > > gets
> > > > > > > > > >> >> > > reviewed
> > > > > > > > > >> >> > > > when the PR is ready?
> > > > > > > > > >> >> > > >
> > > > > > > > > >> >> > > > Thank you,
> > > > > > > > > >> >> > > >
> > > > > > > > > >> >> > > > Rajini
> > > > > > > > > >> >> > > >
> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > > > > > > ismael@juma.me.uk>
> > > > > > > > > >> >> > wrote:
> > > > > > > > > >> >> > > >
> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
> > candidate.
> > > > > > > > > >> >> > > > >
> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as a
> SASL
> > > > > > mechanism
> > > > > > > > > that
> > > > > > > > > >> might
> > > > > > > > > >> >> > be
> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to
> > check
> > > > it
> > > > > in
> > > > > > > > more
> > > > > > > > > >> detail.
> > > > > > > > > >> >> > > Good
> > > > > > > > > >> >> > > > > to know that you are willing to contribute an
> > > > > > > > implementation.
> > > > > > > > > >> Maybe
> > > > > > > > > >> >> > we
> > > > > > > > > >> >> > > > > should file a separate JIRA for this?
> > > > > > > > > >> >> > > > >
> > > > > > > > > >> >> > > > > Ismael
> > > > > > > > > >> >> > > > >
> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
> > Sivaram <
> > > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > > > > >> >> > > > >
> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> > Authentication
> > > > > > > > Mechanism)
> > > > > > > > > >> is a
> > > > > > > > > >> >> > > better
> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't
> come
> > > > with a
> > > > > > > > > built-in
> > > > > > > > > >> SCRAM
> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be
> happy
> > > to
> > > > > add
> > > > > > > > > support
> > > > > > > > > >> in
> > > > > > > > > >> >> > Kafka
> > > > > > > > > >> >> > > > > since
> > > > > > > > > >> >> > > > > > it would be a useful mechanism to support
> > > anyway.
> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
> > describes
> > > > the
> > > > > > > > protocol
> > > > > > > > > >> for
> > > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > > > > > >> >> > > > > >
> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > > > > > > jun@confluent.io
> > > > > > > > > >
> > > > > > > > > >> wrote:
> > > > > > > > > >> >> > > > > >
> > > > > > > > > >> >> > > > > > > Parth,
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of
> > more
> > > > > > > questions.
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting
> and
> > > > > > renewing
> > > > > > > > > >> delegation
> > > > > > > > > >> >> > > > tokens?
> > > > > > > > > >> >> > > > > > > That may have an impact on whether they
> > > should
> > > > be
> > > > > > > > > directed
> > > > > > > > > >> to the
> > > > > > > > > >> >> > > > > > > controller.
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > Jun
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
> > > > > brahmbhatt <
> > > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > > Hi Jun,
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add
> > acls
> > > > on
> > > > > > who
> > > > > > > > can
> > > > > > > > > >> request
> > > > > > > > > >> >> > > > > > delegation
> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for
> that
> > > yet
> > > > > but
> > > > > > > > down
> > > > > > > > > >> the line
> > > > > > > > > >> >> > > > when
> > > > > > > > > >> >> > > > > we
> > > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs
> it
> > > will
> > > > > be
> > > > > > > > > >> necessary.
> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > > > > > > used/distributed
> > > > > > > > > >> over
> > > > > > > > > >> >> > secure
> > > > > > > > > >> >> > > > > > > channels.
> > > > > > > > > >> >> > > > > > > > * Depending on what design we end up
> > > choosing
> > > > > > > > > >> Invalidation will
> > > > > > > > > >> >> > > be
> > > > > > > > > >> >> > > > > > > > responsibility of every broker or
> > > controller.
> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented
> somewhere
> > > > that
> > > > > > > > > >> invalidation
> > > > > > > > > >> >> > will
> > > > > > > > > >> >> > > > > > directly
> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is not
> the
> > > > > intent.
> > > > > > > > > >> Invalidation
> > > > > > > > > >> >> > > will
> > > > > > > > > >> >> > > > > > either
> > > > > > > > > >> >> > > > > > > > be request based or due to expiration.
> No
> > > > > direct
> > > > > > > > > >> zookeeper
> > > > > > > > > >> >> > > > > interaction
> > > > > > > > > >> >> > > > > > > from
> > > > > > > > > >> >> > > > > > > > any client.
> > > > > > > > > >> >> > > > > > > > * "Broker also stores the
> DelegationToken
> > > > > without
> > > > > > > the
> > > > > > > > > >> hmac in
> > > > > > > > > >> >> > the
> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the
> confusion.
> > > The
> > > > > sole
> > > > > > > > > >> purpose of
> > > > > > > > > >> >> > > > > zookeeper
> > > > > > > > > >> >> > > > > > in
> > > > > > > > > >> >> > > > > > > > this design is as distribution channel
> > for
> > > > > tokens
> > > > > > > > > >> between all
> > > > > > > > > >> >> > > > brokers
> > > > > > > > > >> >> > > > > > > and a
> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that
> were
> > > > > > generated
> > > > > > > by
> > > > > > > > > >> making a
> > > > > > > > > >> >> > > > > request
> > > > > > > > > >> >> > > > > > > to a
> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on this
> in
> > > > second
> > > > > > > > > >> paragraph). The
> > > > > > > > > >> >> > > > token
> > > > > > > > > >> >> > > > > > > > consists of few elements (owner,
> renewer,
> > > > uuid
> > > > > ,
> > > > > > > > > >> expiration,
> > > > > > > > > >> >> > > hmac)
> > > > > > > > > >> >> > > > ,
> > > > > > > > > >> >> > > > > > one
> > > > > > > > > >> >> > > > > > > of
> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac but
> > > hmac
> > > > it
> > > > > > > self
> > > > > > > > is
> > > > > > > > > >> >> > derivable
> > > > > > > > > >> >> > > > if
> > > > > > > > > >> >> > > > > > you
> > > > > > > > > >> >> > > > > > > > have all the other elements of the
> token
> > +
> > > > > secret
> > > > > > > key
> > > > > > > > > to
> > > > > > > > > >> >> > generate
> > > > > > > > > >> >> > > > > hmac.
> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL
> > > support
> > > > we
> > > > > > do
> > > > > > > > not
> > > > > > > > > >> want the
> > > > > > > > > >> >> > > > > entire
> > > > > > > > > >> >> > > > > > > > token to be wire transferred to
> zookeeper
> > > as
> > > > > that
> > > > > > > > will
> > > > > > > > > >> be an
> > > > > > > > > >> >> > > > insecure
> > > > > > > > > >> >> > > > > > > wire
> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all the
> > > other
> > > > > > > > elements
> > > > > > > > > >> of a
> > > > > > > > > >> >> > > > > delegation
> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these elements
> > and
> > > > > > because
> > > > > > > > > they
> > > > > > > > > >> also
> > > > > > > > > >> >> > > have
> > > > > > > > > >> >> > > > > > access
> > > > > > > > > >> >> > > > > > > > to secret key they will be able to
> > generate
> > > > > hmac
> > > > > > on
> > > > > > > > > >> their end.
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is to
> > avoid
> > > > > > > zookeeper
> > > > > > > > > >> >> > > altogether. A
> > > > > > > > > >> >> > > > > > > Client
> > > > > > > > > >> >> > > > > > > > will call broker with required
> > information
> > > > > > (owner,
> > > > > > > > > >> renwer,
> > > > > > > > > >> >> > > > > expiration)
> > > > > > > > > >> >> > > > > > > and
> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker
> > won't
> > > > > store
> > > > > > > this
> > > > > > > > > in
> > > > > > > > > >> >> > > zookeeper.
> > > > > > > > > >> >> > > > > > From
> > > > > > > > > >> >> > > > > > > > this point a client can contact any
> > broker
> > > > with
> > > > > > all
> > > > > > > > the
> > > > > > > > > >> >> > > delegation
> > > > > > > > > >> >> > > > > > token
> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac,
> > > uuid)
> > > > > the
> > > > > > > > borker
> > > > > > > > > >> will
> > > > > > > > > >> >> > > > > regenerate
> > > > > > > > > >> >> > > > > > > the
> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with
> hmac
> > > > > > presented
> > > > > > > by
> > > > > > > > > >> client ,
> > > > > > > > > >> >> > > > broker
> > > > > > > > > >> >> > > > > > > will
> > > > > > > > > >> >> > > > > > > > allow the request to authenticate.
> Only
> > > > > problem
> > > > > > > with
> > > > > > > > > >> this
> > > > > > > > > >> >> > > approach
> > > > > > > > > >> >> > > > > is
> > > > > > > > > >> >> > > > > > if
> > > > > > > > > >> >> > > > > > > > the secret key is compromised any
> client
> > > can
> > > > > now
> > > > > > > > > generate
> > > > > > > > > >> >> > random
> > > > > > > > > >> >> > > > > tokens
> > > > > > > > > >> >> > > > > > > and
> > > > > > > > > >> >> > > > > > > > they will still be able to authenticate
> > as
> > > > any
> > > > > > user
> > > > > > > > > they
> > > > > > > > > >> like.
> > > > > > > > > >> >> > > with
> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens
> > > > > acquired
> > > > > > > via
> > > > > > > > a
> > > > > > > > > >> broker
> > > > > > > > > >> >> > > > (using
> > > > > > > > > >> >> > > > > > some
> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation
> token)
> > > will
> > > > > be
> > > > > > > > > >> accepted. We
> > > > > > > > > >> >> > > need
> > > > > > > > > >> >> > > > to
> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more sense
> > and
> > > > we
> > > > > > can
> > > > > > > go
> > > > > > > > > >> over it
> > > > > > > > > >> >> > in
> > > > > > > > > >> >> > > > > > > tomorrow's
> > > > > > > > > >> >> > > > > > > > meeting.
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > Thanks
> > > > > > > > > >> >> > > > > > > > Parth
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun
> > Rao <
> > > > > > > > > >> jun@confluent.io>
> > > > > > > > > >> >> > > > wrote:
> > > > > > > > > >> >> > > > > > > >
> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful
> for
> > > > Kafka
> > > > > > > > Connect
> > > > > > > > > >> and
> > > > > > > > > >> >> > Kafka
> > > > > > > > > >> >> > > > > rest
> > > > > > > > > >> >> > > > > > > > proxy
> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to
> run a
> > > > task
> > > > > on
> > > > > > > > > behalf
> > > > > > > > > >> of a
> > > > > > > > > >> >> > > > client.
> > > > > > > > > >> >> > > > > > We
> > > > > > > > > >> >> > > > > > > > will
> > > > > > > > > >> >> > > > > > > > > likely need to change how those
> > services
> > > > use
> > > > > > > Kafka
> > > > > > > > > >> clients
> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
> > shared
> > > > > client
> > > > > > > per
> > > > > > > > > >> worker,
> > > > > > > > > >> >> > we
> > > > > > > > > >> >> > > > will
> > > > > > > > > >> >> > > > > > > need
> > > > > > > > > >> >> > > > > > > > a
> > > > > > > > > >> >> > > > > > > > > client per user task since the
> > > > authentication
> > > > > > > > happens
> > > > > > > > > >> at the
> > > > > > > > > >> >> > > > > > connection
> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer
> > > will
> > > > be
> > > > > > the
> > > > > > > > > >> workers.
> > > > > > > > > >> >> > So,
> > > > > > > > > >> >> > > we
> > > > > > > > > >> >> > > > > > > > probably
> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For
> > > Kafka
> > > > > rest
> > > > > > > > > proxy,
> > > > > > > > > >> the
> > > > > > > > > >> >> > > > renewer
> > > > > > > > > >> >> > > > > > can
> > > > > > > > > >> >> > > > > > > > > probably just be the creator of the
> > > token.
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can
> > > request
> > > > > > > > delegation
> > > > > > > > > >> tokens?
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
> > > > > delegation
> > > > > > > > tokens
> > > > > > > > > >> in an
> > > > > > > > > >> >> > > > > encrypted
> > > > > > > > > >> >> > > > > > > > > channel?
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
> > > > tokens,
> > > > > > > every
> > > > > > > > > >> broker?
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would
> it
> > be
> > > > > > better
> > > > > > > to
> > > > > > > > > do
> > > > > > > > > >> it in
> > > > > > > > > >> >> > a
> > > > > > > > > >> >> > > > > > request
> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in the
> > > wiki
> > > > > > > > sometimes
> > > > > > > > > >> refers
> > > > > > > > > >> >> > to
> > > > > > > > > >> >> > > > the
> > > > > > > > > >> >> > > > > > end
> > > > > > > > > >> >> > > > > > > > > client and some other times refers to
> > the
> > > > > > client
> > > > > > > > > using
> > > > > > > > > >> the
> > > > > > > > > >> >> > > > > delegation
> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
> > distinguish
> > > > > > between
> > > > > > > > the
> > > > > > > > > >> two.
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence
> > > "Broker
> > > > > > also
> > > > > > > > > >> stores the
> > > > > > > > > >> >> > > > > > > > DelegationToken
> > > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a
> > bit
> > > > > > more? I
> > > > > > > > > >> thought the
> > > > > > > > > >> >> > > > > > > delegation
> > > > > > > > > >> >> > > > > > > > > token is the hmac.
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > Thanks,
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > Jun
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun
> > Rao
> > > <
> > > > > > > > > >> jun@confluent.io>
> > > > > > > > > >> >> > > > wrote:
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > > > > > > >> >> > > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite.
> > We
> > > > can
> > > > > > > > discuss
> > > > > > > > > >> this in
> > > > > > > > > >> >> > > the
> > > > > > > > > >> >> > > > > > > meeting
> > > > > > > > > >> >> > > > > > > > > > tomorrow.
> > > > > > > > > >> >> > > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > > Thanks,
> > > > > > > > > >> >> > > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > > Jun
> > > > > > > > > >> >> > > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM,
> > > Harsha <
> > > > > > > > > >> kafka@harsha.io>
> > > > > > > > > >> >> > > > wrote:
> > > > > > > > > >> >> > > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > >> Hi All,
> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP
> meeting
> > > > > around
> > > > > > > > this.
> > > > > > > > > >> The KIP
> > > > > > > > > >> >> > is
> > > > > > > > > >> >> > > > up
> > > > > > > > > >> >> > > > > > for
> > > > > > > > > >> >> > > > > > > > > >> sometime and if there
> are
> > > any
> > > > > > > > questions
> > > > > > > > > >> lets
> > > > > > > > > >> >> > > > quickly
> > > > > > > > > >> >> > > > > > hash
> > > > > > > > > >> >> > > > > > > > out
> > > > > > > > > >> >> > > > > > > > > >> details.
> > > > > > > > > >> >> > > > > > > > > >>
> > > > > > > > > >> >> > > > > > > > > >> Thanks,
> > > > > > > > > >> >> > > > > > > > > >> Harsha
> > > > > > > > > >> >> > > > > > > > > >>
> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM,
> > > parth
> > > > > > > > > brahmbhatt
> > > > > > > > > >> wrote:
> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
> > system
> > > > uses
> > > > > > so
> > > > > > > no
> > > > > > > > > >> good
> > > > > > > > > >> >> > reason
> > > > > > > > > >> >> > > > > > really.
> > > > > > > > > >> >> > > > > > > > We
> > > > > > > > > >> >> > > > > > > > > >> > could
> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
> > newest
> > > > > > > > recommended
> > > > > > > > > >> standard
> > > > > > > > > >> >> > > is.
> > > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > > >> >> > > > > > > > > >> > Thanks
> > > > > > > > > >> >> > > > > > > > > >> > Parth
> > > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM,
> > > > Ismael
> > > > > > > Juma <
> > > > > > > > > >> >> > > > > ismael@juma.me.uk
> > > > > > > > > >> >> > > > > > >
> > > > > > > > > >> >> > > > > > > > > wrote:
> > > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
> > started
> > > > > > > reviewing
> > > > > > > > > >> this and
> > > > > > > > > >> >> > > may
> > > > > > > > > >> >> > > > > have
> > > > > > > > > >> >> > > > > > > > > >> additional
> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The immediate
> > > > > question
> > > > > > > that
> > > > > > > > > >> came to
> > > > > > > > > >> >> > > mind
> > > > > > > > > >> >> > > > is
> > > > > > > > > >> >> > > > > > our
> > > > > > > > > >> >> > > > > > > > > >> choice of
> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's
> > > marked
> > > > > as
> > > > > > > > > >> OBSOLETE in
> > > > > > > > > >> >> > the
> > > > > > > > > >> >> > > > IANA
> > > > > > > > > >> >> > > > > > > > > Registry
> > > > > > > > > >> >> > > > > > > > > >> of
> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the
> original
> > > RFC
> > > > > > > (2831)
> > > > > > > > > has
> > > > > > > > > >> been
> > > > > > > > > >> >> > > moved
> > > > > > > > > >> >> > > > > to
> > > > > > > > > >> >> > > > > > > > > Historic
> > > > > > > > > >> >> > > > > > > > > >> > > status:
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/
> > > rfc6331
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > >
> > > > > > > > > >> >> > >
> > > > > > > > > >>
> > > > > > >
> > > > http://www.iana.org/assignments/sasl-mechanisms/
> sasl-mechanisms.xhtml
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind
> > that
> > > > > > choice?
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> > > > > > > > > >> >> > > > > > > > > >> > > Ismael
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29
> > PM,
> > > > Gwen
> > > > > > > > > Shapira <
> > > > > > > > > >> >> > > > > > > gwen@confluent.io
> > > > > > > > > >> >> > > > > > > > >
> > > > > > > > > >> >> > > > > > > > > >> wrote:
> > > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that
> > > even
> > > > > > though
> > > > > > > > > >> delegation
> > > > > > > > > >> >> > > > tokens
> > > > > > > > > >> >> > > > > > > are a
> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very
> > > strongly
> > > > > > about
> > > > > > > > not
> > > > > > > > > >> adding
> > > > > > > > > >> >> > > > > > dependency
> > > > > > > > > >> >> > > > > > > > on
> > > > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing
> delegation
> > > > > tokens
> > > > > > > for
> > > > > > > > > >> Kafka. The
> > > > > > > > > >> >> > > KIP
> > > > > > > > > >> >> > > > > > > doesn't
> > > > > > > > > >> >> > > > > > > > > >> imply
> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if
> you
> > > can
> > > > > > > > clarify...
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the
> KIP
> > so
> > > > no
> > > > > > one
> > > > > > > > will
> > > > > > > > > >> read
> > > > > > > > > >> >> > the
> > > > > > > > > >> >> > > > KIP
> > > > > > > > > >> >> > > > > > and
> > > > > > > > > >> >> > > > > > > > > panic
> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
> > > > > release...
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation
> > token
> > > at
> > > > > any
> > > > > > > > time
> > > > > > > > > >> after
> > > > > > > > > >> >> > > > > > > > authenticating?
> > > > > > > > > >> >> > > > > > > > > >> only
> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> > > > authenticated
> > > > > > you
> > > > > > > > can
> > > > > > > > > >> get
> > > > > > > > > >> >> > > > delegation
> > > > > > > > > >> >> > > > > > > > tokens.
> > > > > > > > > >> >> > > > > > > > > >> We
> > > > > > > > > >> >> > > > > > > > > >> > > need
> > > > > > > > > >> >> > > > > > > > > >> > > > to
> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> > > > authenticated
> > > > > > > using
> > > > > > > > > >> delegation
> > > > > > > > > >> >> > > > > token,
> > > > > > > > > >> >> > > > > > > can
> > > > > > > > > >> >> > > > > > > > > also
> > > > > > > > > >> >> > > > > > > > > >> > > > acquire
> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or
> > not.
> > > > > Also
> > > > > > > > there
> > > > > > > > > >> is the
> > > > > > > > > >> >> > > > > question
> > > > > > > > > >> >> > > > > > of
> > > > > > > > > >> >> > > > > > > > do
> > > > > > > > > >> >> > > > > > > > > we
> > > > > > > > > >> >> > > > > > > > > >> > > allow
> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire
> delegation
> > > > token
> > > > > > or
> > > > > > > we
> > > > > > > > > >> want
> > > > > > > > > >> >> > > specific
> > > > > > > > > >> >> > > > > > ACLs
> > > > > > > > > >> >> > > > > > > (I
> > > > > > > > > >> >> > > > > > > > > >> think
> > > > > > > > > >> >> > > > > > > > > >> > > its
> > > > > > > > > >> >> > > > > > > > > >> > > > an
> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
> > > overkill.
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
> > > > options:
> > > > > > > > Either
> > > > > > > > > >> require
> > > > > > > > > >> >> > > > > Kerberos
> > > > > > > > > >> >> > > > > > > > auth
> > > > > > > > > >> >> > > > > > > > > >> for
> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require
> non-owners
> > to
> > > > > > renew.
> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
> > simpler
> > > > (it
> > > > > > > > > basically
> > > > > > > > > >> >> > require
> > > > > > > > > >> >> > > a
> > > > > > > > > >> >> > > > > "job
> > > > > > > > > >> >> > > > > > > > > master"
> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for
> the
> > > > > > renewal,
> > > > > > > it
> > > > > > > > > >> will have
> > > > > > > > > >> >> > > its
> > > > > > > > > >> >> > > > > own
> > > > > > > > > >> >> > > > > > > > > >> identity
> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is
> the
> > > > > correct
> > > > > > > > > design
> > > > > > > > > >> >> > pattern
> > > > > > > > > >> >> > > > > > anyway.
> > > > > > > > > >> >> > > > > > > > For
> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
> > > > > coordinate
> > > > > > > > > >> renewals?),
> > > > > > > > > >> >> > but
> > > > > > > > > >> >> > > > it
> > > > > > > > > >> >> > > > > is
> > > > > > > > > >> >> > > > > > > > hard
> > > > > > > > > >> >> > > > > > > > > to
> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without
> > > looking
> > > > at
> > > > > > the
> > > > > > > > > code
> > > > > > > > > >> >> > changes
> > > > > > > > > >> >> > > > > > > required.
> > > > > > > > > >> >> > > > > > > > If
> > > > > > > > > >> >> > > > > > > > > >> you
> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
> > "require
> > > > > > > Kerberos"
> > > > > > > > > >> will look
> > > > > > > > > >> >> > > in
> > > > > > > > > >> >> > > > > > Kafka
> > > > > > > > > >> >> > > > > > > > > code,
> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a
> look.
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is that
> > > > tokens
> > > > > > will
> > > > > > > > > >> propagate
> > > > > > > > > >> >> > via
> > > > > > > > > >> >> > > > ZK
> > > > > > > > > >> >> > > > > > but
> > > > > > > > > >> >> > > > > > > > > >> without
> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> > > > > UpdateMetadata
> > > > > > > > > >> protocol,
> > > > > > > > > >> >> > > > correct?
> > > > > > > > > >> >> > > > > > > > Clients
> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on
> > SASL
> > > > auth
> > > > > > > > failure
> > > > > > > > > >> (IIRC),
> > > > > > > > > >> >> > > but
> > > > > > > > > >> >> > > > > > since
> > > > > > > > > >> >> > > > > > > > the
> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between
> > > brokers
> > > > > > > asynch,
> > > > > > > > > we
> > > > > > > > > >> will
> > > > > > > > > >> >> > > need
> > > > > > > > > >> >> > > > to
> > > > > > > > > >> >> > > > > > > > retry a
> > > > > > > > > >> >> > > > > > > > > >> bit
> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing
> > auth
> > > > due
> > > > > to
> > > > > > > > > timing
> > > > > > > > > >> >> > issues.
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
> > > alternatives
> > > > > > right
> > > > > > > > > now.
> > > > > > > > > >> The
> > > > > > > > > >> >> > > > current
> > > > > > > > > >> >> > > > > > > > > documented
> > > > > > > > > >> >> > > > > > > > > >> > > > approach
> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it
> > does
> > > > not
> > > > > > > > require
> > > > > > > > > >> any
> > > > > > > > > >> >> > > changes
> > > > > > > > > >> >> > > > > to
> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
> > > > approach
> > > > > > can
> > > > > > > > > remove
> > > > > > > > > >> >> > > zookeeper
> > > > > > > > > >> >> > > > > > > > > dependency
> > > > > > > > > >> >> > > > > > > > > >> as
> > > > > > > > > >> >> > > > > > > > > >> > > well
> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in
> > KIP
> > > > > > > > discussion
> > > > > > > > > >> call.*
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do
> > you
> > > > > want
> > > > > > to
> > > > > > > > > ping
> > > > > > > > > >> Jun to
> > > > > > > > > >> >> > > > > > arrange a
> > > > > > > > > >> >> > > > > > > > > call?
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
> > suggestion
> > > of
> > > > > > > having
> > > > > > > > > >> just the
> > > > > > > > > >> >> > > > > > controller
> > > > > > > > > >> >> > > > > > > > > issue
> > > > > > > > > >> >> > > > > > > > > >> the
> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to
> avoid
> > > > > syncing
> > > > > > a
> > > > > > > > > shared
> > > > > > > > > >> >> > secret.
> > > > > > > > > >> >> > > > Not
> > > > > > > > > >> >> > > > > > > sure
> > > > > > > > > >> >> > > > > > > > if
> > > > > > > > > >> >> > > > > > > > > >> we
> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
> > > discussion
> > > > > here
> > > > > > > or
> > > > > > > > on
> > > > > > > > > >> the
> > > > > > > > > >> >> > > wiki. I
> > > > > > > > > >> >> > > > > > think
> > > > > > > > > >> >> > > > > > > > > that
> > > > > > > > > >> >> > > > > > > > > >> we
> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem
> of
> > > > "token
> > > > > > > > > >> distribution"
> > > > > > > > > >> >> > > from
> > > > > > > > > >> >> > > > > > > "shared
> > > > > > > > > >> >> > > > > > > > > >> secret
> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
> > > > > controller
> > > > > > > as
> > > > > > > > > the
> > > > > > > > > >> only
> > > > > > > > > >> >> > > token
> > > > > > > > > >> >> > > > > > > > generator
> > > > > > > > > >> >> > > > > > > > > >> to
> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue,
> > while
> > > > > still
> > > > > > > > using
> > > > > > > > > >> ZK async
> > > > > > > > > >> >> > > to
> > > > > > > > > >> >> > > > > > > > distribute
> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
> > previous
> > > > > Email
> > > > > > I
> > > > > > > am
> > > > > > > > > >> fine with
> > > > > > > > > >> >> > > > that
> > > > > > > > > >> >> > > > > > > > approach
> > > > > > > > > >> >> > > > > > > > > >> as
> > > > > > > > > >> >> > > > > > > > > >> > > long
> > > > > > > > > >> >> > > > > > > > > >> > > > as
> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
> > > > complexity
> > > > > > of
> > > > > > > > > >> >> > > adding/updating
> > > > > > > > > >> >> > > > > APIS
> > > > > > > > > >> >> > > > > > > > adds
> > > > > > > > > >> >> > > > > > > > > >> enough
> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with
> > the
> > > > > > > controller
> > > > > > > > > >> approach
> > > > > > > > > >> >> > is
> > > > > > > > > >> >> > > > > > secret
> > > > > > > > > >> >> > > > > > > > > >> rotation
> > > > > > > > > >> >> > > > > > > > > >> > > can
> > > > > > > > > >> >> > > > > > > > > >> > > > be
> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and
> would
> > > not
> > > > > > > require
> > > > > > > > > >> >> > > deployment. *
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> > > > complexity
> > > > > > (or
> > > > > > > > > point
> > > > > > > > > >> me to
> > > > > > > > > >> >> > > the
> > > > > > > > > >> >> > > > > > email
> > > > > > > > > >> >> > > > > > > I
> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
> > > > required?
> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell,
> clients
> > > can
> > > > > > > already
> > > > > > > > > >> find the
> > > > > > > > > >> >> > > > > > controller
> > > > > > > > > >> >> > > > > > > > from
> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
> > > concerned
> > > > > > about
> > > > > > > > > >> controller
> > > > > > > > > >> >> > > > load.
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of
> > > > forcing
> > > > > > > > > kerberos
> > > > > > > > > >> auth
> > > > > > > > > >> >> > for
> > > > > > > > > >> >> > > > > > > renwal, I
> > > > > > > > > >> >> > > > > > > > > >> think
> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport
> layer
> > > the
> > > > > the
> > > > > > > > > request
> > > > > > > > > >> >> > content
> > > > > > > > > >> >> > > > in
> > > > > > > > > >> >> > > > > a
> > > > > > > > > >> >> > > > > > > > pretty
> > > > > > > > > >> >> > > > > > > > > >> ugly
> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
> > renewer
> > > to
> > > > > > > > non-owner
> > > > > > > > > >> is
> > > > > > > > > >> >> > > better.
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a
> necessary
> > > > evil.
> > > > > > > While
> > > > > > > > > >> this will
> > > > > > > > > >> >> > > make
> > > > > > > > > >> >> > > > > the
> > > > > > > > > >> >> > > > > > > > kafka
> > > > > > > > > >> >> > > > > > > > > >> code
> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
> > > > forcing
> > > > > > > > renewer
> > > > > > > > > >> to
> > > > > > > > > >> >> > > > non-owner
> > > > > > > > > >> >> > > > > > > pushes
> > > > > > > > > >> >> > > > > > > > > >> the code
> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and
> makes
> > > it
> > > > > even
> > > > > > > > > harder
> > > > > > > > > >> to
> > > > > > > > > >> >> > > > > > integrate. *
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't
> > > think
> > > > > the
> > > > > > > > > >> "renewal by
> > > > > > > > > >> >> > > > other"
> > > > > > > > > >> >> > > > > > > > approach
> > > > > > > > > >> >> > > > > > > > > >> is
> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we
> > > > expect
> > > > > to
> > > > > > > use
> > > > > > > > > >> >> > delegation
> > > > > > > > > >> >> > > > > tokens
> > > > > > > > > >> >> > > > > > > > since
> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an app-master
> > of
> > > > some
> > > > > > > sort
> > > > > > > > > who
> > > > > > > > > >> >> > > requested
> > > > > > > > > >> >> > > > > the
> > > > > > > > > >> >> > > > > > > > token
> > > > > > > > > >> >> > > > > > > > > to
> > > > > > > > > >> >> > > > > > > > > >> > > > begin with.
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my
> question
> > > on
> > > > > how
> > > > > > > > > multiple
> > > > > > > > > >> >> > > > identities
> > > > > > > > > >> >> > > > > > will
> > > > > > > > > >> >> > > > > > > > be
> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear
> > to
> > > > me -
> > > > > > > > AFAIK,
> > > > > > > > > >> we don't
> > > > > > > > > >> >> > > > > > > > authenticate
> > > > > > > > > >> >> > > > > > > > > >> each
> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> > > > > > connections.
> > > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
> > connections,
> > > > and
> > > > > > only
> > > > > > > > > when
> > > > > > > > > >> they
> > > > > > > > > >> >> > are
> > > > > > > > > >> >> > > > > being
> > > > > > > > > >> >> > > > > > > > > >> established.
> > > > > > > > > >> >> > > > > > > > > >> > > > Let
> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a
> > > > > question,
> > > > > > > in
> > > > > > > > > >> absence of
> > > > > > > > > >> >> > > > > > > delegation
> > > > > > > > > >> >> > > > > > > > > >> tokens if
> > > > > > > > > >> >> > > > > > > > > >> > > > we
> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use
> case
> > > > using
> > > > > > user
> > > > > > > > > >> TGT's how
> > > > > > > > > >> >> > > would
> > > > > > > > > >> >> > > > > we
> > > > > > > > > >> >> > > > > > do
> > > > > > > > > >> >> > > > > > > > it?
> > > > > > > > > >> >> > > > > > > > > >> My
> > > > > > > > > >> >> > > > > > > > > >> > > point
> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no
> different
> > > > with
> > > > > > > > > delegation
> > > > > > > > > >> >> > tokens.
> > > > > > > > > >> >> > > > The
> > > > > > > > > >> >> > > > > > use
> > > > > > > > > >> >> > > > > > > > > case
> > > > > > > > > >> >> > > > > > > > > >> you
> > > > > > > > > >> >> > > > > > > > > >> > > are
> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > > > > > > impersonation.*
> > > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of
> > the
> > > > > things
> > > > > > > > that
> > > > > > > > > >> >> > del
>
--
Regards,
Ashish
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Harsha Chintalapani <ka...@harsha.io>.
Ashish,
Yes we are working on it. Lets discuss in the next KIP meeting.
I'll join.
-Harsha
On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <as...@cloudera.com> wrote:
> Hello Harsha,
>
> Are you still working on this? Wondering if we can discuss this in next KIP
> meeting, if you can join.
>
> On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Hi Grant,
> > We are working on it. Will add the details to KIP about the
> > request protocol.
> >
> > Thanks,
> > Harsha
> >
> > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com> wrote:
> >
> > > Hi Parth,
> > >
> > > Are you still working on this? If you need any help please don't
> hesitate
> > > to ask.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Parth,
> > > >
> > > > Thanks for the reply.
> > > >
> > > > It makes sense to only allow the renewal by users that authenticated
> > > using
> > > > *non* delegation token mechanism. Then, should we make the renewal a
> > > list?
> > > > For example, in the case of rest proxy, it will be useful for every
> > > > instance of rest proxy to be able to renew the tokens.
> > > >
> > > > It would be clearer if we can document the request protocol like
> > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> > Commandlineandcentralizedadministrativeoperations-
> > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> > > > .
> > > >
> > > > It would also be useful to document the client APIs.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > > > brahmbhatt.parth@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am suggesting that we will only allow the renewal by users that
> > > > > authenticated using *non* delegation token mechanism. For example,
> If
> > > > user
> > > > > Alice authenticated using kerberos and requested delegation tokens,
> > > only
> > > > > user Alice authenticated via non delegation token mechanism can
> > renew.
> > > > > Clients that have access to delegation tokens can not issue
> renewal
> > > > > request for renewing their own token and this is primarily
> important
> > to
> > > > > reduce the time window for which a compromised token will be valid.
> > > > >
> > > > > To clarify, Yes any authenticated user can request delegation
> tokens
> > > but
> > > > > even here I would recommend to avoid creating a chain where a
> client
> > > > > authenticated via delegation token request for more delegation
> > tokens.
> > > > > Basically anyone can request delegation token, as long as they
> > > > authenticate
> > > > > via a non delegation token mechanism.
> > > > >
> > > > > Aren't classes listed here
> > > > > <
> > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKaf
> > ka-PublicInterfaces
> > > > > >
> > > > > sufficient?
> > > > >
> > > > > Thanks
> > > > > Parth
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Parth,
> > > > > >
> > > > > > Thanks for the reply. A couple of comments inline below.
> > > > > >
> > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > >
> > > > > > > 1. Who / how are tokens renewed? By original requester only? or
> > > using
> > > > > > > Kerberos
> > > > > > > auth only?
> > > > > > > My recommendation is to do this only using Kerberos auth and
> only
> > > > threw
> > > > > > the
> > > > > > > renewer specified during the acquisition request.
> > > > > > >
> > > > > > >
> > > > > > Hmm, not sure that I follow this. Are you saying that any client
> > > > > > authenticated with the delegation token can renew, i.e. there is
> no
> > > > > renewer
> > > > > > needed?
> > > > > >
> > > > > > Also, just to be clear, any authenticated client (either through
> > SASL
> > > > or
> > > > > > SSL) can request a delegation token for the authenticated user,
> > > right?
> > > > > >
> > > > > >
> > > > > > > 2. Are tokens stored on each broker or in ZK?
> > > > > > > My recommendation is still to store in ZK or not store them at
> > all.
> > > > The
> > > > > > > whole controller based distribution is too much overhead with
> not
> > > > much
> > > > > to
> > > > > > > achieve.
> > > > > > >
> > > > > > > 3. How are tokens invalidated / expired?
> > > > > > > Either by expiration time out or through an explicit request to
> > > > > > invalidate.
> > > > > > >
> > > > > > > 4. Which encryption algorithm is used?
> > > > > > > SCRAM
> > > > > > >
> > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP but
> > was
> > > > > > > discussed
> > > > > > > in this thread)?
> > > > > > > There is no imperonation proposal. I tried and explained how
> its
> > a
> > > > > > > different problem and why its not really necessary to discuss
> > that
> > > as
> > > > > > part
> > > > > > > of this KIP. This KIP will not support any impersonation, it
> > will
> > > > just
> > > > > > be
> > > > > > > another way to authenticate.
> > > > > > >
> > > > > > > 6. Do we need new ACLs, if so - for what actions?
> > > > > > > We do not need new ACLs.
> > > > > > >
> > > > > > >
> > > > > > Could we document the format of the new request/response and
> their
> > > > > > associated Resource and Operation for ACL?
> > > > > >
> > > > > >
> > > > > > > 7. How would the delegation token be configured in the client?
> > > > > > > Should be through config. I wasn't planning on supporting JAAS
> > for
> > > > > > tokens.
> > > > > > > I don't believe hadoop does this either.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Parth
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > > > Harsha,
> > > > > > > >
> > > > > > > > Another question.
> > > > > > > >
> > > > > > > > 9. How would the delegation token be configured in the
> client?
> > > The
> > > > > > > standard
> > > > > > > > way is to do this through JAAS. However, we will need to
> think
> > > > > through
> > > > > > if
> > > > > > > > this is convenient in a shared environment. For example,
> when a
> > > new
> > > > > > task
> > > > > > > is
> > > > > > > > added to a Storm worker node, do we need to dynamically add a
> > new
> > > > > > section
> > > > > > > > in the JAAS file? It may be more convenient if we can pass in
> > the
> > > > > token
> > > > > > > > through the config directly w/o going through JAAS.
> > > > > > > >
> > > > > > > > Are you or Parth still actively working on this KIP?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > > > >
> > > > > > > > > Just to add on that list.
> > > > > > > > >
> > > > > > > > > 2. It would be good to document the format of the data
> stored
> > > in
> > > > > ZK.
> > > > > > > > > 7. Earlier, there was a discussion on whether the tokens
> > should
> > > > be
> > > > > > > > > propagated through ZK like config/acl/quota, or through the
> > > > > > controller.
> > > > > > > > > Currently, the controller is only designed for propagating
> > > topic
> > > > > > > > metadata,
> > > > > > > > > but not other data.
> > > > > > > > > 8. Should we use SCRAM to send the token instead of
> > DIGEST-MD5
> > > > > since
> > > > > > > it's
> > > > > > > > > deprecated?
> > > > > > > > >
> > > > > > > > > Also, the images in the wiki seem broken.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> > > > gwen@confluent.io>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> From what I can see, remaining questions are:
> > > > > > > > >>
> > > > > > > > >> 1. Who / how are tokens renewed? By original requester
> only?
> > > or
> > > > > > using
> > > > > > > > >> Kerberos auth only?
> > > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > > > > > >> 3. How are tokens invalidated / expired?
> > > > > > > > >> 4. Which encryption algorithm is used?
> > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the
> KIP
> > > but
> > > > > was
> > > > > > > > >> discussed in this thread)?
> > > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > > > > > >>
> > > > > > > > >> Gwen
> > > > > > > > >>
> > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io>
> > > wrote:
> > > > > > > > >> > Jun & Ismael,
> > > > > > > > >> > Unfortunately I couldn't attend
> > the
> > > > KIP
> > > > > > > > meeting
> > > > > > > > >> > when delegation tokens
> discussed.
> > > > > > > Appreciate
> > > > > > > > if
> > > > > > > > >> > you can update the thread if
> you
> > > have
> > > > > any
> > > > > > > > >> > further questions.
> > > > > > > > >> > Thanks,
> > > > > > > > >> > Harsha
> > > > > > > > >> >
> > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > > > > > >> >> It seems that the links to images in the KIP are
> broken.
> > > > > > > > >> >>
> > > > > > > > >> >> Liquan
> > > > > > > > >> >>
> > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > >> >>
> > > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > > > > > >> >> > In the current proposal we only allow a user to get
> > > > > delegation
> > > > > > > > token
> > > > > > > > >> for
> > > > > > > > >> >> > the identity that it authenticated as using another
> > > > > mechanism,
> > > > > > > i.e.
> > > > > > > > >> A user
> > > > > > > > >> >> > that authenticate using a keytab for principal
> > > > > > user1@EXAMPLE.COM
> > > > > > > > >> will get
> > > > > > > > >> >> > delegation tokens for that user only. In future I
> think
> > > we
> > > > > will
> > > > > > > > have
> > > > > > > > >> to
> > > > > > > > >> >> > extend support such that we allow some set of users (
> > > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM,
> storm-nimbus@EXAMPLE.COM)
> > > to
> > > > > > > acquire
> > > > > > > > >> >> > delegation tokens on behalf of other users whose
> > identity
> > > > > they
> > > > > > > have
> > > > > > > > >> >> > verified independently. Kafka brokers will have ACLs
> > to
> > > > > > control
> > > > > > > > >> which
> > > > > > > > >> >> > users are allowed to impersonate other users and get
> > > tokens
> > > > > on
> > > > > > > > >> behalf of
> > > > > > > > >> >> > them. Overall Impersonation is a whole different
> > problem
> > > in
> > > > > my
> > > > > > > > >> opinion and
> > > > > > > > >> >> > I think we can tackle it in separate KIP.
> > > > > > > > >> >> >
> > > > > > > > >> >> > 111. What's the typical rate of getting and renewing
> > > > > delegation
> > > > > > > > >> tokens?
> > > > > > > > >> >> > Typically this should be very very low, 1 request per
> > > > minute
> > > > > > is a
> > > > > > > > >> >> > relatively high estimate. However it depends on the
> > token
> > > > > > > > >> expiration. I am
> > > > > > > > >> >> > less worried about the extra load it puts on
> controller
> > > vs
> > > > > the
> > > > > > > > added
> > > > > > > > >> >> > complexity and the value it offers.
> > > > > > > > >> >> >
> > > > > > > > >> >> > Thanks
> > > > > > > > >> >> > Parth
> > > > > > > > >> >> >
> > > > > > > > >> >> >
> > > > > > > > >> >> >
> > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > > > > > ismael@juma.me.uk>
> > > > > > > > >> wrote:
> > > > > > > > >> >> >
> > > > > > > > >> >> > > Thanks Rajini. It would probably require a separate
> > KIP
> > > > as
> > > > > it
> > > > > > > > will
> > > > > > > > >> >> > > introduce user visible changes. We could also
> update
> > > > KIP-48
> > > > > > to
> > > > > > > > >> have this
> > > > > > > > >> >> > > information, but it seems cleaner to do it
> > separately.
> > > We
> > > > > can
> > > > > > > > >> discuss
> > > > > > > > >> >> > that
> > > > > > > > >> >> > > in the KIP call today.
> > > > > > > > >> >> > >
> > > > > > > > >> >> > > Ismael
> > > > > > > > >> >> > >
> > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > > > > > >> >> > >
> > > > > > > > >> >> > > > Ismael,
> > > > > > > > >> >> > > >
> > > > > > > > >> >> > > > I have created a JIRA (
> > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that
> > need
> > > > > > another
> > > > > > > > >> KIP? If
> > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just be
> a
> > > JIRA
> > > > > > that
> > > > > > > > gets
> > > > > > > > >> >> > > reviewed
> > > > > > > > >> >> > > > when the PR is ready?
> > > > > > > > >> >> > > >
> > > > > > > > >> >> > > > Thank you,
> > > > > > > > >> >> > > >
> > > > > > > > >> >> > > > Rajini
> > > > > > > > >> >> > > >
> > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > > > > > ismael@juma.me.uk>
> > > > > > > > >> >> > wrote:
> > > > > > > > >> >> > > >
> > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good
> candidate.
> > > > > > > > >> >> > > > >
> > > > > > > > >> >> > > > > Gwen had independently mentioned this as a SASL
> > > > > mechanism
> > > > > > > > that
> > > > > > > > >> might
> > > > > > > > >> >> > be
> > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to
> check
> > > it
> > > > in
> > > > > > > more
> > > > > > > > >> detail.
> > > > > > > > >> >> > > Good
> > > > > > > > >> >> > > > > to know that you are willing to contribute an
> > > > > > > implementation.
> > > > > > > > >> Maybe
> > > > > > > > >> >> > we
> > > > > > > > >> >> > > > > should file a separate JIRA for this?
> > > > > > > > >> >> > > > >
> > > > > > > > >> >> > > > > Ismael
> > > > > > > > >> >> > > > >
> > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini
> Sivaram <
> > > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > > > >> >> > > > >
> > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response
> Authentication
> > > > > > > Mechanism)
> > > > > > > > >> is a
> > > > > > > > >> >> > > better
> > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come
> > > with a
> > > > > > > > built-in
> > > > > > > > >> SCRAM
> > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy
> > to
> > > > add
> > > > > > > > support
> > > > > > > > >> in
> > > > > > > > >> >> > Kafka
> > > > > > > > >> >> > > > > since
> > > > > > > > >> >> > > > > > it would be a useful mechanism to support
> > anyway.
> > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677
> describes
> > > the
> > > > > > > protocol
> > > > > > > > >> for
> > > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > > > > >> >> > > > > >
> > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > > > > > jun@confluent.io
> > > > > > > > >
> > > > > > > > >> wrote:
> > > > > > > > >> >> > > > > >
> > > > > > > > >> >> > > > > > > Parth,
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of
> more
> > > > > > questions.
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting and
> > > > > renewing
> > > > > > > > >> delegation
> > > > > > > > >> >> > > > tokens?
> > > > > > > > >> >> > > > > > > That may have an impact on whether they
> > should
> > > be
> > > > > > > > directed
> > > > > > > > >> to the
> > > > > > > > >> >> > > > > > > controller.
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > Jun
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
> > > > brahmbhatt <
> > > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > > Hi Jun,
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add
> acls
> > > on
> > > > > who
> > > > > > > can
> > > > > > > > >> request
> > > > > > > > >> >> > > > > > delegation
> > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for that
> > yet
> > > > but
> > > > > > > down
> > > > > > > > >> the line
> > > > > > > > >> >> > > > when
> > > > > > > > >> >> > > > > we
> > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs it
> > will
> > > > be
> > > > > > > > >> necessary.
> > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > > > > > used/distributed
> > > > > > > > >> over
> > > > > > > > >> >> > secure
> > > > > > > > >> >> > > > > > > channels.
> > > > > > > > >> >> > > > > > > > * Depending on what design we end up
> > choosing
> > > > > > > > >> Invalidation will
> > > > > > > > >> >> > > be
> > > > > > > > >> >> > > > > > > > responsibility of every broker or
> > controller.
> > > > > > > > >> >> > > > > > > > * I am not sure if I documented somewhere
> > > that
> > > > > > > > >> invalidation
> > > > > > > > >> >> > will
> > > > > > > > >> >> > > > > > directly
> > > > > > > > >> >> > > > > > > > go through zookeeper but that is not the
> > > > intent.
> > > > > > > > >> Invalidation
> > > > > > > > >> >> > > will
> > > > > > > > >> >> > > > > > either
> > > > > > > > >> >> > > > > > > > be request based or due to expiration. No
> > > > direct
> > > > > > > > >> zookeeper
> > > > > > > > >> >> > > > > interaction
> > > > > > > > >> >> > > > > > > from
> > > > > > > > >> >> > > > > > > > any client.
> > > > > > > > >> >> > > > > > > > * "Broker also stores the DelegationToken
> > > > without
> > > > > > the
> > > > > > > > >> hmac in
> > > > > > > > >> >> > the
> > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion.
> > The
> > > > sole
> > > > > > > > >> purpose of
> > > > > > > > >> >> > > > > zookeeper
> > > > > > > > >> >> > > > > > in
> > > > > > > > >> >> > > > > > > > this design is as distribution channel
> for
> > > > tokens
> > > > > > > > >> between all
> > > > > > > > >> >> > > > brokers
> > > > > > > > >> >> > > > > > > and a
> > > > > > > > >> >> > > > > > > > layer that ensures only tokens that were
> > > > > generated
> > > > > > by
> > > > > > > > >> making a
> > > > > > > > >> >> > > > > request
> > > > > > > > >> >> > > > > > > to a
> > > > > > > > >> >> > > > > > > > broker will be accepted (more on this in
> > > second
> > > > > > > > >> paragraph). The
> > > > > > > > >> >> > > > token
> > > > > > > > >> >> > > > > > > > consists of few elements (owner, renewer,
> > > uuid
> > > > ,
> > > > > > > > >> expiration,
> > > > > > > > >> >> > > hmac)
> > > > > > > > >> >> > > > ,
> > > > > > > > >> >> > > > > > one
> > > > > > > > >> >> > > > > > > of
> > > > > > > > >> >> > > > > > > > which is the finally generated hmac but
> > hmac
> > > it
> > > > > > self
> > > > > > > is
> > > > > > > > >> >> > derivable
> > > > > > > > >> >> > > > if
> > > > > > > > >> >> > > > > > you
> > > > > > > > >> >> > > > > > > > have all the other elements of the token
> +
> > > > secret
> > > > > > key
> > > > > > > > to
> > > > > > > > >> >> > generate
> > > > > > > > >> >> > > > > hmac.
> > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL
> > support
> > > we
> > > > > do
> > > > > > > not
> > > > > > > > >> want the
> > > > > > > > >> >> > > > > entire
> > > > > > > > >> >> > > > > > > > token to be wire transferred to zookeeper
> > as
> > > > that
> > > > > > > will
> > > > > > > > >> be an
> > > > > > > > >> >> > > > insecure
> > > > > > > > >> >> > > > > > > wire
> > > > > > > > >> >> > > > > > > > transfer. Instead we only store all the
> > other
> > > > > > > elements
> > > > > > > > >> of a
> > > > > > > > >> >> > > > > delegation
> > > > > > > > >> >> > > > > > > > tokens. Brokers can read these elements
> and
> > > > > because
> > > > > > > > they
> > > > > > > > >> also
> > > > > > > > >> >> > > have
> > > > > > > > >> >> > > > > > access
> > > > > > > > >> >> > > > > > > > to secret key they will be able to
> generate
> > > > hmac
> > > > > on
> > > > > > > > >> their end.
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > One of the alternative proposed is to
> avoid
> > > > > > zookeeper
> > > > > > > > >> >> > > altogether. A
> > > > > > > > >> >> > > > > > > Client
> > > > > > > > >> >> > > > > > > > will call broker with required
> information
> > > > > (owner,
> > > > > > > > >> renwer,
> > > > > > > > >> >> > > > > expiration)
> > > > > > > > >> >> > > > > > > and
> > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker
> won't
> > > > store
> > > > > > this
> > > > > > > > in
> > > > > > > > >> >> > > zookeeper.
> > > > > > > > >> >> > > > > > From
> > > > > > > > >> >> > > > > > > > this point a client can contact any
> broker
> > > with
> > > > > all
> > > > > > > the
> > > > > > > > >> >> > > delegation
> > > > > > > > >> >> > > > > > token
> > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac,
> > uuid)
> > > > the
> > > > > > > borker
> > > > > > > > >> will
> > > > > > > > >> >> > > > > regenerate
> > > > > > > > >> >> > > > > > > the
> > > > > > > > >> >> > > > > > > > hmac and as long as it matches with hmac
> > > > > presented
> > > > > > by
> > > > > > > > >> client ,
> > > > > > > > >> >> > > > broker
> > > > > > > > >> >> > > > > > > will
> > > > > > > > >> >> > > > > > > > allow the request to authenticate. Only
> > > > problem
> > > > > > with
> > > > > > > > >> this
> > > > > > > > >> >> > > approach
> > > > > > > > >> >> > > > > is
> > > > > > > > >> >> > > > > > if
> > > > > > > > >> >> > > > > > > > the secret key is compromised any client
> > can
> > > > now
> > > > > > > > generate
> > > > > > > > >> >> > random
> > > > > > > > >> >> > > > > tokens
> > > > > > > > >> >> > > > > > > and
> > > > > > > > >> >> > > > > > > > they will still be able to authenticate
> as
> > > any
> > > > > user
> > > > > > > > they
> > > > > > > > >> like.
> > > > > > > > >> >> > > with
> > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens
> > > > acquired
> > > > > > via
> > > > > > > a
> > > > > > > > >> broker
> > > > > > > > >> >> > > > (using
> > > > > > > > >> >> > > > > > some
> > > > > > > > >> >> > > > > > > > auth scheme other than delegation token)
> > will
> > > > be
> > > > > > > > >> accepted. We
> > > > > > > > >> >> > > need
> > > > > > > > >> >> > > > to
> > > > > > > > >> >> > > > > > > > discuss which proposal makes more sense
> and
> > > we
> > > > > can
> > > > > > go
> > > > > > > > >> over it
> > > > > > > > >> >> > in
> > > > > > > > >> >> > > > > > > tomorrow's
> > > > > > > > >> >> > > > > > > > meeting.
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > Thanks
> > > > > > > > >> >> > > > > > > > Parth
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun
> Rao <
> > > > > > > > >> jun@confluent.io>
> > > > > > > > >> >> > > > wrote:
> > > > > > > > >> >> > > > > > > >
> > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful for
> > > Kafka
> > > > > > > Connect
> > > > > > > > >> and
> > > > > > > > >> >> > Kafka
> > > > > > > > >> >> > > > > rest
> > > > > > > > >> >> > > > > > > > proxy
> > > > > > > > >> >> > > > > > > > > where a worker agent will need to run a
> > > task
> > > > on
> > > > > > > > behalf
> > > > > > > > >> of a
> > > > > > > > >> >> > > > client.
> > > > > > > > >> >> > > > > > We
> > > > > > > > >> >> > > > > > > > will
> > > > > > > > >> >> > > > > > > > > likely need to change how those
> services
> > > use
> > > > > > Kafka
> > > > > > > > >> clients
> > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a
> shared
> > > > client
> > > > > > per
> > > > > > > > >> worker,
> > > > > > > > >> >> > we
> > > > > > > > >> >> > > > will
> > > > > > > > >> >> > > > > > > need
> > > > > > > > >> >> > > > > > > > a
> > > > > > > > >> >> > > > > > > > > client per user task since the
> > > authentication
> > > > > > > happens
> > > > > > > > >> at the
> > > > > > > > >> >> > > > > > connection
> > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer
> > will
> > > be
> > > > > the
> > > > > > > > >> workers.
> > > > > > > > >> >> > So,
> > > > > > > > >> >> > > we
> > > > > > > > >> >> > > > > > > > probably
> > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For
> > Kafka
> > > > rest
> > > > > > > > proxy,
> > > > > > > > >> the
> > > > > > > > >> >> > > > renewer
> > > > > > > > >> >> > > > > > can
> > > > > > > > >> >> > > > > > > > > probably just be the creator of the
> > token.
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can
> > request
> > > > > > > delegation
> > > > > > > > >> tokens?
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
> > > > delegation
> > > > > > > tokens
> > > > > > > > >> in an
> > > > > > > > >> >> > > > > encrypted
> > > > > > > > >> >> > > > > > > > > channel?
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
> > > tokens,
> > > > > > every
> > > > > > > > >> broker?
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would it
> be
> > > > > better
> > > > > > to
> > > > > > > > do
> > > > > > > > >> it in
> > > > > > > > >> >> > a
> > > > > > > > >> >> > > > > > request
> > > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 105. The terminology of client in the
> > wiki
> > > > > > > sometimes
> > > > > > > > >> refers
> > > > > > > > >> >> > to
> > > > > > > > >> >> > > > the
> > > > > > > > >> >> > > > > > end
> > > > > > > > >> >> > > > > > > > > client and some other times refers to
> the
> > > > > client
> > > > > > > > using
> > > > > > > > >> the
> > > > > > > > >> >> > > > > delegation
> > > > > > > > >> >> > > > > > > > > tokens. It would be useful to
> distinguish
> > > > > between
> > > > > > > the
> > > > > > > > >> two.
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence
> > "Broker
> > > > > also
> > > > > > > > >> stores the
> > > > > > > > >> >> > > > > > > > DelegationToken
> > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a
> bit
> > > > > more? I
> > > > > > > > >> thought the
> > > > > > > > >> >> > > > > > > delegation
> > > > > > > > >> >> > > > > > > > > token is the hmac.
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > Thanks,
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > Jun
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun
> Rao
> > <
> > > > > > > > >> jun@confluent.io>
> > > > > > > > >> >> > > > wrote:
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > > > > > >> >> > > > > > > > > >
> > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite.
> We
> > > can
> > > > > > > discuss
> > > > > > > > >> this in
> > > > > > > > >> >> > > the
> > > > > > > > >> >> > > > > > > meeting
> > > > > > > > >> >> > > > > > > > > > tomorrow.
> > > > > > > > >> >> > > > > > > > > >
> > > > > > > > >> >> > > > > > > > > > Thanks,
> > > > > > > > >> >> > > > > > > > > >
> > > > > > > > >> >> > > > > > > > > > Jun
> > > > > > > > >> >> > > > > > > > > >
> > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM,
> > Harsha <
> > > > > > > > >> kafka@harsha.io>
> > > > > > > > >> >> > > > wrote:
> > > > > > > > >> >> > > > > > > > > >
> > > > > > > > >> >> > > > > > > > > >> Hi All,
> > > > > > > > >> >> > > > > > > > > >> Can we have a KIP meeting
> > > > around
> > > > > > > this.
> > > > > > > > >> The KIP
> > > > > > > > >> >> > is
> > > > > > > > >> >> > > > up
> > > > > > > > >> >> > > > > > for
> > > > > > > > >> >> > > > > > > > > >> sometime and if there are
> > any
> > > > > > > questions
> > > > > > > > >> lets
> > > > > > > > >> >> > > > quickly
> > > > > > > > >> >> > > > > > hash
> > > > > > > > >> >> > > > > > > > out
> > > > > > > > >> >> > > > > > > > > >> details.
> > > > > > > > >> >> > > > > > > > > >>
> > > > > > > > >> >> > > > > > > > > >> Thanks,
> > > > > > > > >> >> > > > > > > > > >> Harsha
> > > > > > > > >> >> > > > > > > > > >>
> > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM,
> > parth
> > > > > > > > brahmbhatt
> > > > > > > > >> wrote:
> > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo
> system
> > > uses
> > > > > so
> > > > > > no
> > > > > > > > >> good
> > > > > > > > >> >> > reason
> > > > > > > > >> >> > > > > > really.
> > > > > > > > >> >> > > > > > > > We
> > > > > > > > >> >> > > > > > > > > >> > could
> > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the
> newest
> > > > > > > recommended
> > > > > > > > >> standard
> > > > > > > > >> >> > > is.
> > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > >> >> > > > > > > > > >> > Thanks
> > > > > > > > >> >> > > > > > > > > >> > Parth
> > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM,
> > > Ismael
> > > > > > Juma <
> > > > > > > > >> >> > > > > ismael@juma.me.uk
> > > > > > > > >> >> > > > > > >
> > > > > > > > >> >> > > > > > > > > wrote:
> > > > > > > > >> >> > > > > > > > > >> >
> > > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only
> started
> > > > > > reviewing
> > > > > > > > >> this and
> > > > > > > > >> >> > > may
> > > > > > > > >> >> > > > > have
> > > > > > > > >> >> > > > > > > > > >> additional
> > > > > > > > >> >> > > > > > > > > >> > > questions later. The immediate
> > > > question
> > > > > > that
> > > > > > > > >> came to
> > > > > > > > >> >> > > mind
> > > > > > > > >> >> > > > is
> > > > > > > > >> >> > > > > > our
> > > > > > > > >> >> > > > > > > > > >> choice of
> > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's
> > marked
> > > > as
> > > > > > > > >> OBSOLETE in
> > > > > > > > >> >> > the
> > > > > > > > >> >> > > > IANA
> > > > > > > > >> >> > > > > > > > > Registry
> > > > > > > > >> >> > > > > > > > > >> of
> > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original
> > RFC
> > > > > > (2831)
> > > > > > > > has
> > > > > > > > >> been
> > > > > > > > >> >> > > moved
> > > > > > > > >> >> > > > > to
> > > > > > > > >> >> > > > > > > > > Historic
> > > > > > > > >> >> > > > > > > > > >> > > status:
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/
> > rfc6331
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > >
> > > > > > > > >> >> > >
> > > > > > > > >>
> > > > > >
> > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind
> that
> > > > > choice?
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > Thanks,
> > > > > > > > >> >> > > > > > > > > >> > > Ismael
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29
> PM,
> > > Gwen
> > > > > > > > Shapira <
> > > > > > > > >> >> > > > > > > gwen@confluent.io
> > > > > > > > >> >> > > > > > > > >
> > > > > > > > >> >> > > > > > > > > >> wrote:
> > > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that
> > even
> > > > > though
> > > > > > > > >> delegation
> > > > > > > > >> >> > > > tokens
> > > > > > > > >> >> > > > > > > are a
> > > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very
> > strongly
> > > > > about
> > > > > > > not
> > > > > > > > >> adding
> > > > > > > > >> >> > > > > > dependency
> > > > > > > > >> >> > > > > > > > on
> > > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > > >> >> > > > > > > > > >> > > > > when implementing delegation
> > > > tokens
> > > > > > for
> > > > > > > > >> Kafka. The
> > > > > > > > >> >> > > KIP
> > > > > > > > >> >> > > > > > > doesn't
> > > > > > > > >> >> > > > > > > > > >> imply
> > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if you
> > can
> > > > > > > clarify...
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP
> so
> > > no
> > > > > one
> > > > > > > will
> > > > > > > > >> read
> > > > > > > > >> >> > the
> > > > > > > > >> >> > > > KIP
> > > > > > > > >> >> > > > > > and
> > > > > > > > >> >> > > > > > > > > panic
> > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
> > > > release...
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation
> token
> > at
> > > > any
> > > > > > > time
> > > > > > > > >> after
> > > > > > > > >> >> > > > > > > > authenticating?
> > > > > > > > >> >> > > > > > > > > >> only
> > > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> > > authenticated
> > > > > you
> > > > > > > can
> > > > > > > > >> get
> > > > > > > > >> >> > > > delegation
> > > > > > > > >> >> > > > > > > > tokens.
> > > > > > > > >> >> > > > > > > > > >> We
> > > > > > > > >> >> > > > > > > > > >> > > need
> > > > > > > > >> >> > > > > > > > > >> > > > to
> > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> > > authenticated
> > > > > > using
> > > > > > > > >> delegation
> > > > > > > > >> >> > > > > token,
> > > > > > > > >> >> > > > > > > can
> > > > > > > > >> >> > > > > > > > > also
> > > > > > > > >> >> > > > > > > > > >> > > > acquire
> > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or
> not.
> > > > Also
> > > > > > > there
> > > > > > > > >> is the
> > > > > > > > >> >> > > > > question
> > > > > > > > >> >> > > > > > of
> > > > > > > > >> >> > > > > > > > do
> > > > > > > > >> >> > > > > > > > > we
> > > > > > > > >> >> > > > > > > > > >> > > allow
> > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation
> > > token
> > > > > or
> > > > > > we
> > > > > > > > >> want
> > > > > > > > >> >> > > specific
> > > > > > > > >> >> > > > > > ACLs
> > > > > > > > >> >> > > > > > > (I
> > > > > > > > >> >> > > > > > > > > >> think
> > > > > > > > >> >> > > > > > > > > >> > > its
> > > > > > > > >> >> > > > > > > > > >> > > > an
> > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
> > overkill.
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
> > > options:
> > > > > > > Either
> > > > > > > > >> require
> > > > > > > > >> >> > > > > Kerberos
> > > > > > > > >> >> > > > > > > > auth
> > > > > > > > >> >> > > > > > > > > >> for
> > > > > > > > >> >> > > > > > > > > >> > > > renewal or require non-owners
> to
> > > > > renew.
> > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is
> simpler
> > > (it
> > > > > > > > basically
> > > > > > > > >> >> > require
> > > > > > > > >> >> > > a
> > > > > > > > >> >> > > > > "job
> > > > > > > > >> >> > > > > > > > > master"
> > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for the
> > > > > renewal,
> > > > > > it
> > > > > > > > >> will have
> > > > > > > > >> >> > > its
> > > > > > > > >> >> > > > > own
> > > > > > > > >> >> > > > > > > > > >> identity
> > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is the
> > > > correct
> > > > > > > > design
> > > > > > > > >> >> > pattern
> > > > > > > > >> >> > > > > > anyway.
> > > > > > > > >> >> > > > > > > > For
> > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
> > > > coordinate
> > > > > > > > >> renewals?),
> > > > > > > > >> >> > but
> > > > > > > > >> >> > > > it
> > > > > > > > >> >> > > > > is
> > > > > > > > >> >> > > > > > > > hard
> > > > > > > > >> >> > > > > > > > > to
> > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without
> > looking
> > > at
> > > > > the
> > > > > > > > code
> > > > > > > > >> >> > changes
> > > > > > > > >> >> > > > > > > required.
> > > > > > > > >> >> > > > > > > > If
> > > > > > > > >> >> > > > > > > > > >> you
> > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the
> "require
> > > > > > Kerberos"
> > > > > > > > >> will look
> > > > > > > > >> >> > > in
> > > > > > > > >> >> > > > > > Kafka
> > > > > > > > >> >> > > > > > > > > code,
> > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is that
> > > tokens
> > > > > will
> > > > > > > > >> propagate
> > > > > > > > >> >> > via
> > > > > > > > >> >> > > > ZK
> > > > > > > > >> >> > > > > > but
> > > > > > > > >> >> > > > > > > > > >> without
> > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> > > > UpdateMetadata
> > > > > > > > >> protocol,
> > > > > > > > >> >> > > > correct?
> > > > > > > > >> >> > > > > > > > Clients
> > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on
> SASL
> > > auth
> > > > > > > failure
> > > > > > > > >> (IIRC),
> > > > > > > > >> >> > > but
> > > > > > > > >> >> > > > > > since
> > > > > > > > >> >> > > > > > > > the
> > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between
> > brokers
> > > > > > asynch,
> > > > > > > > we
> > > > > > > > >> will
> > > > > > > > >> >> > > need
> > > > > > > > >> >> > > > to
> > > > > > > > >> >> > > > > > > > retry a
> > > > > > > > >> >> > > > > > > > > >> bit
> > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing
> auth
> > > due
> > > > to
> > > > > > > > timing
> > > > > > > > >> >> > issues.
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
> > alternatives
> > > > > right
> > > > > > > > now.
> > > > > > > > >> The
> > > > > > > > >> >> > > > current
> > > > > > > > >> >> > > > > > > > > documented
> > > > > > > > >> >> > > > > > > > > >> > > > approach
> > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it
> does
> > > not
> > > > > > > require
> > > > > > > > >> any
> > > > > > > > >> >> > > changes
> > > > > > > > >> >> > > > > to
> > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
> > > approach
> > > > > can
> > > > > > > > remove
> > > > > > > > >> >> > > zookeeper
> > > > > > > > >> >> > > > > > > > > dependency
> > > > > > > > >> >> > > > > > > > > >> as
> > > > > > > > >> >> > > > > > > > > >> > > well
> > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in
> KIP
> > > > > > > discussion
> > > > > > > > >> call.*
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do
> you
> > > > want
> > > > > to
> > > > > > > > ping
> > > > > > > > >> Jun to
> > > > > > > > >> >> > > > > > arrange a
> > > > > > > > >> >> > > > > > > > > call?
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's
> suggestion
> > of
> > > > > > having
> > > > > > > > >> just the
> > > > > > > > >> >> > > > > > controller
> > > > > > > > >> >> > > > > > > > > issue
> > > > > > > > >> >> > > > > > > > > >> the
> > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid
> > > > syncing
> > > > > a
> > > > > > > > shared
> > > > > > > > >> >> > secret.
> > > > > > > > >> >> > > > Not
> > > > > > > > >> >> > > > > > > sure
> > > > > > > > >> >> > > > > > > > if
> > > > > > > > >> >> > > > > > > > > >> we
> > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
> > discussion
> > > > here
> > > > > > or
> > > > > > > on
> > > > > > > > >> the
> > > > > > > > >> >> > > wiki. I
> > > > > > > > >> >> > > > > > think
> > > > > > > > >> >> > > > > > > > > that
> > > > > > > > >> >> > > > > > > > > >> we
> > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem of
> > > "token
> > > > > > > > >> distribution"
> > > > > > > > >> >> > > from
> > > > > > > > >> >> > > > > > > "shared
> > > > > > > > >> >> > > > > > > > > >> secret
> > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
> > > > controller
> > > > > > as
> > > > > > > > the
> > > > > > > > >> only
> > > > > > > > >> >> > > token
> > > > > > > > >> >> > > > > > > > generator
> > > > > > > > >> >> > > > > > > > > >> to
> > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue,
> while
> > > > still
> > > > > > > using
> > > > > > > > >> ZK async
> > > > > > > > >> >> > > to
> > > > > > > > >> >> > > > > > > > distribute
> > > > > > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the
> previous
> > > > Email
> > > > > I
> > > > > > am
> > > > > > > > >> fine with
> > > > > > > > >> >> > > > that
> > > > > > > > >> >> > > > > > > > approach
> > > > > > > > >> >> > > > > > > > > >> as
> > > > > > > > >> >> > > > > > > > > >> > > long
> > > > > > > > >> >> > > > > > > > > >> > > > as
> > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
> > > complexity
> > > > > of
> > > > > > > > >> >> > > adding/updating
> > > > > > > > >> >> > > > > APIS
> > > > > > > > >> >> > > > > > > > adds
> > > > > > > > >> >> > > > > > > > > >> enough
> > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with
> the
> > > > > > controller
> > > > > > > > >> approach
> > > > > > > > >> >> > is
> > > > > > > > >> >> > > > > > secret
> > > > > > > > >> >> > > > > > > > > >> rotation
> > > > > > > > >> >> > > > > > > > > >> > > can
> > > > > > > > >> >> > > > > > > > > >> > > > be
> > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and would
> > not
> > > > > > require
> > > > > > > > >> >> > > deployment. *
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> > > complexity
> > > > > (or
> > > > > > > > point
> > > > > > > > >> me to
> > > > > > > > >> >> > > the
> > > > > > > > >> >> > > > > > email
> > > > > > > > >> >> > > > > > > I
> > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
> > > required?
> > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients
> > can
> > > > > > already
> > > > > > > > >> find the
> > > > > > > > >> >> > > > > > controller
> > > > > > > > >> >> > > > > > > > from
> > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
> > concerned
> > > > > about
> > > > > > > > >> controller
> > > > > > > > >> >> > > > load.
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of
> > > forcing
> > > > > > > > kerberos
> > > > > > > > >> auth
> > > > > > > > >> >> > for
> > > > > > > > >> >> > > > > > > renwal, I
> > > > > > > > >> >> > > > > > > > > >> think
> > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer
> > the
> > > > the
> > > > > > > > request
> > > > > > > > >> >> > content
> > > > > > > > >> >> > > > in
> > > > > > > > >> >> > > > > a
> > > > > > > > >> >> > > > > > > > pretty
> > > > > > > > >> >> > > > > > > > > >> ugly
> > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting
> renewer
> > to
> > > > > > > non-owner
> > > > > > > > >> is
> > > > > > > > >> >> > > better.
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary
> > > evil.
> > > > > > While
> > > > > > > > >> this will
> > > > > > > > >> >> > > make
> > > > > > > > >> >> > > > > the
> > > > > > > > >> >> > > > > > > > kafka
> > > > > > > > >> >> > > > > > > > > >> code
> > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
> > > forcing
> > > > > > > renewer
> > > > > > > > >> to
> > > > > > > > >> >> > > > non-owner
> > > > > > > > >> >> > > > > > > pushes
> > > > > > > > >> >> > > > > > > > > >> the code
> > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes
> > it
> > > > even
> > > > > > > > harder
> > > > > > > > >> to
> > > > > > > > >> >> > > > > > integrate. *
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't
> > think
> > > > the
> > > > > > > > >> "renewal by
> > > > > > > > >> >> > > > other"
> > > > > > > > >> >> > > > > > > > approach
> > > > > > > > >> >> > > > > > > > > >> is
> > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we
> > > expect
> > > > to
> > > > > > use
> > > > > > > > >> >> > delegation
> > > > > > > > >> >> > > > > tokens
> > > > > > > > >> >> > > > > > > > since
> > > > > > > > >> >> > > > > > > > > >> > > > they will have an app-master
> of
> > > some
> > > > > > sort
> > > > > > > > who
> > > > > > > > >> >> > > requested
> > > > > > > > >> >> > > > > the
> > > > > > > > >> >> > > > > > > > token
> > > > > > > > >> >> > > > > > > > > to
> > > > > > > > >> >> > > > > > > > > >> > > > begin with.
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > The response for my question
> > on
> > > > how
> > > > > > > > multiple
> > > > > > > > >> >> > > > identities
> > > > > > > > >> >> > > > > > will
> > > > > > > > >> >> > > > > > > > be
> > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear
> to
> > > me -
> > > > > > > AFAIK,
> > > > > > > > >> we don't
> > > > > > > > >> >> > > > > > > > authenticate
> > > > > > > > >> >> > > > > > > > > >> each
> > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> > > > > connections.
> > > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate
> connections,
> > > and
> > > > > only
> > > > > > > > when
> > > > > > > > >> they
> > > > > > > > >> >> > are
> > > > > > > > >> >> > > > > being
> > > > > > > > >> >> > > > > > > > > >> established.
> > > > > > > > >> >> > > > > > > > > >> > > > Let
> > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a
> > > > question,
> > > > > > in
> > > > > > > > >> absence of
> > > > > > > > >> >> > > > > > > delegation
> > > > > > > > >> >> > > > > > > > > >> tokens if
> > > > > > > > >> >> > > > > > > > > >> > > > we
> > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use case
> > > using
> > > > > user
> > > > > > > > >> TGT's how
> > > > > > > > >> >> > > would
> > > > > > > > >> >> > > > > we
> > > > > > > > >> >> > > > > > do
> > > > > > > > >> >> > > > > > > > it?
> > > > > > > > >> >> > > > > > > > > >> My
> > > > > > > > >> >> > > > > > > > > >> > > point
> > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no different
> > > with
> > > > > > > > delegation
> > > > > > > > >> >> > tokens.
> > > > > > > > >> >> > > > The
> > > > > > > > >> >> > > > > > use
> > > > > > > > >> >> > > > > > > > > case
> > > > > > > > >> >> > > > > > > > > >> you
> > > > > > > > >> >> > > > > > > > > >> > > are
> > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > > > > > impersonation.*
> > > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of
> the
> > > > things
> > > > > > > that
> > > > > > > > >> >> > del
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Ashish Singh <as...@cloudera.com>.
Hello Harsha,
Are you still working on this? Wondering if we can discuss this in next KIP
meeting, if you can join.
On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi Grant,
> We are working on it. Will add the details to KIP about the
> request protocol.
>
> Thanks,
> Harsha
>
> On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com> wrote:
>
> > Hi Parth,
> >
> > Are you still working on this? If you need any help please don't hesitate
> > to ask.
> >
> > Thanks,
> > Grant
> >
> > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Parth,
> > >
> > > Thanks for the reply.
> > >
> > > It makes sense to only allow the renewal by users that authenticated
> > using
> > > *non* delegation token mechanism. Then, should we make the renewal a
> > list?
> > > For example, in the case of rest proxy, it will be useful for every
> > > instance of rest proxy to be able to renew the tokens.
> > >
> > > It would be clearer if we can document the request protocol like
> > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 4+-+Command+line+and+centralized+administrative+operations#KIP-4-
> Commandlineandcentralizedadministrativeoperations-
> CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> > > .
> > >
> > > It would also be useful to document the client APIs.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > > brahmbhatt.parth@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I am suggesting that we will only allow the renewal by users that
> > > > authenticated using *non* delegation token mechanism. For example, If
> > > user
> > > > Alice authenticated using kerberos and requested delegation tokens,
> > only
> > > > user Alice authenticated via non delegation token mechanism can
> renew.
> > > > Clients that have access to delegation tokens can not issue renewal
> > > > request for renewing their own token and this is primarily important
> to
> > > > reduce the time window for which a compromised token will be valid.
> > > >
> > > > To clarify, Yes any authenticated user can request delegation tokens
> > but
> > > > even here I would recommend to avoid creating a chain where a client
> > > > authenticated via delegation token request for more delegation
> tokens.
> > > > Basically anyone can request delegation token, as long as they
> > > authenticate
> > > > via a non delegation token mechanism.
> > > >
> > > > Aren't classes listed here
> > > > <
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKaf
> ka-PublicInterfaces
> > > > >
> > > > sufficient?
> > > >
> > > > Thanks
> > > > Parth
> > > >
> > > >
> > > >
> > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Parth,
> > > > >
> > > > > Thanks for the reply. A couple of comments inline below.
> > > > >
> > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > >
> > > > > > 1. Who / how are tokens renewed? By original requester only? or
> > using
> > > > > > Kerberos
> > > > > > auth only?
> > > > > > My recommendation is to do this only using Kerberos auth and only
> > > threw
> > > > > the
> > > > > > renewer specified during the acquisition request.
> > > > > >
> > > > > >
> > > > > Hmm, not sure that I follow this. Are you saying that any client
> > > > > authenticated with the delegation token can renew, i.e. there is no
> > > > renewer
> > > > > needed?
> > > > >
> > > > > Also, just to be clear, any authenticated client (either through
> SASL
> > > or
> > > > > SSL) can request a delegation token for the authenticated user,
> > right?
> > > > >
> > > > >
> > > > > > 2. Are tokens stored on each broker or in ZK?
> > > > > > My recommendation is still to store in ZK or not store them at
> all.
> > > The
> > > > > > whole controller based distribution is too much overhead with not
> > > much
> > > > to
> > > > > > achieve.
> > > > > >
> > > > > > 3. How are tokens invalidated / expired?
> > > > > > Either by expiration time out or through an explicit request to
> > > > > invalidate.
> > > > > >
> > > > > > 4. Which encryption algorithm is used?
> > > > > > SCRAM
> > > > > >
> > > > > > 5. What is the impersonation proposal (it wasn't in the KIP but
> was
> > > > > > discussed
> > > > > > in this thread)?
> > > > > > There is no imperonation proposal. I tried and explained how its
> a
> > > > > > different problem and why its not really necessary to discuss
> that
> > as
> > > > > part
> > > > > > of this KIP. This KIP will not support any impersonation, it
> will
> > > just
> > > > > be
> > > > > > another way to authenticate.
> > > > > >
> > > > > > 6. Do we need new ACLs, if so - for what actions?
> > > > > > We do not need new ACLs.
> > > > > >
> > > > > >
> > > > > Could we document the format of the new request/response and their
> > > > > associated Resource and Operation for ACL?
> > > > >
> > > > >
> > > > > > 7. How would the delegation token be configured in the client?
> > > > > > Should be through config. I wasn't planning on supporting JAAS
> for
> > > > > tokens.
> > > > > > I don't believe hadoop does this either.
> > > > > >
> > > > > > Thanks
> > > > > > Parth
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Harsha,
> > > > > > >
> > > > > > > Another question.
> > > > > > >
> > > > > > > 9. How would the delegation token be configured in the client?
> > The
> > > > > > standard
> > > > > > > way is to do this through JAAS. However, we will need to think
> > > > through
> > > > > if
> > > > > > > this is convenient in a shared environment. For example, when a
> > new
> > > > > task
> > > > > > is
> > > > > > > added to a Storm worker node, do we need to dynamically add a
> new
> > > > > section
> > > > > > > in the JAAS file? It may be more convenient if we can pass in
> the
> > > > token
> > > > > > > through the config directly w/o going through JAAS.
> > > > > > >
> > > > > > > Are you or Parth still actively working on this KIP?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > > > Just to add on that list.
> > > > > > > >
> > > > > > > > 2. It would be good to document the format of the data stored
> > in
> > > > ZK.
> > > > > > > > 7. Earlier, there was a discussion on whether the tokens
> should
> > > be
> > > > > > > > propagated through ZK like config/acl/quota, or through the
> > > > > controller.
> > > > > > > > Currently, the controller is only designed for propagating
> > topic
> > > > > > > metadata,
> > > > > > > > but not other data.
> > > > > > > > 8. Should we use SCRAM to send the token instead of
> DIGEST-MD5
> > > > since
> > > > > > it's
> > > > > > > > deprecated?
> > > > > > > >
> > > > > > > > Also, the images in the wiki seem broken.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> > > gwen@confluent.io>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> From what I can see, remaining questions are:
> > > > > > > >>
> > > > > > > >> 1. Who / how are tokens renewed? By original requester only?
> > or
> > > > > using
> > > > > > > >> Kerberos auth only?
> > > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > > > > >> 3. How are tokens invalidated / expired?
> > > > > > > >> 4. Which encryption algorithm is used?
> > > > > > > >> 5. What is the impersonation proposal (it wasn't in the KIP
> > but
> > > > was
> > > > > > > >> discussed in this thread)?
> > > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > > > > >>
> > > > > > > >> Gwen
> > > > > > > >>
> > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io>
> > wrote:
> > > > > > > >> > Jun & Ismael,
> > > > > > > >> > Unfortunately I couldn't attend
> the
> > > KIP
> > > > > > > meeting
> > > > > > > >> > when delegation tokens discussed.
> > > > > > Appreciate
> > > > > > > if
> > > > > > > >> > you can update the thread if you
> > have
> > > > any
> > > > > > > >> > further questions.
> > > > > > > >> > Thanks,
> > > > > > > >> > Harsha
> > > > > > > >> >
> > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > > > > >> >> It seems that the links to images in the KIP are broken.
> > > > > > > >> >>
> > > > > > > >> >> Liquan
> > > > > > > >> >>
> > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > > > > >> >> > In the current proposal we only allow a user to get
> > > > delegation
> > > > > > > token
> > > > > > > >> for
> > > > > > > >> >> > the identity that it authenticated as using another
> > > > mechanism,
> > > > > > i.e.
> > > > > > > >> A user
> > > > > > > >> >> > that authenticate using a keytab for principal
> > > > > user1@EXAMPLE.COM
> > > > > > > >> will get
> > > > > > > >> >> > delegation tokens for that user only. In future I think
> > we
> > > > will
> > > > > > > have
> > > > > > > >> to
> > > > > > > >> >> > extend support such that we allow some set of users (
> > > > > > > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM)
> > to
> > > > > > acquire
> > > > > > > >> >> > delegation tokens on behalf of other users whose
> identity
> > > > they
> > > > > > have
> > > > > > > >> >> > verified independently. Kafka brokers will have ACLs
> to
> > > > > control
> > > > > > > >> which
> > > > > > > >> >> > users are allowed to impersonate other users and get
> > tokens
> > > > on
> > > > > > > >> behalf of
> > > > > > > >> >> > them. Overall Impersonation is a whole different
> problem
> > in
> > > > my
> > > > > > > >> opinion and
> > > > > > > >> >> > I think we can tackle it in separate KIP.
> > > > > > > >> >> >
> > > > > > > >> >> > 111. What's the typical rate of getting and renewing
> > > > delegation
> > > > > > > >> tokens?
> > > > > > > >> >> > Typically this should be very very low, 1 request per
> > > minute
> > > > > is a
> > > > > > > >> >> > relatively high estimate. However it depends on the
> token
> > > > > > > >> expiration. I am
> > > > > > > >> >> > less worried about the extra load it puts on controller
> > vs
> > > > the
> > > > > > > added
> > > > > > > >> >> > complexity and the value it offers.
> > > > > > > >> >> >
> > > > > > > >> >> > Thanks
> > > > > > > >> >> > Parth
> > > > > > > >> >> >
> > > > > > > >> >> >
> > > > > > > >> >> >
> > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > > > > ismael@juma.me.uk>
> > > > > > > >> wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > Thanks Rajini. It would probably require a separate
> KIP
> > > as
> > > > it
> > > > > > > will
> > > > > > > >> >> > > introduce user visible changes. We could also update
> > > KIP-48
> > > > > to
> > > > > > > >> have this
> > > > > > > >> >> > > information, but it seems cleaner to do it
> separately.
> > We
> > > > can
> > > > > > > >> discuss
> > > > > > > >> >> > that
> > > > > > > >> >> > > in the KIP call today.
> > > > > > > >> >> > >
> > > > > > > >> >> > > Ismael
> > > > > > > >> >> > >
> > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > > Ismael,
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > I have created a JIRA (
> > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that
> need
> > > > > another
> > > > > > > >> KIP? If
> > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just be a
> > JIRA
> > > > > that
> > > > > > > gets
> > > > > > > >> >> > > reviewed
> > > > > > > >> >> > > > when the PR is ready?
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Thank you,
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Rajini
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > > > > ismael@juma.me.uk>
> > > > > > > >> >> > wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > Gwen had independently mentioned this as a SASL
> > > > mechanism
> > > > > > > that
> > > > > > > >> might
> > > > > > > >> >> > be
> > > > > > > >> >> > > > > useful for Kafka and I have been meaning to check
> > it
> > > in
> > > > > > more
> > > > > > > >> detail.
> > > > > > > >> >> > > Good
> > > > > > > >> >> > > > > to know that you are willing to contribute an
> > > > > > implementation.
> > > > > > > >> Maybe
> > > > > > > >> >> > we
> > > > > > > >> >> > > > > should file a separate JIRA for this?
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > Ismael
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> > > > > > Mechanism)
> > > > > > > >> is a
> > > > > > > >> >> > > better
> > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come
> > with a
> > > > > > > built-in
> > > > > > > >> SCRAM
> > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy
> to
> > > add
> > > > > > > support
> > > > > > > >> in
> > > > > > > >> >> > Kafka
> > > > > > > >> >> > > > > since
> > > > > > > >> >> > > > > > it would be a useful mechanism to support
> anyway.
> > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes
> > the
> > > > > > protocol
> > > > > > > >> for
> > > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > > > > jun@confluent.io
> > > > > > > >
> > > > > > > >> wrote:
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > > Parth,
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of more
> > > > > questions.
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > 111. What's the typical rate of getting and
> > > > renewing
> > > > > > > >> delegation
> > > > > > > >> >> > > > tokens?
> > > > > > > >> >> > > > > > > That may have an impact on whether they
> should
> > be
> > > > > > > directed
> > > > > > > >> to the
> > > > > > > >> >> > > > > > > controller.
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > Jun
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
> > > brahmbhatt <
> > > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > > Hi Jun,
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > Thanks for reviewing.
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > * We could add a Cluster action to add acls
> > on
> > > > who
> > > > > > can
> > > > > > > >> request
> > > > > > > >> >> > > > > > delegation
> > > > > > > >> >> > > > > > > > tokens. I don't see the use case for that
> yet
> > > but
> > > > > > down
> > > > > > > >> the line
> > > > > > > >> >> > > > when
> > > > > > > >> >> > > > > we
> > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs it
> will
> > > be
> > > > > > > >> necessary.
> > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > > > > used/distributed
> > > > > > > >> over
> > > > > > > >> >> > secure
> > > > > > > >> >> > > > > > > channels.
> > > > > > > >> >> > > > > > > > * Depending on what design we end up
> choosing
> > > > > > > >> Invalidation will
> > > > > > > >> >> > > be
> > > > > > > >> >> > > > > > > > responsibility of every broker or
> controller.
> > > > > > > >> >> > > > > > > > * I am not sure if I documented somewhere
> > that
> > > > > > > >> invalidation
> > > > > > > >> >> > will
> > > > > > > >> >> > > > > > directly
> > > > > > > >> >> > > > > > > > go through zookeeper but that is not the
> > > intent.
> > > > > > > >> Invalidation
> > > > > > > >> >> > > will
> > > > > > > >> >> > > > > > either
> > > > > > > >> >> > > > > > > > be request based or due to expiration. No
> > > direct
> > > > > > > >> zookeeper
> > > > > > > >> >> > > > > interaction
> > > > > > > >> >> > > > > > > from
> > > > > > > >> >> > > > > > > > any client.
> > > > > > > >> >> > > > > > > > * "Broker also stores the DelegationToken
> > > without
> > > > > the
> > > > > > > >> hmac in
> > > > > > > >> >> > the
> > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion.
> The
> > > sole
> > > > > > > >> purpose of
> > > > > > > >> >> > > > > zookeeper
> > > > > > > >> >> > > > > > in
> > > > > > > >> >> > > > > > > > this design is as distribution channel for
> > > tokens
> > > > > > > >> between all
> > > > > > > >> >> > > > brokers
> > > > > > > >> >> > > > > > > and a
> > > > > > > >> >> > > > > > > > layer that ensures only tokens that were
> > > > generated
> > > > > by
> > > > > > > >> making a
> > > > > > > >> >> > > > > request
> > > > > > > >> >> > > > > > > to a
> > > > > > > >> >> > > > > > > > broker will be accepted (more on this in
> > second
> > > > > > > >> paragraph). The
> > > > > > > >> >> > > > token
> > > > > > > >> >> > > > > > > > consists of few elements (owner, renewer,
> > uuid
> > > ,
> > > > > > > >> expiration,
> > > > > > > >> >> > > hmac)
> > > > > > > >> >> > > > ,
> > > > > > > >> >> > > > > > one
> > > > > > > >> >> > > > > > > of
> > > > > > > >> >> > > > > > > > which is the finally generated hmac but
> hmac
> > it
> > > > > self
> > > > > > is
> > > > > > > >> >> > derivable
> > > > > > > >> >> > > > if
> > > > > > > >> >> > > > > > you
> > > > > > > >> >> > > > > > > > have all the other elements of the token +
> > > secret
> > > > > key
> > > > > > > to
> > > > > > > >> >> > generate
> > > > > > > >> >> > > > > hmac.
> > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL
> support
> > we
> > > > do
> > > > > > not
> > > > > > > >> want the
> > > > > > > >> >> > > > > entire
> > > > > > > >> >> > > > > > > > token to be wire transferred to zookeeper
> as
> > > that
> > > > > > will
> > > > > > > >> be an
> > > > > > > >> >> > > > insecure
> > > > > > > >> >> > > > > > > wire
> > > > > > > >> >> > > > > > > > transfer. Instead we only store all the
> other
> > > > > > elements
> > > > > > > >> of a
> > > > > > > >> >> > > > > delegation
> > > > > > > >> >> > > > > > > > tokens. Brokers can read these elements and
> > > > because
> > > > > > > they
> > > > > > > >> also
> > > > > > > >> >> > > have
> > > > > > > >> >> > > > > > access
> > > > > > > >> >> > > > > > > > to secret key they will be able to generate
> > > hmac
> > > > on
> > > > > > > >> their end.
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > One of the alternative proposed is to avoid
> > > > > zookeeper
> > > > > > > >> >> > > altogether. A
> > > > > > > >> >> > > > > > > Client
> > > > > > > >> >> > > > > > > > will call broker with required information
> > > > (owner,
> > > > > > > >> renwer,
> > > > > > > >> >> > > > > expiration)
> > > > > > > >> >> > > > > > > and
> > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't
> > > store
> > > > > this
> > > > > > > in
> > > > > > > >> >> > > zookeeper.
> > > > > > > >> >> > > > > > From
> > > > > > > >> >> > > > > > > > this point a client can contact any broker
> > with
> > > > all
> > > > > > the
> > > > > > > >> >> > > delegation
> > > > > > > >> >> > > > > > token
> > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac,
> uuid)
> > > the
> > > > > > borker
> > > > > > > >> will
> > > > > > > >> >> > > > > regenerate
> > > > > > > >> >> > > > > > > the
> > > > > > > >> >> > > > > > > > hmac and as long as it matches with hmac
> > > > presented
> > > > > by
> > > > > > > >> client ,
> > > > > > > >> >> > > > broker
> > > > > > > >> >> > > > > > > will
> > > > > > > >> >> > > > > > > > allow the request to authenticate. Only
> > > problem
> > > > > with
> > > > > > > >> this
> > > > > > > >> >> > > approach
> > > > > > > >> >> > > > > is
> > > > > > > >> >> > > > > > if
> > > > > > > >> >> > > > > > > > the secret key is compromised any client
> can
> > > now
> > > > > > > generate
> > > > > > > >> >> > random
> > > > > > > >> >> > > > > tokens
> > > > > > > >> >> > > > > > > and
> > > > > > > >> >> > > > > > > > they will still be able to authenticate as
> > any
> > > > user
> > > > > > > they
> > > > > > > >> like.
> > > > > > > >> >> > > with
> > > > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens
> > > acquired
> > > > > via
> > > > > > a
> > > > > > > >> broker
> > > > > > > >> >> > > > (using
> > > > > > > >> >> > > > > > some
> > > > > > > >> >> > > > > > > > auth scheme other than delegation token)
> will
> > > be
> > > > > > > >> accepted. We
> > > > > > > >> >> > > need
> > > > > > > >> >> > > > to
> > > > > > > >> >> > > > > > > > discuss which proposal makes more sense and
> > we
> > > > can
> > > > > go
> > > > > > > >> over it
> > > > > > > >> >> > in
> > > > > > > >> >> > > > > > > tomorrow's
> > > > > > > >> >> > > > > > > > meeting.
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > Thanks
> > > > > > > >> >> > > > > > > > Parth
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > > > > > > >> jun@confluent.io>
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > > > >
> > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 100. This potentially can be useful for
> > Kafka
> > > > > > Connect
> > > > > > > >> and
> > > > > > > >> >> > Kafka
> > > > > > > >> >> > > > > rest
> > > > > > > >> >> > > > > > > > proxy
> > > > > > > >> >> > > > > > > > > where a worker agent will need to run a
> > task
> > > on
> > > > > > > behalf
> > > > > > > >> of a
> > > > > > > >> >> > > > client.
> > > > > > > >> >> > > > > > We
> > > > > > > >> >> > > > > > > > will
> > > > > > > >> >> > > > > > > > > likely need to change how those services
> > use
> > > > > Kafka
> > > > > > > >> clients
> > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a shared
> > > client
> > > > > per
> > > > > > > >> worker,
> > > > > > > >> >> > we
> > > > > > > >> >> > > > will
> > > > > > > >> >> > > > > > > need
> > > > > > > >> >> > > > > > > > a
> > > > > > > >> >> > > > > > > > > client per user task since the
> > authentication
> > > > > > happens
> > > > > > > >> at the
> > > > > > > >> >> > > > > > connection
> > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer
> will
> > be
> > > > the
> > > > > > > >> workers.
> > > > > > > >> >> > So,
> > > > > > > >> >> > > we
> > > > > > > >> >> > > > > > > > probably
> > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For
> Kafka
> > > rest
> > > > > > > proxy,
> > > > > > > >> the
> > > > > > > >> >> > > > renewer
> > > > > > > >> >> > > > > > can
> > > > > > > >> >> > > > > > > > > probably just be the creator of the
> token.
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can
> request
> > > > > > delegation
> > > > > > > >> tokens?
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
> > > delegation
> > > > > > tokens
> > > > > > > >> in an
> > > > > > > >> >> > > > > encrypted
> > > > > > > >> >> > > > > > > > > channel?
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
> > tokens,
> > > > > every
> > > > > > > >> broker?
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would it be
> > > > better
> > > > > to
> > > > > > > do
> > > > > > > >> it in
> > > > > > > >> >> > a
> > > > > > > >> >> > > > > > request
> > > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 105. The terminology of client in the
> wiki
> > > > > > sometimes
> > > > > > > >> refers
> > > > > > > >> >> > to
> > > > > > > >> >> > > > the
> > > > > > > >> >> > > > > > end
> > > > > > > >> >> > > > > > > > > client and some other times refers to the
> > > > client
> > > > > > > using
> > > > > > > >> the
> > > > > > > >> >> > > > > delegation
> > > > > > > >> >> > > > > > > > > tokens. It would be useful to distinguish
> > > > between
> > > > > > the
> > > > > > > >> two.
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence
> "Broker
> > > > also
> > > > > > > >> stores the
> > > > > > > >> >> > > > > > > > DelegationToken
> > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit
> > > > more? I
> > > > > > > >> thought the
> > > > > > > >> >> > > > > > > delegation
> > > > > > > >> >> > > > > > > > > token is the hmac.
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > Thanks,
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > Jun
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao
> <
> > > > > > > >> jun@confluent.io>
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > > > > >> >> > > > > > > > > >
> > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We
> > can
> > > > > > discuss
> > > > > > > >> this in
> > > > > > > >> >> > > the
> > > > > > > >> >> > > > > > > meeting
> > > > > > > >> >> > > > > > > > > > tomorrow.
> > > > > > > >> >> > > > > > > > > >
> > > > > > > >> >> > > > > > > > > > Thanks,
> > > > > > > >> >> > > > > > > > > >
> > > > > > > >> >> > > > > > > > > > Jun
> > > > > > > >> >> > > > > > > > > >
> > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM,
> Harsha <
> > > > > > > >> kafka@harsha.io>
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > > > > > >
> > > > > > > >> >> > > > > > > > > >> Hi All,
> > > > > > > >> >> > > > > > > > > >> Can we have a KIP meeting
> > > around
> > > > > > this.
> > > > > > > >> The KIP
> > > > > > > >> >> > is
> > > > > > > >> >> > > > up
> > > > > > > >> >> > > > > > for
> > > > > > > >> >> > > > > > > > > >> sometime and if there are
> any
> > > > > > questions
> > > > > > > >> lets
> > > > > > > >> >> > > > quickly
> > > > > > > >> >> > > > > > hash
> > > > > > > >> >> > > > > > > > out
> > > > > > > >> >> > > > > > > > > >> details.
> > > > > > > >> >> > > > > > > > > >>
> > > > > > > >> >> > > > > > > > > >> Thanks,
> > > > > > > >> >> > > > > > > > > >> Harsha
> > > > > > > >> >> > > > > > > > > >>
> > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM,
> parth
> > > > > > > brahmbhatt
> > > > > > > >> wrote:
> > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo system
> > uses
> > > > so
> > > > > no
> > > > > > > >> good
> > > > > > > >> >> > reason
> > > > > > > >> >> > > > > > really.
> > > > > > > >> >> > > > > > > > We
> > > > > > > >> >> > > > > > > > > >> > could
> > > > > > > >> >> > > > > > > > > >> > change it to whatever is the newest
> > > > > > recommended
> > > > > > > >> standard
> > > > > > > >> >> > > is.
> > > > > > > >> >> > > > > > > > > >> >
> > > > > > > >> >> > > > > > > > > >> > Thanks
> > > > > > > >> >> > > > > > > > > >> > Parth
> > > > > > > >> >> > > > > > > > > >> >
> > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM,
> > Ismael
> > > > > Juma <
> > > > > > > >> >> > > > > ismael@juma.me.uk
> > > > > > > >> >> > > > > > >
> > > > > > > >> >> > > > > > > > > wrote:
> > > > > > > >> >> > > > > > > > > >> >
> > > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started
> > > > > reviewing
> > > > > > > >> this and
> > > > > > > >> >> > > may
> > > > > > > >> >> > > > > have
> > > > > > > >> >> > > > > > > > > >> additional
> > > > > > > >> >> > > > > > > > > >> > > questions later. The immediate
> > > question
> > > > > that
> > > > > > > >> came to
> > > > > > > >> >> > > mind
> > > > > > > >> >> > > > is
> > > > > > > >> >> > > > > > our
> > > > > > > >> >> > > > > > > > > >> choice of
> > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's
> marked
> > > as
> > > > > > > >> OBSOLETE in
> > > > > > > >> >> > the
> > > > > > > >> >> > > > IANA
> > > > > > > >> >> > > > > > > > > Registry
> > > > > > > >> >> > > > > > > > > >> of
> > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original
> RFC
> > > > > (2831)
> > > > > > > has
> > > > > > > >> been
> > > > > > > >> >> > > moved
> > > > > > > >> >> > > > > to
> > > > > > > >> >> > > > > > > > > Historic
> > > > > > > >> >> > > > > > > > > >> > > status:
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/
> rfc6331
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > >
> > > > > > > >>
> > > > >
> > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind that
> > > > choice?
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > Thanks,
> > > > > > > >> >> > > > > > > > > >> > > Ismael
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM,
> > Gwen
> > > > > > > Shapira <
> > > > > > > >> >> > > > > > > gwen@confluent.io
> > > > > > > >> >> > > > > > > > >
> > > > > > > >> >> > > > > > > > > >> wrote:
> > > > > > > >> >> > > > > > > > > >> > >
> > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that
> even
> > > > though
> > > > > > > >> delegation
> > > > > > > >> >> > > > tokens
> > > > > > > >> >> > > > > > > are a
> > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very
> strongly
> > > > about
> > > > > > not
> > > > > > > >> adding
> > > > > > > >> >> > > > > > dependency
> > > > > > > >> >> > > > > > > > on
> > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > >> >> > > > > > > > > >> > > > > when implementing delegation
> > > tokens
> > > > > for
> > > > > > > >> Kafka. The
> > > > > > > >> >> > > KIP
> > > > > > > >> >> > > > > > > doesn't
> > > > > > > >> >> > > > > > > > > >> imply
> > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if you
> can
> > > > > > clarify...
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so
> > no
> > > > one
> > > > > > will
> > > > > > > >> read
> > > > > > > >> >> > the
> > > > > > > >> >> > > > KIP
> > > > > > > >> >> > > > > > and
> > > > > > > >> >> > > > > > > > > panic
> > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
> > > release...
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation token
> at
> > > any
> > > > > > time
> > > > > > > >> after
> > > > > > > >> >> > > > > > > > authenticating?
> > > > > > > >> >> > > > > > > > > >> only
> > > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> > authenticated
> > > > you
> > > > > > can
> > > > > > > >> get
> > > > > > > >> >> > > > delegation
> > > > > > > >> >> > > > > > > > tokens.
> > > > > > > >> >> > > > > > > > > >> We
> > > > > > > >> >> > > > > > > > > >> > > need
> > > > > > > >> >> > > > > > > > > >> > > > to
> > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> > authenticated
> > > > > using
> > > > > > > >> delegation
> > > > > > > >> >> > > > > token,
> > > > > > > >> >> > > > > > > can
> > > > > > > >> >> > > > > > > > > also
> > > > > > > >> >> > > > > > > > > >> > > > acquire
> > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or not.
> > > Also
> > > > > > there
> > > > > > > >> is the
> > > > > > > >> >> > > > > question
> > > > > > > >> >> > > > > > of
> > > > > > > >> >> > > > > > > > do
> > > > > > > >> >> > > > > > > > > we
> > > > > > > >> >> > > > > > > > > >> > > allow
> > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation
> > token
> > > > or
> > > > > we
> > > > > > > >> want
> > > > > > > >> >> > > specific
> > > > > > > >> >> > > > > > ACLs
> > > > > > > >> >> > > > > > > (I
> > > > > > > >> >> > > > > > > > > >> think
> > > > > > > >> >> > > > > > > > > >> > > its
> > > > > > > >> >> > > > > > > > > >> > > > an
> > > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an
> overkill.
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
> > options:
> > > > > > Either
> > > > > > > >> require
> > > > > > > >> >> > > > > Kerberos
> > > > > > > >> >> > > > > > > > auth
> > > > > > > >> >> > > > > > > > > >> for
> > > > > > > >> >> > > > > > > > > >> > > > renewal or require non-owners to
> > > > renew.
> > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler
> > (it
> > > > > > > basically
> > > > > > > >> >> > require
> > > > > > > >> >> > > a
> > > > > > > >> >> > > > > "job
> > > > > > > >> >> > > > > > > > > master"
> > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for the
> > > > renewal,
> > > > > it
> > > > > > > >> will have
> > > > > > > >> >> > > its
> > > > > > > >> >> > > > > own
> > > > > > > >> >> > > > > > > > > >> identity
> > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is the
> > > correct
> > > > > > > design
> > > > > > > >> >> > pattern
> > > > > > > >> >> > > > > > anyway.
> > > > > > > >> >> > > > > > > > For
> > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
> > > coordinate
> > > > > > > >> renewals?),
> > > > > > > >> >> > but
> > > > > > > >> >> > > > it
> > > > > > > >> >> > > > > is
> > > > > > > >> >> > > > > > > > hard
> > > > > > > >> >> > > > > > > > > to
> > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without
> looking
> > at
> > > > the
> > > > > > > code
> > > > > > > >> >> > changes
> > > > > > > >> >> > > > > > > required.
> > > > > > > >> >> > > > > > > > If
> > > > > > > >> >> > > > > > > > > >> you
> > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the "require
> > > > > Kerberos"
> > > > > > > >> will look
> > > > > > > >> >> > > in
> > > > > > > >> >> > > > > > Kafka
> > > > > > > >> >> > > > > > > > > code,
> > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is that
> > tokens
> > > > will
> > > > > > > >> propagate
> > > > > > > >> >> > via
> > > > > > > >> >> > > > ZK
> > > > > > > >> >> > > > > > but
> > > > > > > >> >> > > > > > > > > >> without
> > > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> > > UpdateMetadata
> > > > > > > >> protocol,
> > > > > > > >> >> > > > correct?
> > > > > > > >> >> > > > > > > > Clients
> > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL
> > auth
> > > > > > failure
> > > > > > > >> (IIRC),
> > > > > > > >> >> > > but
> > > > > > > >> >> > > > > > since
> > > > > > > >> >> > > > > > > > the
> > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between
> brokers
> > > > > asynch,
> > > > > > > we
> > > > > > > >> will
> > > > > > > >> >> > > need
> > > > > > > >> >> > > > to
> > > > > > > >> >> > > > > > > > retry a
> > > > > > > >> >> > > > > > > > > >> bit
> > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth
> > due
> > > to
> > > > > > > timing
> > > > > > > >> >> > issues.
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2
> alternatives
> > > > right
> > > > > > > now.
> > > > > > > >> The
> > > > > > > >> >> > > > current
> > > > > > > >> >> > > > > > > > > documented
> > > > > > > >> >> > > > > > > > > >> > > > approach
> > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does
> > not
> > > > > > require
> > > > > > > >> any
> > > > > > > >> >> > > changes
> > > > > > > >> >> > > > > to
> > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
> > approach
> > > > can
> > > > > > > remove
> > > > > > > >> >> > > zookeeper
> > > > > > > >> >> > > > > > > > > dependency
> > > > > > > >> >> > > > > > > > > >> as
> > > > > > > >> >> > > > > > > > > >> > > well
> > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> > > > > > discussion
> > > > > > > >> call.*
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you
> > > want
> > > > to
> > > > > > > ping
> > > > > > > >> Jun to
> > > > > > > >> >> > > > > > arrange a
> > > > > > > >> >> > > > > > > > > call?
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion
> of
> > > > > having
> > > > > > > >> just the
> > > > > > > >> >> > > > > > controller
> > > > > > > >> >> > > > > > > > > issue
> > > > > > > >> >> > > > > > > > > >> the
> > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid
> > > syncing
> > > > a
> > > > > > > shared
> > > > > > > >> >> > secret.
> > > > > > > >> >> > > > Not
> > > > > > > >> >> > > > > > > sure
> > > > > > > >> >> > > > > > > > if
> > > > > > > >> >> > > > > > > > > >> we
> > > > > > > >> >> > > > > > > > > >> > > > > want to continue the
> discussion
> > > here
> > > > > or
> > > > > > on
> > > > > > > >> the
> > > > > > > >> >> > > wiki. I
> > > > > > > >> >> > > > > > think
> > > > > > > >> >> > > > > > > > > that
> > > > > > > >> >> > > > > > > > > >> we
> > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem of
> > "token
> > > > > > > >> distribution"
> > > > > > > >> >> > > from
> > > > > > > >> >> > > > > > > "shared
> > > > > > > >> >> > > > > > > > > >> secret
> > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
> > > controller
> > > > > as
> > > > > > > the
> > > > > > > >> only
> > > > > > > >> >> > > token
> > > > > > > >> >> > > > > > > > generator
> > > > > > > >> >> > > > > > > > > >> to
> > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue, while
> > > still
> > > > > > using
> > > > > > > >> ZK async
> > > > > > > >> >> > > to
> > > > > > > >> >> > > > > > > > distribute
> > > > > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous
> > > Email
> > > > I
> > > > > am
> > > > > > > >> fine with
> > > > > > > >> >> > > > that
> > > > > > > >> >> > > > > > > > approach
> > > > > > > >> >> > > > > > > > > >> as
> > > > > > > >> >> > > > > > > > > >> > > long
> > > > > > > >> >> > > > > > > > > >> > > > as
> > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
> > complexity
> > > > of
> > > > > > > >> >> > > adding/updating
> > > > > > > >> >> > > > > APIS
> > > > > > > >> >> > > > > > > > adds
> > > > > > > >> >> > > > > > > > > >> enough
> > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with the
> > > > > controller
> > > > > > > >> approach
> > > > > > > >> >> > is
> > > > > > > >> >> > > > > > secret
> > > > > > > >> >> > > > > > > > > >> rotation
> > > > > > > >> >> > > > > > > > > >> > > can
> > > > > > > >> >> > > > > > > > > >> > > > be
> > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and would
> not
> > > > > require
> > > > > > > >> >> > > deployment. *
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> > complexity
> > > > (or
> > > > > > > point
> > > > > > > >> me to
> > > > > > > >> >> > > the
> > > > > > > >> >> > > > > > email
> > > > > > > >> >> > > > > > > I
> > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
> > required?
> > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients
> can
> > > > > already
> > > > > > > >> find the
> > > > > > > >> >> > > > > > controller
> > > > > > > >> >> > > > > > > > from
> > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more
> concerned
> > > > about
> > > > > > > >> controller
> > > > > > > >> >> > > > load.
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of
> > forcing
> > > > > > > kerberos
> > > > > > > >> auth
> > > > > > > >> >> > for
> > > > > > > >> >> > > > > > > renwal, I
> > > > > > > >> >> > > > > > > > > >> think
> > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer
> the
> > > the
> > > > > > > request
> > > > > > > >> >> > content
> > > > > > > >> >> > > > in
> > > > > > > >> >> > > > > a
> > > > > > > >> >> > > > > > > > pretty
> > > > > > > >> >> > > > > > > > > >> ugly
> > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer
> to
> > > > > > non-owner
> > > > > > > >> is
> > > > > > > >> >> > > better.
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary
> > evil.
> > > > > While
> > > > > > > >> this will
> > > > > > > >> >> > > make
> > > > > > > >> >> > > > > the
> > > > > > > >> >> > > > > > > > kafka
> > > > > > > >> >> > > > > > > > > >> code
> > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
> > forcing
> > > > > > renewer
> > > > > > > >> to
> > > > > > > >> >> > > > non-owner
> > > > > > > >> >> > > > > > > pushes
> > > > > > > >> >> > > > > > > > > >> the code
> > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes
> it
> > > even
> > > > > > > harder
> > > > > > > >> to
> > > > > > > >> >> > > > > > integrate. *
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't
> think
> > > the
> > > > > > > >> "renewal by
> > > > > > > >> >> > > > other"
> > > > > > > >> >> > > > > > > > approach
> > > > > > > >> >> > > > > > > > > >> is
> > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we
> > expect
> > > to
> > > > > use
> > > > > > > >> >> > delegation
> > > > > > > >> >> > > > > tokens
> > > > > > > >> >> > > > > > > > since
> > > > > > > >> >> > > > > > > > > >> > > > they will have an app-master of
> > some
> > > > > sort
> > > > > > > who
> > > > > > > >> >> > > requested
> > > > > > > >> >> > > > > the
> > > > > > > >> >> > > > > > > > token
> > > > > > > >> >> > > > > > > > > to
> > > > > > > >> >> > > > > > > > > >> > > > begin with.
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > The response for my question
> on
> > > how
> > > > > > > multiple
> > > > > > > >> >> > > > identities
> > > > > > > >> >> > > > > > will
> > > > > > > >> >> > > > > > > > be
> > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to
> > me -
> > > > > > AFAIK,
> > > > > > > >> we don't
> > > > > > > >> >> > > > > > > > authenticate
> > > > > > > >> >> > > > > > > > > >> each
> > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> > > > connections.
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate connections,
> > and
> > > > only
> > > > > > > when
> > > > > > > >> they
> > > > > > > >> >> > are
> > > > > > > >> >> > > > > being
> > > > > > > >> >> > > > > > > > > >> established.
> > > > > > > >> >> > > > > > > > > >> > > > Let
> > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a
> > > question,
> > > > > in
> > > > > > > >> absence of
> > > > > > > >> >> > > > > > > delegation
> > > > > > > >> >> > > > > > > > > >> tokens if
> > > > > > > >> >> > > > > > > > > >> > > > we
> > > > > > > >> >> > > > > > > > > >> > > > > had to support the use case
> > using
> > > > user
> > > > > > > >> TGT's how
> > > > > > > >> >> > > would
> > > > > > > >> >> > > > > we
> > > > > > > >> >> > > > > > do
> > > > > > > >> >> > > > > > > > it?
> > > > > > > >> >> > > > > > > > > >> My
> > > > > > > >> >> > > > > > > > > >> > > point
> > > > > > > >> >> > > > > > > > > >> > > > > was it would be no different
> > with
> > > > > > > delegation
> > > > > > > >> >> > tokens.
> > > > > > > >> >> > > > The
> > > > > > > >> >> > > > > > use
> > > > > > > >> >> > > > > > > > > case
> > > > > > > >> >> > > > > > > > > >> you
> > > > > > > >> >> > > > > > > > > >> > > are
> > > > > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > > > > impersonation.*
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the
> > > things
> > > > > > that
> > > > > > > >> >> > delegation
> > > > > > > >> >> > > > > > tokens
> > > > > > > >> >> > > > > > > > > >> handled.
> > > > > > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > Gwen
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > >
> > > > > > > >> >> > > > > > > > > >> > > > > Thanks
> > > > > > > >> >> > > > > > > > > >> > > > > Parth
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19
> > AM,
> > > > Gwen
> > > > > > > >> Shapira <
> > > > > > > >> >> > > > > > > > > gwen@confluent.io
> > > > > > > >> >> > > > > > > > > >> >
> > > > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> Few more comments:
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse
> > > section
> > > > > > > >> doesn't seem
> > > > > > > >> >> > to
> > > > > > > >> >> > > > > have
> > > > > > > >> >> > > > > > > good
> > > > > > > >> >> > > > > > > > > >> > > > >> description of the changes to
> > the
> > > > > Kafka
> > > > > > > >> Protocol.
> > > > > > > >> >> > > > > Sounds
> > > > > > > >> >> > > > > > > like
> > > > > > > >> >> > > > > > > > > >> you are
> > > > > > > >> >> > > > > > > > > >> > > > >> proposing new
> > > > DelegationTokenRequest
> > > > > > and
> > > > > > > >> >> > > > > > RenewTokenRequest
> > > > > > > >> >> > > > > > > > (and
> > > > > > > >> >> > > > > > > > > >> > > > >> matching responses), without
> > > > > detailing
> > > > > > > the
> > > > > > > >> >> > contents
> > > > > > > >> >> > > > of
> > > > > > > >> >> > > > > > the
> > > > > > > >> >> > > > > > > > > >> requests
> > > > > > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you
> > > show
> > > > > the
> > > > > > > >> class
> > > > > > > >> >> > > > interface,
> > > > > > > >> >> > > > > > but
> > > > > > > >> >> > > > > > > > not
> > > > > > > >> >> > > > > > > > > >> the
> > > > > > > >> >> > > > > > > > > >> > > > >> underlying protocol. This
> could
> > > be
> > > > > seen
> > > > > > > as
> > > > > > > >> an
> > > > > > > >> >> > > > > > > implementation
> > > > > > > >> >> > > > > > > > > >> detail,
> > > > > > > >> >> > > > > > > > > >> > > > >> but since the binary protocol
> > is
> > > > what
> > > > > > we
> > > > > > > >> provide
> > > > > > > >> >> > to
> > > > > > > >> >> > > > > > > non-Java
> > > > > > > >> >> > > > > > > > > >> clients,
> > > > > > > >> >> > > > > > > > > >> > > > >> we need to show the changes
> > > there.
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds
> > like
> > > > > > > >> >> > > > > > > > delegationTokenRequestHandler?
> > > > > > > >> >> > > > > > > > > >> Is it
> > > > > > > >> >> > > > > > > > > >> > > > >> planned to be part of
> KafkaApi?
> > > or
> > > > > > > Client?
> > > > > > > >> Its
> > > > > > > >> >> > > > > unclear...
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that
> even
> > > > > though
> > > > > > > >> delegation
> > > > > > > >> >> > > > > tokens
> > > > > > > >> >> > > > > > > are
> > > > > > > >> >> > > > > > > > a
> > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > >> >> > > > > > > > > >> > > > >> innovation, I feel very
> > strongly
> > > > > about
> > > > > > > not
> > > > > > > >> adding
> > > > > > > >> >> > > > > > > dependency
> > > > > > > >> >> > > > > > > > on
> > > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > > >> >> > > > > > > > > >> > > > >> when implementing delegation
> > > tokens
> > > > > for
> > > > > > > >> Kafka.
> > > > > > > >> >> > The
> > > > > > > >> >> > > > KIP
> > > > > > > >> >> > > > > > > > doesn't
> > > > > > > >> >> > > > > > > > > >> imply
> > > > > > > >> >> > > > > > > > > >> > > > >> such dependency, but if you
> can
> > > > > > > clarify...
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token
> > at
> > > > any
> > > > > > time
> > > > > > > >> after
> > > > > > > >> >> > > > > > > > authenticating?
> > > > > > > >> >> > > > > > > > > >> only
> > > > > > > >> >> > > > > > > > > >> > > > >> immediately after?
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * My understanding is that
> > tokens
> > > > > will
> > > > > > > >> propagate
> > > > > > > >> >> > > via
> > > > > > > >> >> > > > ZK
> > > > > > > >> >> > > > > > but
> > > > > > > >> >> > > > > > > > > >> without
> > > > > > > >> >> > > > > > > > > >> > > > >> additional changes to
> > > > UpdateMetadata
> > > > > > > >> protocol,
> > > > > > > >> >> > > > correct?
> > > > > > > >> >> > > > > > > > Clients
> > > > > > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL
> > > auth
> > > > > > > failure
> > > > > > > >> >> > (IIRC),
> > > > > > > >> >> > > > but
> > > > > > > >> >> > > > > > > since
> > > > > > > >> >> > > > > > > > > the
> > > > > > > >> >> > > > > > > > > >> > > > >> tokens propagate between
> > brokers
> > > > > > asynch,
> > > > > > > >> we will
> > > > > > > >> >> > > need
> > > > > > > >> >> > > > > to
> > > > > > > >> >> > > > > > > > retry
> > > > > > > >> >> > > > > > > > > a
> > > > > > > >> >> > > > > > > > > >> bit
> > > > > > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth
> > due
> > > > to
> > > > > > > timing
> > > > > > > >> >> > issues.
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on
> clients
> > > not
> > > > > > > >> touching ZK
> > > > > > > >> >> > > > directly
> > > > > > > >> >> > > > > > :)
> > > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion
> > of
> > > > > having
> > > > > > > >> just the
> > > > > > > >> >> > > > > > controller
> > > > > > > >> >> > > > > > > > > >> issue the
> > > > > > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid
> > > > syncing a
> > > > > > > >> shared
> > > > > > > >> >> > > secret.
> > > > > > > >> >> > > > > Not
> > > > > > > >> >> > > > > > > sure
> > > > > > > >> >> > > > > > > > > if
> > > > > > > >> >> > > > > > > > > >> we
> > > > > > > >> >> > > > > > > > > >> > > > >> want to continue the
> discussion
> > > > here
> > > > > or
> > > > > > > on
> > > > > > > >> the
> > > > > > > >> >> > > wiki.
> > > > > > > >> >> > >
>
--
Regards,
Ashish
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Harsha Chintalapani <ka...@harsha.io>.
Hi Grant,
We are working on it. Will add the details to KIP about the
request protocol.
Thanks,
Harsha
On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <gh...@cloudera.com> wrote:
> Hi Parth,
>
> Are you still working on this? If you need any help please don't hesitate
> to ask.
>
> Thanks,
> Grant
>
> On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Parth,
> >
> > Thanks for the reply.
> >
> > It makes sense to only allow the renewal by users that authenticated
> using
> > *non* delegation token mechanism. Then, should we make the renewal a
> list?
> > For example, in the case of rest proxy, it will be useful for every
> > instance of rest proxy to be able to renew the tokens.
> >
> > It would be clearer if we can document the request protocol like
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> > .
> >
> > It would also be useful to document the client APIs.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> > brahmbhatt.parth@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I am suggesting that we will only allow the renewal by users that
> > > authenticated using *non* delegation token mechanism. For example, If
> > user
> > > Alice authenticated using kerberos and requested delegation tokens,
> only
> > > user Alice authenticated via non delegation token mechanism can renew.
> > > Clients that have access to delegation tokens can not issue renewal
> > > request for renewing their own token and this is primarily important to
> > > reduce the time window for which a compromised token will be valid.
> > >
> > > To clarify, Yes any authenticated user can request delegation tokens
> but
> > > even here I would recommend to avoid creating a chain where a client
> > > authenticated via delegation token request for more delegation tokens.
> > > Basically anyone can request delegation token, as long as they
> > authenticate
> > > via a non delegation token mechanism.
> > >
> > > Aren't classes listed here
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-PublicInterfaces
> > > >
> > > sufficient?
> > >
> > > Thanks
> > > Parth
> > >
> > >
> > >
> > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Parth,
> > > >
> > > > Thanks for the reply. A couple of comments inline below.
> > > >
> > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > > > brahmbhatt.parth@gmail.com> wrote:
> > > >
> > > > > 1. Who / how are tokens renewed? By original requester only? or
> using
> > > > > Kerberos
> > > > > auth only?
> > > > > My recommendation is to do this only using Kerberos auth and only
> > threw
> > > > the
> > > > > renewer specified during the acquisition request.
> > > > >
> > > > >
> > > > Hmm, not sure that I follow this. Are you saying that any client
> > > > authenticated with the delegation token can renew, i.e. there is no
> > > renewer
> > > > needed?
> > > >
> > > > Also, just to be clear, any authenticated client (either through SASL
> > or
> > > > SSL) can request a delegation token for the authenticated user,
> right?
> > > >
> > > >
> > > > > 2. Are tokens stored on each broker or in ZK?
> > > > > My recommendation is still to store in ZK or not store them at all.
> > The
> > > > > whole controller based distribution is too much overhead with not
> > much
> > > to
> > > > > achieve.
> > > > >
> > > > > 3. How are tokens invalidated / expired?
> > > > > Either by expiration time out or through an explicit request to
> > > > invalidate.
> > > > >
> > > > > 4. Which encryption algorithm is used?
> > > > > SCRAM
> > > > >
> > > > > 5. What is the impersonation proposal (it wasn't in the KIP but was
> > > > > discussed
> > > > > in this thread)?
> > > > > There is no imperonation proposal. I tried and explained how its a
> > > > > different problem and why its not really necessary to discuss that
> as
> > > > part
> > > > > of this KIP. This KIP will not support any impersonation, it will
> > just
> > > > be
> > > > > another way to authenticate.
> > > > >
> > > > > 6. Do we need new ACLs, if so - for what actions?
> > > > > We do not need new ACLs.
> > > > >
> > > > >
> > > > Could we document the format of the new request/response and their
> > > > associated Resource and Operation for ACL?
> > > >
> > > >
> > > > > 7. How would the delegation token be configured in the client?
> > > > > Should be through config. I wasn't planning on supporting JAAS for
> > > > tokens.
> > > > > I don't believe hadoop does this either.
> > > > >
> > > > > Thanks
> > > > > Parth
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Harsha,
> > > > > >
> > > > > > Another question.
> > > > > >
> > > > > > 9. How would the delegation token be configured in the client?
> The
> > > > > standard
> > > > > > way is to do this through JAAS. However, we will need to think
> > > through
> > > > if
> > > > > > this is convenient in a shared environment. For example, when a
> new
> > > > task
> > > > > is
> > > > > > added to a Storm worker node, do we need to dynamically add a new
> > > > section
> > > > > > in the JAAS file? It may be more convenient if we can pass in the
> > > token
> > > > > > through the config directly w/o going through JAAS.
> > > > > >
> > > > > > Are you or Parth still actively working on this KIP?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Just to add on that list.
> > > > > > >
> > > > > > > 2. It would be good to document the format of the data stored
> in
> > > ZK.
> > > > > > > 7. Earlier, there was a discussion on whether the tokens should
> > be
> > > > > > > propagated through ZK like config/acl/quota, or through the
> > > > controller.
> > > > > > > Currently, the controller is only designed for propagating
> topic
> > > > > > metadata,
> > > > > > > but not other data.
> > > > > > > 8. Should we use SCRAM to send the token instead of DIGEST-MD5
> > > since
> > > > > it's
> > > > > > > deprecated?
> > > > > > >
> > > > > > > Also, the images in the wiki seem broken.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> > gwen@confluent.io>
> > > > > > wrote:
> > > > > > >
> > > > > > >> From what I can see, remaining questions are:
> > > > > > >>
> > > > > > >> 1. Who / how are tokens renewed? By original requester only?
> or
> > > > using
> > > > > > >> Kerberos auth only?
> > > > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > > > >> 3. How are tokens invalidated / expired?
> > > > > > >> 4. Which encryption algorithm is used?
> > > > > > >> 5. What is the impersonation proposal (it wasn't in the KIP
> but
> > > was
> > > > > > >> discussed in this thread)?
> > > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > > > >>
> > > > > > >> Gwen
> > > > > > >>
> > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io>
> wrote:
> > > > > > >> > Jun & Ismael,
> > > > > > >> > Unfortunately I couldn't attend the
> > KIP
> > > > > > meeting
> > > > > > >> > when delegation tokens discussed.
> > > > > Appreciate
> > > > > > if
> > > > > > >> > you can update the thread if you
> have
> > > any
> > > > > > >> > further questions.
> > > > > > >> > Thanks,
> > > > > > >> > Harsha
> > > > > > >> >
> > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > > > >> >> It seems that the links to images in the KIP are broken.
> > > > > > >> >>
> > > > > > >> >> Liquan
> > > > > > >> >>
> > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > > > >> >>
> > > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > > > >> >> > In the current proposal we only allow a user to get
> > > delegation
> > > > > > token
> > > > > > >> for
> > > > > > >> >> > the identity that it authenticated as using another
> > > mechanism,
> > > > > i.e.
> > > > > > >> A user
> > > > > > >> >> > that authenticate using a keytab for principal
> > > > user1@EXAMPLE.COM
> > > > > > >> will get
> > > > > > >> >> > delegation tokens for that user only. In future I think
> we
> > > will
> > > > > > have
> > > > > > >> to
> > > > > > >> >> > extend support such that we allow some set of users (
> > > > > > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM)
> to
> > > > > acquire
> > > > > > >> >> > delegation tokens on behalf of other users whose identity
> > > they
> > > > > have
> > > > > > >> >> > verified independently. Kafka brokers will have ACLs to
> > > > control
> > > > > > >> which
> > > > > > >> >> > users are allowed to impersonate other users and get
> tokens
> > > on
> > > > > > >> behalf of
> > > > > > >> >> > them. Overall Impersonation is a whole different problem
> in
> > > my
> > > > > > >> opinion and
> > > > > > >> >> > I think we can tackle it in separate KIP.
> > > > > > >> >> >
> > > > > > >> >> > 111. What's the typical rate of getting and renewing
> > > delegation
> > > > > > >> tokens?
> > > > > > >> >> > Typically this should be very very low, 1 request per
> > minute
> > > > is a
> > > > > > >> >> > relatively high estimate. However it depends on the token
> > > > > > >> expiration. I am
> > > > > > >> >> > less worried about the extra load it puts on controller
> vs
> > > the
> > > > > > added
> > > > > > >> >> > complexity and the value it offers.
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Parth
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > > > ismael@juma.me.uk>
> > > > > > >> wrote:
> > > > > > >> >> >
> > > > > > >> >> > > Thanks Rajini. It would probably require a separate KIP
> > as
> > > it
> > > > > > will
> > > > > > >> >> > > introduce user visible changes. We could also update
> > KIP-48
> > > > to
> > > > > > >> have this
> > > > > > >> >> > > information, but it seems cleaner to do it separately.
> We
> > > can
> > > > > > >> discuss
> > > > > > >> >> > that
> > > > > > >> >> > > in the KIP call today.
> > > > > > >> >> > >
> > > > > > >> >> > > Ismael
> > > > > > >> >> > >
> > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > > Ismael,
> > > > > > >> >> > > >
> > > > > > >> >> > > > I have created a JIRA (
> > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that need
> > > > another
> > > > > > >> KIP? If
> > > > > > >> >> > > > KIP-48 will use this mechanism, can this just be a
> JIRA
> > > > that
> > > > > > gets
> > > > > > >> >> > > reviewed
> > > > > > >> >> > > > when the PR is ready?
> > > > > > >> >> > > >
> > > > > > >> >> > > > Thank you,
> > > > > > >> >> > > >
> > > > > > >> >> > > > Rajini
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > > > ismael@juma.me.uk>
> > > > > > >> >> > wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > Gwen had independently mentioned this as a SASL
> > > mechanism
> > > > > > that
> > > > > > >> might
> > > > > > >> >> > be
> > > > > > >> >> > > > > useful for Kafka and I have been meaning to check
> it
> > in
> > > > > more
> > > > > > >> detail.
> > > > > > >> >> > > Good
> > > > > > >> >> > > > > to know that you are willing to contribute an
> > > > > implementation.
> > > > > > >> Maybe
> > > > > > >> >> > we
> > > > > > >> >> > > > > should file a separate JIRA for this?
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > Ismael
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> > > > > Mechanism)
> > > > > > >> is a
> > > > > > >> >> > > better
> > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come
> with a
> > > > > > built-in
> > > > > > >> SCRAM
> > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy to
> > add
> > > > > > support
> > > > > > >> in
> > > > > > >> >> > Kafka
> > > > > > >> >> > > > > since
> > > > > > >> >> > > > > > it would be a useful mechanism to support anyway.
> > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes
> the
> > > > > protocol
> > > > > > >> for
> > > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > > > jun@confluent.io
> > > > > > >
> > > > > > >> wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > > Parth,
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > Thanks for the explanation. A couple of more
> > > > questions.
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > 111. What's the typical rate of getting and
> > > renewing
> > > > > > >> delegation
> > > > > > >> >> > > > tokens?
> > > > > > >> >> > > > > > > That may have an impact on whether they should
> be
> > > > > > directed
> > > > > > >> to the
> > > > > > >> >> > > > > > > controller.
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > Jun
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
> > brahmbhatt <
> > > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > > Hi Jun,
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > Thanks for reviewing.
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > * We could add a Cluster action to add acls
> on
> > > who
> > > > > can
> > > > > > >> request
> > > > > > >> >> > > > > > delegation
> > > > > > >> >> > > > > > > > tokens. I don't see the use case for that yet
> > but
> > > > > down
> > > > > > >> the line
> > > > > > >> >> > > > when
> > > > > > >> >> > > > > we
> > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs it will
> > be
> > > > > > >> necessary.
> > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > > > used/distributed
> > > > > > >> over
> > > > > > >> >> > secure
> > > > > > >> >> > > > > > > channels.
> > > > > > >> >> > > > > > > > * Depending on what design we end up choosing
> > > > > > >> Invalidation will
> > > > > > >> >> > > be
> > > > > > >> >> > > > > > > > responsibility of every broker or controller.
> > > > > > >> >> > > > > > > > * I am not sure if I documented somewhere
> that
> > > > > > >> invalidation
> > > > > > >> >> > will
> > > > > > >> >> > > > > > directly
> > > > > > >> >> > > > > > > > go through zookeeper but that is not the
> > intent.
> > > > > > >> Invalidation
> > > > > > >> >> > > will
> > > > > > >> >> > > > > > either
> > > > > > >> >> > > > > > > > be request based or due to expiration. No
> > direct
> > > > > > >> zookeeper
> > > > > > >> >> > > > > interaction
> > > > > > >> >> > > > > > > from
> > > > > > >> >> > > > > > > > any client.
> > > > > > >> >> > > > > > > > * "Broker also stores the DelegationToken
> > without
> > > > the
> > > > > > >> hmac in
> > > > > > >> >> > the
> > > > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion. The
> > sole
> > > > > > >> purpose of
> > > > > > >> >> > > > > zookeeper
> > > > > > >> >> > > > > > in
> > > > > > >> >> > > > > > > > this design is as distribution channel for
> > tokens
> > > > > > >> between all
> > > > > > >> >> > > > brokers
> > > > > > >> >> > > > > > > and a
> > > > > > >> >> > > > > > > > layer that ensures only tokens that were
> > > generated
> > > > by
> > > > > > >> making a
> > > > > > >> >> > > > > request
> > > > > > >> >> > > > > > > to a
> > > > > > >> >> > > > > > > > broker will be accepted (more on this in
> second
> > > > > > >> paragraph). The
> > > > > > >> >> > > > token
> > > > > > >> >> > > > > > > > consists of few elements (owner, renewer,
> uuid
> > ,
> > > > > > >> expiration,
> > > > > > >> >> > > hmac)
> > > > > > >> >> > > > ,
> > > > > > >> >> > > > > > one
> > > > > > >> >> > > > > > > of
> > > > > > >> >> > > > > > > > which is the finally generated hmac but hmac
> it
> > > > self
> > > > > is
> > > > > > >> >> > derivable
> > > > > > >> >> > > > if
> > > > > > >> >> > > > > > you
> > > > > > >> >> > > > > > > > have all the other elements of the token +
> > secret
> > > > key
> > > > > > to
> > > > > > >> >> > generate
> > > > > > >> >> > > > > hmac.
> > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL support
> we
> > > do
> > > > > not
> > > > > > >> want the
> > > > > > >> >> > > > > entire
> > > > > > >> >> > > > > > > > token to be wire transferred to zookeeper as
> > that
> > > > > will
> > > > > > >> be an
> > > > > > >> >> > > > insecure
> > > > > > >> >> > > > > > > wire
> > > > > > >> >> > > > > > > > transfer. Instead we only store all the other
> > > > > elements
> > > > > > >> of a
> > > > > > >> >> > > > > delegation
> > > > > > >> >> > > > > > > > tokens. Brokers can read these elements and
> > > because
> > > > > > they
> > > > > > >> also
> > > > > > >> >> > > have
> > > > > > >> >> > > > > > access
> > > > > > >> >> > > > > > > > to secret key they will be able to generate
> > hmac
> > > on
> > > > > > >> their end.
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > One of the alternative proposed is to avoid
> > > > zookeeper
> > > > > > >> >> > > altogether. A
> > > > > > >> >> > > > > > > Client
> > > > > > >> >> > > > > > > > will call broker with required information
> > > (owner,
> > > > > > >> renwer,
> > > > > > >> >> > > > > expiration)
> > > > > > >> >> > > > > > > and
> > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't
> > store
> > > > this
> > > > > > in
> > > > > > >> >> > > zookeeper.
> > > > > > >> >> > > > > > From
> > > > > > >> >> > > > > > > > this point a client can contact any broker
> with
> > > all
> > > > > the
> > > > > > >> >> > > delegation
> > > > > > >> >> > > > > > token
> > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid)
> > the
> > > > > borker
> > > > > > >> will
> > > > > > >> >> > > > > regenerate
> > > > > > >> >> > > > > > > the
> > > > > > >> >> > > > > > > > hmac and as long as it matches with hmac
> > > presented
> > > > by
> > > > > > >> client ,
> > > > > > >> >> > > > broker
> > > > > > >> >> > > > > > > will
> > > > > > >> >> > > > > > > > allow the request to authenticate. Only
> > problem
> > > > with
> > > > > > >> this
> > > > > > >> >> > > approach
> > > > > > >> >> > > > > is
> > > > > > >> >> > > > > > if
> > > > > > >> >> > > > > > > > the secret key is compromised any client can
> > now
> > > > > > generate
> > > > > > >> >> > random
> > > > > > >> >> > > > > tokens
> > > > > > >> >> > > > > > > and
> > > > > > >> >> > > > > > > > they will still be able to authenticate as
> any
> > > user
> > > > > > they
> > > > > > >> like.
> > > > > > >> >> > > with
> > > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens
> > acquired
> > > > via
> > > > > a
> > > > > > >> broker
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > some
> > > > > > >> >> > > > > > > > auth scheme other than delegation token) will
> > be
> > > > > > >> accepted. We
> > > > > > >> >> > > need
> > > > > > >> >> > > > to
> > > > > > >> >> > > > > > > > discuss which proposal makes more sense and
> we
> > > can
> > > > go
> > > > > > >> over it
> > > > > > >> >> > in
> > > > > > >> >> > > > > > > tomorrow's
> > > > > > >> >> > > > > > > > meeting.
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > Thanks
> > > > > > >> >> > > > > > > > Parth
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > > > > > >> jun@confluent.io>
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > > > >
> > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 100. This potentially can be useful for
> Kafka
> > > > > Connect
> > > > > > >> and
> > > > > > >> >> > Kafka
> > > > > > >> >> > > > > rest
> > > > > > >> >> > > > > > > > proxy
> > > > > > >> >> > > > > > > > > where a worker agent will need to run a
> task
> > on
> > > > > > behalf
> > > > > > >> of a
> > > > > > >> >> > > > client.
> > > > > > >> >> > > > > > We
> > > > > > >> >> > > > > > > > will
> > > > > > >> >> > > > > > > > > likely need to change how those services
> use
> > > > Kafka
> > > > > > >> clients
> > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a shared
> > client
> > > > per
> > > > > > >> worker,
> > > > > > >> >> > we
> > > > > > >> >> > > > will
> > > > > > >> >> > > > > > > need
> > > > > > >> >> > > > > > > > a
> > > > > > >> >> > > > > > > > > client per user task since the
> authentication
> > > > > happens
> > > > > > >> at the
> > > > > > >> >> > > > > > connection
> > > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer will
> be
> > > the
> > > > > > >> workers.
> > > > > > >> >> > So,
> > > > > > >> >> > > we
> > > > > > >> >> > > > > > > > probably
> > > > > > >> >> > > > > > > > > need to allow multiple renewers. For Kafka
> > rest
> > > > > > proxy,
> > > > > > >> the
> > > > > > >> >> > > > renewer
> > > > > > >> >> > > > > > can
> > > > > > >> >> > > > > > > > > probably just be the creator of the token.
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can request
> > > > > delegation
> > > > > > >> tokens?
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 102. Do we recommend people to send
> > delegation
> > > > > tokens
> > > > > > >> in an
> > > > > > >> >> > > > > encrypted
> > > > > > >> >> > > > > > > > > channel?
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring
> tokens,
> > > > every
> > > > > > >> broker?
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would it be
> > > better
> > > > to
> > > > > > do
> > > > > > >> it in
> > > > > > >> >> > a
> > > > > > >> >> > > > > > request
> > > > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 105. The terminology of client in the wiki
> > > > > sometimes
> > > > > > >> refers
> > > > > > >> >> > to
> > > > > > >> >> > > > the
> > > > > > >> >> > > > > > end
> > > > > > >> >> > > > > > > > > client and some other times refers to the
> > > client
> > > > > > using
> > > > > > >> the
> > > > > > >> >> > > > > delegation
> > > > > > >> >> > > > > > > > > tokens. It would be useful to distinguish
> > > between
> > > > > the
> > > > > > >> two.
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > 106. Could you explain the sentence "Broker
> > > also
> > > > > > >> stores the
> > > > > > >> >> > > > > > > > DelegationToken
> > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit
> > > more? I
> > > > > > >> thought the
> > > > > > >> >> > > > > > > delegation
> > > > > > >> >> > > > > > > > > token is the hmac.
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > Thanks,
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > Jun
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> > > > > > >> jun@confluent.io>
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > > > >> >> > > > > > > > > >
> > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We
> can
> > > > > discuss
> > > > > > >> this in
> > > > > > >> >> > > the
> > > > > > >> >> > > > > > > meeting
> > > > > > >> >> > > > > > > > > > tomorrow.
> > > > > > >> >> > > > > > > > > >
> > > > > > >> >> > > > > > > > > > Thanks,
> > > > > > >> >> > > > > > > > > >
> > > > > > >> >> > > > > > > > > > Jun
> > > > > > >> >> > > > > > > > > >
> > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> > > > > > >> kafka@harsha.io>
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > > > > > >
> > > > > > >> >> > > > > > > > > >> Hi All,
> > > > > > >> >> > > > > > > > > >> Can we have a KIP meeting
> > around
> > > > > this.
> > > > > > >> The KIP
> > > > > > >> >> > is
> > > > > > >> >> > > > up
> > > > > > >> >> > > > > > for
> > > > > > >> >> > > > > > > > > >> sometime and if there are any
> > > > > questions
> > > > > > >> lets
> > > > > > >> >> > > > quickly
> > > > > > >> >> > > > > > hash
> > > > > > >> >> > > > > > > > out
> > > > > > >> >> > > > > > > > > >> details.
> > > > > > >> >> > > > > > > > > >>
> > > > > > >> >> > > > > > > > > >> Thanks,
> > > > > > >> >> > > > > > > > > >> Harsha
> > > > > > >> >> > > > > > > > > >>
> > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> > > > > > brahmbhatt
> > > > > > >> wrote:
> > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo system
> uses
> > > so
> > > > no
> > > > > > >> good
> > > > > > >> >> > reason
> > > > > > >> >> > > > > > really.
> > > > > > >> >> > > > > > > > We
> > > > > > >> >> > > > > > > > > >> > could
> > > > > > >> >> > > > > > > > > >> > change it to whatever is the newest
> > > > > recommended
> > > > > > >> standard
> > > > > > >> >> > > is.
> > > > > > >> >> > > > > > > > > >> >
> > > > > > >> >> > > > > > > > > >> > Thanks
> > > > > > >> >> > > > > > > > > >> > Parth
> > > > > > >> >> > > > > > > > > >> >
> > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM,
> Ismael
> > > > Juma <
> > > > > > >> >> > > > > ismael@juma.me.uk
> > > > > > >> >> > > > > > >
> > > > > > >> >> > > > > > > > > wrote:
> > > > > > >> >> > > > > > > > > >> >
> > > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started
> > > > reviewing
> > > > > > >> this and
> > > > > > >> >> > > may
> > > > > > >> >> > > > > have
> > > > > > >> >> > > > > > > > > >> additional
> > > > > > >> >> > > > > > > > > >> > > questions later. The immediate
> > question
> > > > that
> > > > > > >> came to
> > > > > > >> >> > > mind
> > > > > > >> >> > > > is
> > > > > > >> >> > > > > > our
> > > > > > >> >> > > > > > > > > >> choice of
> > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked
> > as
> > > > > > >> OBSOLETE in
> > > > > > >> >> > the
> > > > > > >> >> > > > IANA
> > > > > > >> >> > > > > > > > > Registry
> > > > > > >> >> > > > > > > > > >> of
> > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC
> > > > (2831)
> > > > > > has
> > > > > > >> been
> > > > > > >> >> > > moved
> > > > > > >> >> > > > > to
> > > > > > >> >> > > > > > > > > Historic
> > > > > > >> >> > > > > > > > > >> > > status:
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > >
> > > > > > >>
> > > >
> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind that
> > > choice?
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > Thanks,
> > > > > > >> >> > > > > > > > > >> > > Ismael
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM,
> Gwen
> > > > > > Shapira <
> > > > > > >> >> > > > > > > gwen@confluent.io
> > > > > > >> >> > > > > > > > >
> > > > > > >> >> > > > > > > > > >> wrote:
> > > > > > >> >> > > > > > > > > >> > >
> > > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that even
> > > though
> > > > > > >> delegation
> > > > > > >> >> > > > tokens
> > > > > > >> >> > > > > > > are a
> > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very strongly
> > > about
> > > > > not
> > > > > > >> adding
> > > > > > >> >> > > > > > dependency
> > > > > > >> >> > > > > > > > on
> > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > >> >> > > > > > > > > >> > > > > when implementing delegation
> > tokens
> > > > for
> > > > > > >> Kafka. The
> > > > > > >> >> > > KIP
> > > > > > >> >> > > > > > > doesn't
> > > > > > >> >> > > > > > > > > >> imply
> > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if you can
> > > > > clarify...
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so
> no
> > > one
> > > > > will
> > > > > > >> read
> > > > > > >> >> > the
> > > > > > >> >> > > > KIP
> > > > > > >> >> > > > > > and
> > > > > > >> >> > > > > > > > > panic
> > > > > > >> >> > > > > > > > > >> > > > three weeks before the next
> > release...
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation token at
> > any
> > > > > time
> > > > > > >> after
> > > > > > >> >> > > > > > > > authenticating?
> > > > > > >> >> > > > > > > > > >> only
> > > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *As long as you are
> authenticated
> > > you
> > > > > can
> > > > > > >> get
> > > > > > >> >> > > > delegation
> > > > > > >> >> > > > > > > > tokens.
> > > > > > >> >> > > > > > > > > >> We
> > > > > > >> >> > > > > > > > > >> > > need
> > > > > > >> >> > > > > > > > > >> > > > to
> > > > > > >> >> > > > > > > > > >> > > > > discuss if a client
> authenticated
> > > > using
> > > > > > >> delegation
> > > > > > >> >> > > > > token,
> > > > > > >> >> > > > > > > can
> > > > > > >> >> > > > > > > > > also
> > > > > > >> >> > > > > > > > > >> > > > acquire
> > > > > > >> >> > > > > > > > > >> > > > > delegation token again or not.
> > Also
> > > > > there
> > > > > > >> is the
> > > > > > >> >> > > > > question
> > > > > > >> >> > > > > > of
> > > > > > >> >> > > > > > > > do
> > > > > > >> >> > > > > > > > > we
> > > > > > >> >> > > > > > > > > >> > > allow
> > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation
> token
> > > or
> > > > we
> > > > > > >> want
> > > > > > >> >> > > specific
> > > > > > >> >> > > > > > ACLs
> > > > > > >> >> > > > > > > (I
> > > > > > >> >> > > > > > > > > >> think
> > > > > > >> >> > > > > > > > > >> > > its
> > > > > > >> >> > > > > > > > > >> > > > an
> > > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > I think we are debating two
> options:
> > > > > Either
> > > > > > >> require
> > > > > > >> >> > > > > Kerberos
> > > > > > >> >> > > > > > > > auth
> > > > > > >> >> > > > > > > > > >> for
> > > > > > >> >> > > > > > > > > >> > > > renewal or require non-owners to
> > > renew.
> > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler
> (it
> > > > > > basically
> > > > > > >> >> > require
> > > > > > >> >> > > a
> > > > > > >> >> > > > > "job
> > > > > > >> >> > > > > > > > > master"
> > > > > > >> >> > > > > > > > > >> > > > to take responsibility for the
> > > renewal,
> > > > it
> > > > > > >> will have
> > > > > > >> >> > > its
> > > > > > >> >> > > > > own
> > > > > > >> >> > > > > > > > > >> identity
> > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is the
> > correct
> > > > > > design
> > > > > > >> >> > pattern
> > > > > > >> >> > > > > > anyway.
> > > > > > >> >> > > > > > > > For
> > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
> > coordinate
> > > > > > >> renewals?),
> > > > > > >> >> > but
> > > > > > >> >> > > > it
> > > > > > >> >> > > > > is
> > > > > > >> >> > > > > > > > hard
> > > > > > >> >> > > > > > > > > to
> > > > > > >> >> > > > > > > > > >> > > > debate simplicity without looking
> at
> > > the
> > > > > > code
> > > > > > >> >> > changes
> > > > > > >> >> > > > > > > required.
> > > > > > >> >> > > > > > > > If
> > > > > > >> >> > > > > > > > > >> you
> > > > > > >> >> > > > > > > > > >> > > > have a draft of how the "require
> > > > Kerberos"
> > > > > > >> will look
> > > > > > >> >> > > in
> > > > > > >> >> > > > > > Kafka
> > > > > > >> >> > > > > > > > > code,
> > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > > * My understanding is that
> tokens
> > > will
> > > > > > >> propagate
> > > > > > >> >> > via
> > > > > > >> >> > > > ZK
> > > > > > >> >> > > > > > but
> > > > > > >> >> > > > > > > > > >> without
> > > > > > >> >> > > > > > > > > >> > > > > additional changes to
> > UpdateMetadata
> > > > > > >> protocol,
> > > > > > >> >> > > > correct?
> > > > > > >> >> > > > > > > > Clients
> > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL
> auth
> > > > > failure
> > > > > > >> (IIRC),
> > > > > > >> >> > > but
> > > > > > >> >> > > > > > since
> > > > > > >> >> > > > > > > > the
> > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between brokers
> > > > asynch,
> > > > > > we
> > > > > > >> will
> > > > > > >> >> > > need
> > > > > > >> >> > > > to
> > > > > > >> >> > > > > > > > retry a
> > > > > > >> >> > > > > > > > > >> bit
> > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth
> due
> > to
> > > > > > timing
> > > > > > >> >> > issues.
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives
> > > right
> > > > > > now.
> > > > > > >> The
> > > > > > >> >> > > > current
> > > > > > >> >> > > > > > > > > documented
> > > > > > >> >> > > > > > > > > >> > > > approach
> > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does
> not
> > > > > require
> > > > > > >> any
> > > > > > >> >> > > changes
> > > > > > >> >> > > > > to
> > > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative
> approach
> > > can
> > > > > > remove
> > > > > > >> >> > > zookeeper
> > > > > > >> >> > > > > > > > > dependency
> > > > > > >> >> > > > > > > > > >> as
> > > > > > >> >> > > > > > > > > >> > > well
> > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> > > > > discussion
> > > > > > >> call.*
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you
> > want
> > > to
> > > > > > ping
> > > > > > >> Jun to
> > > > > > >> >> > > > > > arrange a
> > > > > > >> >> > > > > > > > > call?
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of
> > > > having
> > > > > > >> just the
> > > > > > >> >> > > > > > controller
> > > > > > >> >> > > > > > > > > issue
> > > > > > >> >> > > > > > > > > >> the
> > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid
> > syncing
> > > a
> > > > > > shared
> > > > > > >> >> > secret.
> > > > > > >> >> > > > Not
> > > > > > >> >> > > > > > > sure
> > > > > > >> >> > > > > > > > if
> > > > > > >> >> > > > > > > > > >> we
> > > > > > >> >> > > > > > > > > >> > > > > want to continue the discussion
> > here
> > > > or
> > > > > on
> > > > > > >> the
> > > > > > >> >> > > wiki. I
> > > > > > >> >> > > > > > think
> > > > > > >> >> > > > > > > > > that
> > > > > > >> >> > > > > > > > > >> we
> > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem of
> "token
> > > > > > >> distribution"
> > > > > > >> >> > > from
> > > > > > >> >> > > > > > > "shared
> > > > > > >> >> > > > > > > > > >> secret
> > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
> > controller
> > > > as
> > > > > > the
> > > > > > >> only
> > > > > > >> >> > > token
> > > > > > >> >> > > > > > > > generator
> > > > > > >> >> > > > > > > > > >> to
> > > > > > >> >> > > > > > > > > >> > > > > solve the second issue, while
> > still
> > > > > using
> > > > > > >> ZK async
> > > > > > >> >> > > to
> > > > > > >> >> > > > > > > > distribute
> > > > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous
> > Email
> > > I
> > > > am
> > > > > > >> fine with
> > > > > > >> >> > > > that
> > > > > > >> >> > > > > > > > approach
> > > > > > >> >> > > > > > > > > >> as
> > > > > > >> >> > > > > > > > > >> > > long
> > > > > > >> >> > > > > > > > > >> > > > as
> > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra
> complexity
> > > of
> > > > > > >> >> > > adding/updating
> > > > > > >> >> > > > > APIS
> > > > > > >> >> > > > > > > > adds
> > > > > > >> >> > > > > > > > > >> enough
> > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with the
> > > > controller
> > > > > > >> approach
> > > > > > >> >> > is
> > > > > > >> >> > > > > > secret
> > > > > > >> >> > > > > > > > > >> rotation
> > > > > > >> >> > > > > > > > > >> > > can
> > > > > > >> >> > > > > > > > > >> > > > be
> > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and would not
> > > > require
> > > > > > >> >> > > deployment. *
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra
> complexity
> > > (or
> > > > > > point
> > > > > > >> me to
> > > > > > >> >> > > the
> > > > > > >> >> > > > > > email
> > > > > > >> >> > > > > > > I
> > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are
> required?
> > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients can
> > > > already
> > > > > > >> find the
> > > > > > >> >> > > > > > controller
> > > > > > >> >> > > > > > > > from
> > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned
> > > about
> > > > > > >> controller
> > > > > > >> >> > > > load.
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of
> forcing
> > > > > > kerberos
> > > > > > >> auth
> > > > > > >> >> > for
> > > > > > >> >> > > > > > > renwal, I
> > > > > > >> >> > > > > > > > > >> think
> > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer the
> > the
> > > > > > request
> > > > > > >> >> > content
> > > > > > >> >> > > > in
> > > > > > >> >> > > > > a
> > > > > > >> >> > > > > > > > pretty
> > > > > > >> >> > > > > > > > > >> ugly
> > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to
> > > > > non-owner
> > > > > > >> is
> > > > > > >> >> > > better.
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary
> evil.
> > > > While
> > > > > > >> this will
> > > > > > >> >> > > make
> > > > > > >> >> > > > > the
> > > > > > >> >> > > > > > > > kafka
> > > > > > >> >> > > > > > > > > >> code
> > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward ,
> forcing
> > > > > renewer
> > > > > > >> to
> > > > > > >> >> > > > non-owner
> > > > > > >> >> > > > > > > pushes
> > > > > > >> >> > > > > > > > > >> the code
> > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes it
> > even
> > > > > > harder
> > > > > > >> to
> > > > > > >> >> > > > > > integrate. *
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't think
> > the
> > > > > > >> "renewal by
> > > > > > >> >> > > > other"
> > > > > > >> >> > > > > > > > approach
> > > > > > >> >> > > > > > > > > >> is
> > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we
> expect
> > to
> > > > use
> > > > > > >> >> > delegation
> > > > > > >> >> > > > > tokens
> > > > > > >> >> > > > > > > > since
> > > > > > >> >> > > > > > > > > >> > > > they will have an app-master of
> some
> > > > sort
> > > > > > who
> > > > > > >> >> > > requested
> > > > > > >> >> > > > > the
> > > > > > >> >> > > > > > > > token
> > > > > > >> >> > > > > > > > > to
> > > > > > >> >> > > > > > > > > >> > > > begin with.
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > The response for my question on
> > how
> > > > > > multiple
> > > > > > >> >> > > > identities
> > > > > > >> >> > > > > > will
> > > > > > >> >> > > > > > > > be
> > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to
> me -
> > > > > AFAIK,
> > > > > > >> we don't
> > > > > > >> >> > > > > > > > authenticate
> > > > > > >> >> > > > > > > > > >> each
> > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> > > connections.
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > *We authenticate connections,
> and
> > > only
> > > > > > when
> > > > > > >> they
> > > > > > >> >> > are
> > > > > > >> >> > > > > being
> > > > > > >> >> > > > > > > > > >> established.
> > > > > > >> >> > > > > > > > > >> > > > Let
> > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a
> > question,
> > > > in
> > > > > > >> absence of
> > > > > > >> >> > > > > > > delegation
> > > > > > >> >> > > > > > > > > >> tokens if
> > > > > > >> >> > > > > > > > > >> > > > we
> > > > > > >> >> > > > > > > > > >> > > > > had to support the use case
> using
> > > user
> > > > > > >> TGT's how
> > > > > > >> >> > > would
> > > > > > >> >> > > > > we
> > > > > > >> >> > > > > > do
> > > > > > >> >> > > > > > > > it?
> > > > > > >> >> > > > > > > > > >> My
> > > > > > >> >> > > > > > > > > >> > > point
> > > > > > >> >> > > > > > > > > >> > > > > was it would be no different
> with
> > > > > > delegation
> > > > > > >> >> > tokens.
> > > > > > >> >> > > > The
> > > > > > >> >> > > > > > use
> > > > > > >> >> > > > > > > > > case
> > > > > > >> >> > > > > > > > > >> you
> > > > > > >> >> > > > > > > > > >> > > are
> > > > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > > > impersonation.*
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the
> > things
> > > > > that
> > > > > > >> >> > delegation
> > > > > > >> >> > > > > > tokens
> > > > > > >> >> > > > > > > > > >> handled.
> > > > > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > Gwen
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > >
> > > > > > >> >> > > > > > > > > >> > > > > Thanks
> > > > > > >> >> > > > > > > > > >> > > > > Parth
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19
> AM,
> > > Gwen
> > > > > > >> Shapira <
> > > > > > >> >> > > > > > > > > gwen@confluent.io
> > > > > > >> >> > > > > > > > > >> >
> > > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > > >> >> > > > > > > > > >> > > > >
> > > > > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> Few more comments:
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse
> > section
> > > > > > >> doesn't seem
> > > > > > >> >> > to
> > > > > > >> >> > > > > have
> > > > > > >> >> > > > > > > good
> > > > > > >> >> > > > > > > > > >> > > > >> description of the changes to
> the
> > > > Kafka
> > > > > > >> Protocol.
> > > > > > >> >> > > > > Sounds
> > > > > > >> >> > > > > > > like
> > > > > > >> >> > > > > > > > > >> you are
> > > > > > >> >> > > > > > > > > >> > > > >> proposing new
> > > DelegationTokenRequest
> > > > > and
> > > > > > >> >> > > > > > RenewTokenRequest
> > > > > > >> >> > > > > > > > (and
> > > > > > >> >> > > > > > > > > >> > > > >> matching responses), without
> > > > detailing
> > > > > > the
> > > > > > >> >> > contents
> > > > > > >> >> > > > of
> > > > > > >> >> > > > > > the
> > > > > > >> >> > > > > > > > > >> requests
> > > > > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you
> > show
> > > > the
> > > > > > >> class
> > > > > > >> >> > > > interface,
> > > > > > >> >> > > > > > but
> > > > > > >> >> > > > > > > > not
> > > > > > >> >> > > > > > > > > >> the
> > > > > > >> >> > > > > > > > > >> > > > >> underlying protocol. This could
> > be
> > > > seen
> > > > > > as
> > > > > > >> an
> > > > > > >> >> > > > > > > implementation
> > > > > > >> >> > > > > > > > > >> detail,
> > > > > > >> >> > > > > > > > > >> > > > >> but since the binary protocol
> is
> > > what
> > > > > we
> > > > > > >> provide
> > > > > > >> >> > to
> > > > > > >> >> > > > > > > non-Java
> > > > > > >> >> > > > > > > > > >> clients,
> > > > > > >> >> > > > > > > > > >> > > > >> we need to show the changes
> > there.
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds
> like
> > > > > > >> >> > > > > > > > delegationTokenRequestHandler?
> > > > > > >> >> > > > > > > > > >> Is it
> > > > > > >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi?
> > or
> > > > > > Client?
> > > > > > >> Its
> > > > > > >> >> > > > > unclear...
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even
> > > > though
> > > > > > >> delegation
> > > > > > >> >> > > > > tokens
> > > > > > >> >> > > > > > > are
> > > > > > >> >> > > > > > > > a
> > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > >> >> > > > > > > > > >> > > > >> innovation, I feel very
> strongly
> > > > about
> > > > > > not
> > > > > > >> adding
> > > > > > >> >> > > > > > > dependency
> > > > > > >> >> > > > > > > > on
> > > > > > >> >> > > > > > > > > >> Hadoop
> > > > > > >> >> > > > > > > > > >> > > > >> when implementing delegation
> > tokens
> > > > for
> > > > > > >> Kafka.
> > > > > > >> >> > The
> > > > > > >> >> > > > KIP
> > > > > > >> >> > > > > > > > doesn't
> > > > > > >> >> > > > > > > > > >> imply
> > > > > > >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> > > > > > clarify...
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token
> at
> > > any
> > > > > time
> > > > > > >> after
> > > > > > >> >> > > > > > > > authenticating?
> > > > > > >> >> > > > > > > > > >> only
> > > > > > >> >> > > > > > > > > >> > > > >> immediately after?
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * My understanding is that
> tokens
> > > > will
> > > > > > >> propagate
> > > > > > >> >> > > via
> > > > > > >> >> > > > ZK
> > > > > > >> >> > > > > > but
> > > > > > >> >> > > > > > > > > >> without
> > > > > > >> >> > > > > > > > > >> > > > >> additional changes to
> > > UpdateMetadata
> > > > > > >> protocol,
> > > > > > >> >> > > > correct?
> > > > > > >> >> > > > > > > > Clients
> > > > > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL
> > auth
> > > > > > failure
> > > > > > >> >> > (IIRC),
> > > > > > >> >> > > > but
> > > > > > >> >> > > > > > > since
> > > > > > >> >> > > > > > > > > the
> > > > > > >> >> > > > > > > > > >> > > > >> tokens propagate between
> brokers
> > > > > asynch,
> > > > > > >> we will
> > > > > > >> >> > > need
> > > > > > >> >> > > > > to
> > > > > > >> >> > > > > > > > retry
> > > > > > >> >> > > > > > > > > a
> > > > > > >> >> > > > > > > > > >> bit
> > > > > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth
> due
> > > to
> > > > > > timing
> > > > > > >> >> > issues.
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients
> > not
> > > > > > >> touching ZK
> > > > > > >> >> > > > directly
> > > > > > >> >> > > > > > :)
> > > > > > >> >> > > > > > > > > >> > > > >>
> > > > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion
> of
> > > > having
> > > > > > >> just the
> > > > > > >> >> > > > > > controller
> > > > > > >> >> > > > > > > > > >> issue the
> > > > > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid
> > > syncing a
> > > > > > >> shared
> > > > > > >> >> > > secret.
> > > > > > >> >> > > > > Not
> > > > > > >> >> > > > > > > sure
> > > > > > >> >> > > > > > > > > if
> > > > > > >> >> > > > > > > > > >> we
> > > > > > >> >> > > > > > > > > >> > > > >> want to continue the discussion
> > > here
> > > > or
> > > > > > on
> > > > > > >> the
> > > > > > >> >> > > wiki.
> > > > > > >> >> > >
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Grant Henke <gh...@cloudera.com>.
Hi Parth,
Are you still working on this? If you need any help please don't hesitate
to ask.
Thanks,
Grant
On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <ju...@confluent.io> wrote:
> Parth,
>
> Thanks for the reply.
>
> It makes sense to only allow the renewal by users that authenticated using
> *non* delegation token mechanism. Then, should we make the renewal a list?
> For example, in the case of rest proxy, it will be useful for every
> instance of rest proxy to be able to renew the tokens.
>
> It would be clearer if we can document the request protocol like
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
> .
>
> It would also be useful to document the client APIs.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
> brahmbhatt.parth@gmail.com> wrote:
>
> > Hi,
> >
> > I am suggesting that we will only allow the renewal by users that
> > authenticated using *non* delegation token mechanism. For example, If
> user
> > Alice authenticated using kerberos and requested delegation tokens, only
> > user Alice authenticated via non delegation token mechanism can renew.
> > Clients that have access to delegation tokens can not issue renewal
> > request for renewing their own token and this is primarily important to
> > reduce the time window for which a compromised token will be valid.
> >
> > To clarify, Yes any authenticated user can request delegation tokens but
> > even here I would recommend to avoid creating a chain where a client
> > authenticated via delegation token request for more delegation tokens.
> > Basically anyone can request delegation token, as long as they
> authenticate
> > via a non delegation token mechanism.
> >
> > Aren't classes listed here
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-PublicInterfaces
> > >
> > sufficient?
> >
> > Thanks
> > Parth
> >
> >
> >
> > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Parth,
> > >
> > > Thanks for the reply. A couple of comments inline below.
> > >
> > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > > brahmbhatt.parth@gmail.com> wrote:
> > >
> > > > 1. Who / how are tokens renewed? By original requester only? or using
> > > > Kerberos
> > > > auth only?
> > > > My recommendation is to do this only using Kerberos auth and only
> threw
> > > the
> > > > renewer specified during the acquisition request.
> > > >
> > > >
> > > Hmm, not sure that I follow this. Are you saying that any client
> > > authenticated with the delegation token can renew, i.e. there is no
> > renewer
> > > needed?
> > >
> > > Also, just to be clear, any authenticated client (either through SASL
> or
> > > SSL) can request a delegation token for the authenticated user, right?
> > >
> > >
> > > > 2. Are tokens stored on each broker or in ZK?
> > > > My recommendation is still to store in ZK or not store them at all.
> The
> > > > whole controller based distribution is too much overhead with not
> much
> > to
> > > > achieve.
> > > >
> > > > 3. How are tokens invalidated / expired?
> > > > Either by expiration time out or through an explicit request to
> > > invalidate.
> > > >
> > > > 4. Which encryption algorithm is used?
> > > > SCRAM
> > > >
> > > > 5. What is the impersonation proposal (it wasn't in the KIP but was
> > > > discussed
> > > > in this thread)?
> > > > There is no imperonation proposal. I tried and explained how its a
> > > > different problem and why its not really necessary to discuss that as
> > > part
> > > > of this KIP. This KIP will not support any impersonation, it will
> just
> > > be
> > > > another way to authenticate.
> > > >
> > > > 6. Do we need new ACLs, if so - for what actions?
> > > > We do not need new ACLs.
> > > >
> > > >
> > > Could we document the format of the new request/response and their
> > > associated Resource and Operation for ACL?
> > >
> > >
> > > > 7. How would the delegation token be configured in the client?
> > > > Should be through config. I wasn't planning on supporting JAAS for
> > > tokens.
> > > > I don't believe hadoop does this either.
> > > >
> > > > Thanks
> > > > Parth
> > > >
> > > >
> > > >
> > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Harsha,
> > > > >
> > > > > Another question.
> > > > >
> > > > > 9. How would the delegation token be configured in the client? The
> > > > standard
> > > > > way is to do this through JAAS. However, we will need to think
> > through
> > > if
> > > > > this is convenient in a shared environment. For example, when a new
> > > task
> > > > is
> > > > > added to a Storm worker node, do we need to dynamically add a new
> > > section
> > > > > in the JAAS file? It may be more convenient if we can pass in the
> > token
> > > > > through the config directly w/o going through JAAS.
> > > > >
> > > > > Are you or Parth still actively working on this KIP?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > >
> > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Just to add on that list.
> > > > > >
> > > > > > 2. It would be good to document the format of the data stored in
> > ZK.
> > > > > > 7. Earlier, there was a discussion on whether the tokens should
> be
> > > > > > propagated through ZK like config/acl/quota, or through the
> > > controller.
> > > > > > Currently, the controller is only designed for propagating topic
> > > > > metadata,
> > > > > > but not other data.
> > > > > > 8. Should we use SCRAM to send the token instead of DIGEST-MD5
> > since
> > > > it's
> > > > > > deprecated?
> > > > > >
> > > > > > Also, the images in the wiki seem broken.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <
> gwen@confluent.io>
> > > > > wrote:
> > > > > >
> > > > > >> From what I can see, remaining questions are:
> > > > > >>
> > > > > >> 1. Who / how are tokens renewed? By original requester only? or
> > > using
> > > > > >> Kerberos auth only?
> > > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > > >> 3. How are tokens invalidated / expired?
> > > > > >> 4. Which encryption algorithm is used?
> > > > > >> 5. What is the impersonation proposal (it wasn't in the KIP but
> > was
> > > > > >> discussed in this thread)?
> > > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > > >>
> > > > > >> Gwen
> > > > > >>
> > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> > > > > >> > Jun & Ismael,
> > > > > >> > Unfortunately I couldn't attend the
> KIP
> > > > > meeting
> > > > > >> > when delegation tokens discussed.
> > > > Appreciate
> > > > > if
> > > > > >> > you can update the thread if you have
> > any
> > > > > >> > further questions.
> > > > > >> > Thanks,
> > > > > >> > Harsha
> > > > > >> >
> > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > > >> >> It seems that the links to images in the KIP are broken.
> > > > > >> >>
> > > > > >> >> Liquan
> > > > > >> >>
> > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > > >> >>
> > > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > > >> >> > In the current proposal we only allow a user to get
> > delegation
> > > > > token
> > > > > >> for
> > > > > >> >> > the identity that it authenticated as using another
> > mechanism,
> > > > i.e.
> > > > > >> A user
> > > > > >> >> > that authenticate using a keytab for principal
> > > user1@EXAMPLE.COM
> > > > > >> will get
> > > > > >> >> > delegation tokens for that user only. In future I think we
> > will
> > > > > have
> > > > > >> to
> > > > > >> >> > extend support such that we allow some set of users (
> > > > > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to
> > > > acquire
> > > > > >> >> > delegation tokens on behalf of other users whose identity
> > they
> > > > have
> > > > > >> >> > verified independently. Kafka brokers will have ACLs to
> > > control
> > > > > >> which
> > > > > >> >> > users are allowed to impersonate other users and get tokens
> > on
> > > > > >> behalf of
> > > > > >> >> > them. Overall Impersonation is a whole different problem in
> > my
> > > > > >> opinion and
> > > > > >> >> > I think we can tackle it in separate KIP.
> > > > > >> >> >
> > > > > >> >> > 111. What's the typical rate of getting and renewing
> > delegation
> > > > > >> tokens?
> > > > > >> >> > Typically this should be very very low, 1 request per
> minute
> > > is a
> > > > > >> >> > relatively high estimate. However it depends on the token
> > > > > >> expiration. I am
> > > > > >> >> > less worried about the extra load it puts on controller vs
> > the
> > > > > added
> > > > > >> >> > complexity and the value it offers.
> > > > > >> >> >
> > > > > >> >> > Thanks
> > > > > >> >> > Parth
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > > ismael@juma.me.uk>
> > > > > >> wrote:
> > > > > >> >> >
> > > > > >> >> > > Thanks Rajini. It would probably require a separate KIP
> as
> > it
> > > > > will
> > > > > >> >> > > introduce user visible changes. We could also update
> KIP-48
> > > to
> > > > > >> have this
> > > > > >> >> > > information, but it seems cleaner to do it separately. We
> > can
> > > > > >> discuss
> > > > > >> >> > that
> > > > > >> >> > > in the KIP call today.
> > > > > >> >> > >
> > > > > >> >> > > Ismael
> > > > > >> >> > >
> > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > > >> >> > >
> > > > > >> >> > > > Ismael,
> > > > > >> >> > > >
> > > > > >> >> > > > I have created a JIRA (
> > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that need
> > > another
> > > > > >> KIP? If
> > > > > >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA
> > > that
> > > > > gets
> > > > > >> >> > > reviewed
> > > > > >> >> > > > when the PR is ready?
> > > > > >> >> > > >
> > > > > >> >> > > > Thank you,
> > > > > >> >> > > >
> > > > > >> >> > > > Rajini
> > > > > >> >> > > >
> > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > > ismael@juma.me.uk>
> > > > > >> >> > wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > > > >> >> > > > >
> > > > > >> >> > > > > Gwen had independently mentioned this as a SASL
> > mechanism
> > > > > that
> > > > > >> might
> > > > > >> >> > be
> > > > > >> >> > > > > useful for Kafka and I have been meaning to check it
> in
> > > > more
> > > > > >> detail.
> > > > > >> >> > > Good
> > > > > >> >> > > > > to know that you are willing to contribute an
> > > > implementation.
> > > > > >> Maybe
> > > > > >> >> > we
> > > > > >> >> > > > > should file a separate JIRA for this?
> > > > > >> >> > > > >
> > > > > >> >> > > > > Ismael
> > > > > >> >> > > > >
> > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > > >> >> > > > >
> > > > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> > > > Mechanism)
> > > > > >> is a
> > > > > >> >> > > better
> > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> > > > > built-in
> > > > > >> SCRAM
> > > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy to
> add
> > > > > support
> > > > > >> in
> > > > > >> >> > Kafka
> > > > > >> >> > > > > since
> > > > > >> >> > > > > > it would be a useful mechanism to support anyway.
> > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the
> > > > protocol
> > > > > >> for
> > > > > >> >> > > > > > SCRAM-SHA-256.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > > jun@confluent.io
> > > > > >
> > > > > >> wrote:
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > > Parth,
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > Thanks for the explanation. A couple of more
> > > questions.
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > 111. What's the typical rate of getting and
> > renewing
> > > > > >> delegation
> > > > > >> >> > > > tokens?
> > > > > >> >> > > > > > > That may have an impact on whether they should be
> > > > > directed
> > > > > >> to the
> > > > > >> >> > > > > > > controller.
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > Jun
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth
> brahmbhatt <
> > > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > > Hi Jun,
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > Thanks for reviewing.
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > * We could add a Cluster action to add acls on
> > who
> > > > can
> > > > > >> request
> > > > > >> >> > > > > > delegation
> > > > > >> >> > > > > > > > tokens. I don't see the use case for that yet
> but
> > > > down
> > > > > >> the line
> > > > > >> >> > > > when
> > > > > >> >> > > > > we
> > > > > >> >> > > > > > > > start supporting getDelegationTokenAs it will
> be
> > > > > >> necessary.
> > > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > > used/distributed
> > > > > >> over
> > > > > >> >> > secure
> > > > > >> >> > > > > > > channels.
> > > > > >> >> > > > > > > > * Depending on what design we end up choosing
> > > > > >> Invalidation will
> > > > > >> >> > > be
> > > > > >> >> > > > > > > > responsibility of every broker or controller.
> > > > > >> >> > > > > > > > * I am not sure if I documented somewhere that
> > > > > >> invalidation
> > > > > >> >> > will
> > > > > >> >> > > > > > directly
> > > > > >> >> > > > > > > > go through zookeeper but that is not the
> intent.
> > > > > >> Invalidation
> > > > > >> >> > > will
> > > > > >> >> > > > > > either
> > > > > >> >> > > > > > > > be request based or due to expiration. No
> direct
> > > > > >> zookeeper
> > > > > >> >> > > > > interaction
> > > > > >> >> > > > > > > from
> > > > > >> >> > > > > > > > any client.
> > > > > >> >> > > > > > > > * "Broker also stores the DelegationToken
> without
> > > the
> > > > > >> hmac in
> > > > > >> >> > the
> > > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion. The
> sole
> > > > > >> purpose of
> > > > > >> >> > > > > zookeeper
> > > > > >> >> > > > > > in
> > > > > >> >> > > > > > > > this design is as distribution channel for
> tokens
> > > > > >> between all
> > > > > >> >> > > > brokers
> > > > > >> >> > > > > > > and a
> > > > > >> >> > > > > > > > layer that ensures only tokens that were
> > generated
> > > by
> > > > > >> making a
> > > > > >> >> > > > > request
> > > > > >> >> > > > > > > to a
> > > > > >> >> > > > > > > > broker will be accepted (more on this in second
> > > > > >> paragraph). The
> > > > > >> >> > > > token
> > > > > >> >> > > > > > > > consists of few elements (owner, renewer, uuid
> ,
> > > > > >> expiration,
> > > > > >> >> > > hmac)
> > > > > >> >> > > > ,
> > > > > >> >> > > > > > one
> > > > > >> >> > > > > > > of
> > > > > >> >> > > > > > > > which is the finally generated hmac but hmac it
> > > self
> > > > is
> > > > > >> >> > derivable
> > > > > >> >> > > > if
> > > > > >> >> > > > > > you
> > > > > >> >> > > > > > > > have all the other elements of the token +
> secret
> > > key
> > > > > to
> > > > > >> >> > generate
> > > > > >> >> > > > > hmac.
> > > > > >> >> > > > > > > > Given zookeeper does not provide SSL support we
> > do
> > > > not
> > > > > >> want the
> > > > > >> >> > > > > entire
> > > > > >> >> > > > > > > > token to be wire transferred to zookeeper as
> that
> > > > will
> > > > > >> be an
> > > > > >> >> > > > insecure
> > > > > >> >> > > > > > > wire
> > > > > >> >> > > > > > > > transfer. Instead we only store all the other
> > > > elements
> > > > > >> of a
> > > > > >> >> > > > > delegation
> > > > > >> >> > > > > > > > tokens. Brokers can read these elements and
> > because
> > > > > they
> > > > > >> also
> > > > > >> >> > > have
> > > > > >> >> > > > > > access
> > > > > >> >> > > > > > > > to secret key they will be able to generate
> hmac
> > on
> > > > > >> their end.
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > One of the alternative proposed is to avoid
> > > zookeeper
> > > > > >> >> > > altogether. A
> > > > > >> >> > > > > > > Client
> > > > > >> >> > > > > > > > will call broker with required information
> > (owner,
> > > > > >> renwer,
> > > > > >> >> > > > > expiration)
> > > > > >> >> > > > > > > and
> > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't
> store
> > > this
> > > > > in
> > > > > >> >> > > zookeeper.
> > > > > >> >> > > > > > From
> > > > > >> >> > > > > > > > this point a client can contact any broker with
> > all
> > > > the
> > > > > >> >> > > delegation
> > > > > >> >> > > > > > token
> > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid)
> the
> > > > borker
> > > > > >> will
> > > > > >> >> > > > > regenerate
> > > > > >> >> > > > > > > the
> > > > > >> >> > > > > > > > hmac and as long as it matches with hmac
> > presented
> > > by
> > > > > >> client ,
> > > > > >> >> > > > broker
> > > > > >> >> > > > > > > will
> > > > > >> >> > > > > > > > allow the request to authenticate. Only
> problem
> > > with
> > > > > >> this
> > > > > >> >> > > approach
> > > > > >> >> > > > > is
> > > > > >> >> > > > > > if
> > > > > >> >> > > > > > > > the secret key is compromised any client can
> now
> > > > > generate
> > > > > >> >> > random
> > > > > >> >> > > > > tokens
> > > > > >> >> > > > > > > and
> > > > > >> >> > > > > > > > they will still be able to authenticate as any
> > user
> > > > > they
> > > > > >> like.
> > > > > >> >> > > with
> > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens
> acquired
> > > via
> > > > a
> > > > > >> broker
> > > > > >> >> > > > (using
> > > > > >> >> > > > > > some
> > > > > >> >> > > > > > > > auth scheme other than delegation token) will
> be
> > > > > >> accepted. We
> > > > > >> >> > > need
> > > > > >> >> > > > to
> > > > > >> >> > > > > > > > discuss which proposal makes more sense and we
> > can
> > > go
> > > > > >> over it
> > > > > >> >> > in
> > > > > >> >> > > > > > > tomorrow's
> > > > > >> >> > > > > > > > meeting.
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > Thanks
> > > > > >> >> > > > > > > > Parth
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > > > > >> jun@confluent.io>
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 100. This potentially can be useful for Kafka
> > > > Connect
> > > > > >> and
> > > > > >> >> > Kafka
> > > > > >> >> > > > > rest
> > > > > >> >> > > > > > > > proxy
> > > > > >> >> > > > > > > > > where a worker agent will need to run a task
> on
> > > > > behalf
> > > > > >> of a
> > > > > >> >> > > > client.
> > > > > >> >> > > > > > We
> > > > > >> >> > > > > > > > will
> > > > > >> >> > > > > > > > > likely need to change how those services use
> > > Kafka
> > > > > >> clients
> > > > > >> >> > > > > > > > > (producer/consumer). Instead of a shared
> client
> > > per
> > > > > >> worker,
> > > > > >> >> > we
> > > > > >> >> > > > will
> > > > > >> >> > > > > > > need
> > > > > >> >> > > > > > > > a
> > > > > >> >> > > > > > > > > client per user task since the authentication
> > > > happens
> > > > > >> at the
> > > > > >> >> > > > > > connection
> > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer will be
> > the
> > > > > >> workers.
> > > > > >> >> > So,
> > > > > >> >> > > we
> > > > > >> >> > > > > > > > probably
> > > > > >> >> > > > > > > > > need to allow multiple renewers. For Kafka
> rest
> > > > > proxy,
> > > > > >> the
> > > > > >> >> > > > renewer
> > > > > >> >> > > > > > can
> > > > > >> >> > > > > > > > > probably just be the creator of the token.
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 101. Do we need new acl on who can request
> > > > delegation
> > > > > >> tokens?
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 102. Do we recommend people to send
> delegation
> > > > tokens
> > > > > >> in an
> > > > > >> >> > > > > encrypted
> > > > > >> >> > > > > > > > > channel?
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 103. Who is responsible for expiring tokens,
> > > every
> > > > > >> broker?
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 104. For invalidating tokens, would it be
> > better
> > > to
> > > > > do
> > > > > >> it in
> > > > > >> >> > a
> > > > > >> >> > > > > > request
> > > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 105. The terminology of client in the wiki
> > > > sometimes
> > > > > >> refers
> > > > > >> >> > to
> > > > > >> >> > > > the
> > > > > >> >> > > > > > end
> > > > > >> >> > > > > > > > > client and some other times refers to the
> > client
> > > > > using
> > > > > >> the
> > > > > >> >> > > > > delegation
> > > > > >> >> > > > > > > > > tokens. It would be useful to distinguish
> > between
> > > > the
> > > > > >> two.
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > 106. Could you explain the sentence "Broker
> > also
> > > > > >> stores the
> > > > > >> >> > > > > > > > DelegationToken
> > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit
> > more? I
> > > > > >> thought the
> > > > > >> >> > > > > > > delegation
> > > > > >> >> > > > > > > > > token is the hmac.
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > Thanks,
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > Jun
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> > > > > >> jun@confluent.io>
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can
> > > > discuss
> > > > > >> this in
> > > > > >> >> > > the
> > > > > >> >> > > > > > > meeting
> > > > > >> >> > > > > > > > > > tomorrow.
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > > > Thanks,
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > > > Jun
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> > > > > >> kafka@harsha.io>
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > > >> Hi All,
> > > > > >> >> > > > > > > > > >> Can we have a KIP meeting
> around
> > > > this.
> > > > > >> The KIP
> > > > > >> >> > is
> > > > > >> >> > > > up
> > > > > >> >> > > > > > for
> > > > > >> >> > > > > > > > > >> sometime and if there are any
> > > > questions
> > > > > >> lets
> > > > > >> >> > > > quickly
> > > > > >> >> > > > > > hash
> > > > > >> >> > > > > > > > out
> > > > > >> >> > > > > > > > > >> details.
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > > >> Thanks,
> > > > > >> >> > > > > > > > > >> Harsha
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> > > > > brahmbhatt
> > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > That is what the hadoop echo system uses
> > so
> > > no
> > > > > >> good
> > > > > >> >> > reason
> > > > > >> >> > > > > > really.
> > > > > >> >> > > > > > > > We
> > > > > >> >> > > > > > > > > >> > could
> > > > > >> >> > > > > > > > > >> > change it to whatever is the newest
> > > > recommended
> > > > > >> standard
> > > > > >> >> > > is.
> > > > > >> >> > > > > > > > > >> >
> > > > > >> >> > > > > > > > > >> > Thanks
> > > > > >> >> > > > > > > > > >> > Parth
> > > > > >> >> > > > > > > > > >> >
> > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael
> > > Juma <
> > > > > >> >> > > > > ismael@juma.me.uk
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > > > wrote:
> > > > > >> >> > > > > > > > > >> >
> > > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started
> > > reviewing
> > > > > >> this and
> > > > > >> >> > > may
> > > > > >> >> > > > > have
> > > > > >> >> > > > > > > > > >> additional
> > > > > >> >> > > > > > > > > >> > > questions later. The immediate
> question
> > > that
> > > > > >> came to
> > > > > >> >> > > mind
> > > > > >> >> > > > is
> > > > > >> >> > > > > > our
> > > > > >> >> > > > > > > > > >> choice of
> > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked
> as
> > > > > >> OBSOLETE in
> > > > > >> >> > the
> > > > > >> >> > > > IANA
> > > > > >> >> > > > > > > > > Registry
> > > > > >> >> > > > > > > > > >> of
> > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC
> > > (2831)
> > > > > has
> > > > > >> been
> > > > > >> >> > > moved
> > > > > >> >> > > > > to
> > > > > >> >> > > > > > > > > Historic
> > > > > >> >> > > > > > > > > >> > > status:
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > >
> > > > > >>
> > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > What is the reasoning behind that
> > choice?
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > Thanks,
> > > > > >> >> > > > > > > > > >> > > Ismael
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> > > > > Shapira <
> > > > > >> >> > > > > > > gwen@confluent.io
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that even
> > though
> > > > > >> delegation
> > > > > >> >> > > > tokens
> > > > > >> >> > > > > > > are a
> > > > > >> >> > > > > > > > > >> Hadoop
> > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very strongly
> > about
> > > > not
> > > > > >> adding
> > > > > >> >> > > > > > dependency
> > > > > >> >> > > > > > > > on
> > > > > >> >> > > > > > > > > >> Hadoop
> > > > > >> >> > > > > > > > > >> > > > > when implementing delegation
> tokens
> > > for
> > > > > >> Kafka. The
> > > > > >> >> > > KIP
> > > > > >> >> > > > > > > doesn't
> > > > > >> >> > > > > > > > > >> imply
> > > > > >> >> > > > > > > > > >> > > > > such dependency, but if you can
> > > > clarify...
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no
> > one
> > > > will
> > > > > >> read
> > > > > >> >> > the
> > > > > >> >> > > > KIP
> > > > > >> >> > > > > > and
> > > > > >> >> > > > > > > > > panic
> > > > > >> >> > > > > > > > > >> > > > three weeks before the next
> release...
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation token at
> any
> > > > time
> > > > > >> after
> > > > > >> >> > > > > > > > authenticating?
> > > > > >> >> > > > > > > > > >> only
> > > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *As long as you are authenticated
> > you
> > > > can
> > > > > >> get
> > > > > >> >> > > > delegation
> > > > > >> >> > > > > > > > tokens.
> > > > > >> >> > > > > > > > > >> We
> > > > > >> >> > > > > > > > > >> > > need
> > > > > >> >> > > > > > > > > >> > > > to
> > > > > >> >> > > > > > > > > >> > > > > discuss if a client authenticated
> > > using
> > > > > >> delegation
> > > > > >> >> > > > > token,
> > > > > >> >> > > > > > > can
> > > > > >> >> > > > > > > > > also
> > > > > >> >> > > > > > > > > >> > > > acquire
> > > > > >> >> > > > > > > > > >> > > > > delegation token again or not.
> Also
> > > > there
> > > > > >> is the
> > > > > >> >> > > > > question
> > > > > >> >> > > > > > of
> > > > > >> >> > > > > > > > do
> > > > > >> >> > > > > > > > > we
> > > > > >> >> > > > > > > > > >> > > allow
> > > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation token
> > or
> > > we
> > > > > >> want
> > > > > >> >> > > specific
> > > > > >> >> > > > > > ACLs
> > > > > >> >> > > > > > > (I
> > > > > >> >> > > > > > > > > >> think
> > > > > >> >> > > > > > > > > >> > > its
> > > > > >> >> > > > > > > > > >> > > > an
> > > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > I think we are debating two options:
> > > > Either
> > > > > >> require
> > > > > >> >> > > > > Kerberos
> > > > > >> >> > > > > > > > auth
> > > > > >> >> > > > > > > > > >> for
> > > > > >> >> > > > > > > > > >> > > > renewal or require non-owners to
> > renew.
> > > > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> > > > > basically
> > > > > >> >> > require
> > > > > >> >> > > a
> > > > > >> >> > > > > "job
> > > > > >> >> > > > > > > > > master"
> > > > > >> >> > > > > > > > > >> > > > to take responsibility for the
> > renewal,
> > > it
> > > > > >> will have
> > > > > >> >> > > its
> > > > > >> >> > > > > own
> > > > > >> >> > > > > > > > > >> identity
> > > > > >> >> > > > > > > > > >> > > > anyway and I think this is the
> correct
> > > > > design
> > > > > >> >> > pattern
> > > > > >> >> > > > > > anyway.
> > > > > >> >> > > > > > > > For
> > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to
> coordinate
> > > > > >> renewals?),
> > > > > >> >> > but
> > > > > >> >> > > > it
> > > > > >> >> > > > > is
> > > > > >> >> > > > > > > > hard
> > > > > >> >> > > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > > debate simplicity without looking at
> > the
> > > > > code
> > > > > >> >> > changes
> > > > > >> >> > > > > > > required.
> > > > > >> >> > > > > > > > If
> > > > > >> >> > > > > > > > > >> you
> > > > > >> >> > > > > > > > > >> > > > have a draft of how the "require
> > > Kerberos"
> > > > > >> will look
> > > > > >> >> > > in
> > > > > >> >> > > > > > Kafka
> > > > > >> >> > > > > > > > > code,
> > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > > * My understanding is that tokens
> > will
> > > > > >> propagate
> > > > > >> >> > via
> > > > > >> >> > > > ZK
> > > > > >> >> > > > > > but
> > > > > >> >> > > > > > > > > >> without
> > > > > >> >> > > > > > > > > >> > > > > additional changes to
> UpdateMetadata
> > > > > >> protocol,
> > > > > >> >> > > > correct?
> > > > > >> >> > > > > > > > Clients
> > > > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth
> > > > failure
> > > > > >> (IIRC),
> > > > > >> >> > > but
> > > > > >> >> > > > > > since
> > > > > >> >> > > > > > > > the
> > > > > >> >> > > > > > > > > >> > > > > tokens propagate between brokers
> > > asynch,
> > > > > we
> > > > > >> will
> > > > > >> >> > > need
> > > > > >> >> > > > to
> > > > > >> >> > > > > > > > retry a
> > > > > >> >> > > > > > > > > >> bit
> > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth due
> to
> > > > > timing
> > > > > >> >> > issues.
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives
> > right
> > > > > now.
> > > > > >> The
> > > > > >> >> > > > current
> > > > > >> >> > > > > > > > > documented
> > > > > >> >> > > > > > > > > >> > > > approach
> > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does not
> > > > require
> > > > > >> any
> > > > > >> >> > > changes
> > > > > >> >> > > > > to
> > > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative approach
> > can
> > > > > remove
> > > > > >> >> > > zookeeper
> > > > > >> >> > > > > > > > > dependency
> > > > > >> >> > > > > > > > > >> as
> > > > > >> >> > > > > > > > > >> > > well
> > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> > > > discussion
> > > > > >> call.*
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you
> want
> > to
> > > > > ping
> > > > > >> Jun to
> > > > > >> >> > > > > > arrange a
> > > > > >> >> > > > > > > > > call?
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of
> > > having
> > > > > >> just the
> > > > > >> >> > > > > > controller
> > > > > >> >> > > > > > > > > issue
> > > > > >> >> > > > > > > > > >> the
> > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid
> syncing
> > a
> > > > > shared
> > > > > >> >> > secret.
> > > > > >> >> > > > Not
> > > > > >> >> > > > > > > sure
> > > > > >> >> > > > > > > > if
> > > > > >> >> > > > > > > > > >> we
> > > > > >> >> > > > > > > > > >> > > > > want to continue the discussion
> here
> > > or
> > > > on
> > > > > >> the
> > > > > >> >> > > wiki. I
> > > > > >> >> > > > > > think
> > > > > >> >> > > > > > > > > that
> > > > > >> >> > > > > > > > > >> we
> > > > > >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> > > > > >> distribution"
> > > > > >> >> > > from
> > > > > >> >> > > > > > > "shared
> > > > > >> >> > > > > > > > > >> secret
> > > > > >> >> > > > > > > > > >> > > > > distribution" and use the
> controller
> > > as
> > > > > the
> > > > > >> only
> > > > > >> >> > > token
> > > > > >> >> > > > > > > > generator
> > > > > >> >> > > > > > > > > >> to
> > > > > >> >> > > > > > > > > >> > > > > solve the second issue, while
> still
> > > > using
> > > > > >> ZK async
> > > > > >> >> > > to
> > > > > >> >> > > > > > > > distribute
> > > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous
> Email
> > I
> > > am
> > > > > >> fine with
> > > > > >> >> > > > that
> > > > > >> >> > > > > > > > approach
> > > > > >> >> > > > > > > > > >> as
> > > > > >> >> > > > > > > > > >> > > long
> > > > > >> >> > > > > > > > > >> > > > as
> > > > > >> >> > > > > > > > > >> > > > > we agree that the extra complexity
> > of
> > > > > >> >> > > adding/updating
> > > > > >> >> > > > > APIS
> > > > > >> >> > > > > > > > adds
> > > > > >> >> > > > > > > > > >> enough
> > > > > >> >> > > > > > > > > >> > > > > value. The advantage with the
> > > controller
> > > > > >> approach
> > > > > >> >> > is
> > > > > >> >> > > > > > secret
> > > > > >> >> > > > > > > > > >> rotation
> > > > > >> >> > > > > > > > > >> > > can
> > > > > >> >> > > > > > > > > >> > > > be
> > > > > >> >> > > > > > > > > >> > > > > automated,frequent and would not
> > > require
> > > > > >> >> > > deployment. *
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Can you detail the extra complexity
> > (or
> > > > > point
> > > > > >> me to
> > > > > >> >> > > the
> > > > > >> >> > > > > > email
> > > > > >> >> > > > > > > I
> > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> > > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients can
> > > already
> > > > > >> find the
> > > > > >> >> > > > > > controller
> > > > > >> >> > > > > > > > from
> > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned
> > about
> > > > > >> controller
> > > > > >> >> > > > load.
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> > > > > kerberos
> > > > > >> auth
> > > > > >> >> > for
> > > > > >> >> > > > > > > renwal, I
> > > > > >> >> > > > > > > > > >> think
> > > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer the
> the
> > > > > request
> > > > > >> >> > content
> > > > > >> >> > > > in
> > > > > >> >> > > > > a
> > > > > >> >> > > > > > > > pretty
> > > > > >> >> > > > > > > > > >> ugly
> > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to
> > > > non-owner
> > > > > >> is
> > > > > >> >> > > better.
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil.
> > > While
> > > > > >> this will
> > > > > >> >> > > make
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > > > kafka
> > > > > >> >> > > > > > > > > >> code
> > > > > >> >> > > > > > > > > >> > > > > pretty straight forward , forcing
> > > > renewer
> > > > > >> to
> > > > > >> >> > > > non-owner
> > > > > >> >> > > > > > > pushes
> > > > > >> >> > > > > > > > > >> the code
> > > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes it
> even
> > > > > harder
> > > > > >> to
> > > > > >> >> > > > > > integrate. *
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't think
> the
> > > > > >> "renewal by
> > > > > >> >> > > > other"
> > > > > >> >> > > > > > > > approach
> > > > > >> >> > > > > > > > > >> is
> > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we expect
> to
> > > use
> > > > > >> >> > delegation
> > > > > >> >> > > > > tokens
> > > > > >> >> > > > > > > > since
> > > > > >> >> > > > > > > > > >> > > > they will have an app-master of some
> > > sort
> > > > > who
> > > > > >> >> > > requested
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > > > token
> > > > > >> >> > > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > > begin with.
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > The response for my question on
> how
> > > > > multiple
> > > > > >> >> > > > identities
> > > > > >> >> > > > > > will
> > > > > >> >> > > > > > > > be
> > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to me -
> > > > AFAIK,
> > > > > >> we don't
> > > > > >> >> > > > > > > > authenticate
> > > > > >> >> > > > > > > > > >> each
> > > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> > connections.
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > *We authenticate connections, and
> > only
> > > > > when
> > > > > >> they
> > > > > >> >> > are
> > > > > >> >> > > > > being
> > > > > >> >> > > > > > > > > >> established.
> > > > > >> >> > > > > > > > > >> > > > Let
> > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a
> question,
> > > in
> > > > > >> absence of
> > > > > >> >> > > > > > > delegation
> > > > > >> >> > > > > > > > > >> tokens if
> > > > > >> >> > > > > > > > > >> > > > we
> > > > > >> >> > > > > > > > > >> > > > > had to support the use case using
> > user
> > > > > >> TGT's how
> > > > > >> >> > > would
> > > > > >> >> > > > > we
> > > > > >> >> > > > > > do
> > > > > >> >> > > > > > > > it?
> > > > > >> >> > > > > > > > > >> My
> > > > > >> >> > > > > > > > > >> > > point
> > > > > >> >> > > > > > > > > >> > > > > was it would be no different with
> > > > > delegation
> > > > > >> >> > tokens.
> > > > > >> >> > > > The
> > > > > >> >> > > > > > use
> > > > > >> >> > > > > > > > > case
> > > > > >> >> > > > > > > > > >> you
> > > > > >> >> > > > > > > > > >> > > are
> > > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > > impersonation.*
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the
> things
> > > > that
> > > > > >> >> > delegation
> > > > > >> >> > > > > > tokens
> > > > > >> >> > > > > > > > > >> handled.
> > > > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > Gwen
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > > > > Thanks
> > > > > >> >> > > > > > > > > >> > > > > Parth
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM,
> > Gwen
> > > > > >> Shapira <
> > > > > >> >> > > > > > > > > gwen@confluent.io
> > > > > >> >> > > > > > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >
> > > > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> Few more comments:
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse
> section
> > > > > >> doesn't seem
> > > > > >> >> > to
> > > > > >> >> > > > > have
> > > > > >> >> > > > > > > good
> > > > > >> >> > > > > > > > > >> > > > >> description of the changes to the
> > > Kafka
> > > > > >> Protocol.
> > > > > >> >> > > > > Sounds
> > > > > >> >> > > > > > > like
> > > > > >> >> > > > > > > > > >> you are
> > > > > >> >> > > > > > > > > >> > > > >> proposing new
> > DelegationTokenRequest
> > > > and
> > > > > >> >> > > > > > RenewTokenRequest
> > > > > >> >> > > > > > > > (and
> > > > > >> >> > > > > > > > > >> > > > >> matching responses), without
> > > detailing
> > > > > the
> > > > > >> >> > contents
> > > > > >> >> > > > of
> > > > > >> >> > > > > > the
> > > > > >> >> > > > > > > > > >> requests
> > > > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you
> show
> > > the
> > > > > >> class
> > > > > >> >> > > > interface,
> > > > > >> >> > > > > > but
> > > > > >> >> > > > > > > > not
> > > > > >> >> > > > > > > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> underlying protocol. This could
> be
> > > seen
> > > > > as
> > > > > >> an
> > > > > >> >> > > > > > > implementation
> > > > > >> >> > > > > > > > > >> detail,
> > > > > >> >> > > > > > > > > >> > > > >> but since the binary protocol is
> > what
> > > > we
> > > > > >> provide
> > > > > >> >> > to
> > > > > >> >> > > > > > > non-Java
> > > > > >> >> > > > > > > > > >> clients,
> > > > > >> >> > > > > > > > > >> > > > >> we need to show the changes
> there.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> > > > > >> >> > > > > > > > delegationTokenRequestHandler?
> > > > > >> >> > > > > > > > > >> Is it
> > > > > >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi?
> or
> > > > > Client?
> > > > > >> Its
> > > > > >> >> > > > > unclear...
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even
> > > though
> > > > > >> delegation
> > > > > >> >> > > > > tokens
> > > > > >> >> > > > > > > are
> > > > > >> >> > > > > > > > a
> > > > > >> >> > > > > > > > > >> Hadoop
> > > > > >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly
> > > about
> > > > > not
> > > > > >> adding
> > > > > >> >> > > > > > > dependency
> > > > > >> >> > > > > > > > on
> > > > > >> >> > > > > > > > > >> Hadoop
> > > > > >> >> > > > > > > > > >> > > > >> when implementing delegation
> tokens
> > > for
> > > > > >> Kafka.
> > > > > >> >> > The
> > > > > >> >> > > > KIP
> > > > > >> >> > > > > > > > doesn't
> > > > > >> >> > > > > > > > > >> imply
> > > > > >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> > > > > clarify...
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token at
> > any
> > > > time
> > > > > >> after
> > > > > >> >> > > > > > > > authenticating?
> > > > > >> >> > > > > > > > > >> only
> > > > > >> >> > > > > > > > > >> > > > >> immediately after?
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * My understanding is that tokens
> > > will
> > > > > >> propagate
> > > > > >> >> > > via
> > > > > >> >> > > > ZK
> > > > > >> >> > > > > > but
> > > > > >> >> > > > > > > > > >> without
> > > > > >> >> > > > > > > > > >> > > > >> additional changes to
> > UpdateMetadata
> > > > > >> protocol,
> > > > > >> >> > > > correct?
> > > > > >> >> > > > > > > > Clients
> > > > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL
> auth
> > > > > failure
> > > > > >> >> > (IIRC),
> > > > > >> >> > > > but
> > > > > >> >> > > > > > > since
> > > > > >> >> > > > > > > > > the
> > > > > >> >> > > > > > > > > >> > > > >> tokens propagate between brokers
> > > > asynch,
> > > > > >> we will
> > > > > >> >> > > need
> > > > > >> >> > > > > to
> > > > > >> >> > > > > > > > retry
> > > > > >> >> > > > > > > > > a
> > > > > >> >> > > > > > > > > >> bit
> > > > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due
> > to
> > > > > timing
> > > > > >> >> > issues.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients
> not
> > > > > >> touching ZK
> > > > > >> >> > > > directly
> > > > > >> >> > > > > > :)
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of
> > > having
> > > > > >> just the
> > > > > >> >> > > > > > controller
> > > > > >> >> > > > > > > > > >> issue the
> > > > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid
> > syncing a
> > > > > >> shared
> > > > > >> >> > > secret.
> > > > > >> >> > > > > Not
> > > > > >> >> > > > > > > sure
> > > > > >> >> > > > > > > > > if
> > > > > >> >> > > > > > > > > >> we
> > > > > >> >> > > > > > > > > >> > > > >> want to continue the discussion
> > here
> > > or
> > > > > on
> > > > > >> the
> > > > > >> >> > > wiki.
> > > > > >> >> > > > I
> > > > > >> >> > > > > > > think
> > > > > >> >> > > > > > > > > >> that we
> > > > > >> >> > > > > > > > > >> > > > >> can decouple the problem of
> "token
> > > > > >> distribution"
> > > > > >> >> > > from
> > > > > >> >> > > > > > > "shared
> > > > > >> >> > > > > > > > > >> secret
> > > > > >> >> > > > > > > > > >> > > > >> distribution" and use the
> > controller
> > > as
> > > > > >> the only
> > > > > >> >> > > > token
> > > > > >> >> > > > > > > > > generator
> > > > > >> >> > > > > > > > > >> to
> > > > > >> >> > > > > > > > > >> > > > >> solve the second issue, while
> still
> > > > using
> > > > > >> ZK
> > > > > >> >> > async
> > > > > >> >> > > to
> > > > > >> >> > > > > > > > > distribute
> > > > > >> >> > > > > > > > > >> > > > >> tokens.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with
> > > infinite
> > > > > >> lifetime
> > > > > >> >> > of
> > > > > >> >> > > > > > tokens
> > > > > >> >> > > > > > > > (and
> > > > > >> >> > > > > > > > > >> hoped
> > > > > >> >> > > > > > > > > >> > > > >> to hear from others in the
> > > community) -
> > > > > but
> > > > > >> >> > having
> > > > > >> >> > > > the
> > > > > >> >> > > > > > > option
> > > > > >> >> > > > > > > > > and
> > > > > >> >> > > > > > > > > >> > > > >> documenting it as less secure,
> > allows
> > > > > >> users to
> > > > > >> >> > > > > configure
> > > > > >> >> > > > > > > > their
> > > > > >> >> > > > > > > > > >> system
> > > > > >> >> > > > > > > > > >> > > > >> as the wish.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * While I like the idea of
> forcing
> > > > > >> kerberos auth
> > > > > >> >> > > for
> > > > > >> >> > > > > > > renwal,
> > > > > >> >> > > > > > > > I
> > > > > >> >> > > > > > > > > >> think
> > > > > >> >> > > > > > > > > >> > > > >> it mixes the transport layer the
> > the
> > > > > >> request
> > > > > >> >> > > content
> > > > > >> >> > > > > in a
> > > > > >> >> > > > > > > > > pretty
> > > > > >> >> > > > > > > > > >> ugly
> > > > > >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> > > > > non-owner
> > > > > >> is
> > > > > >> >> > > better.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * More detailed explanation on
> what
> > > we
> > > > > >> plan to do
> > > > > >> >> > > for
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > > > java
> > > > > >> >> > > > > > > > > >> clients
> > > > > >> >> > > > > > > > > >> > > > >> specifically - new configuration?
> > new
> > > > > APIs?
> > > > > >> >> > > > > > > > > >> > > > >> The response for my question on
> how
> > > > > >> multiple
> > > > > >> >> > > > identities
> > > > > >> >> > > > > > > will
> > > > > >> >> > > > > > > > be
> > > > > >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me
> -
> > > > AFAIK,
> > > > > >> we
> > > > > >> >> > don't
> > > > > >> >> > > > > > > > authenticate
> > > > > >> >> > > > > > > > > >> each
> > > > > >> >> > > > > > > > > >> > > > >> request, we authenticate
> > connections.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens
> > are
> > > > > only
> > > > > >> used
> > > > > >> >> > in
> > > > > >> >> > > > the
> > > > > >> >> > > > > > > Hadoop
> > > > > >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there
> > are
> > > > > >> >> > alternatives
> > > > > >> >> > > in
> > > > > >> >> > > > > > other
> > > > > >> >> > > > > > > > > >> ecosystems
> > > > > >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and
> > > > whether
> > > > > >> there
> > > > > >> >> > are
> > > > > >> >> > > > some
> > > > > >> >> > > > > > > > > >> advantages
> > > > > >> >> > > > > > > > > >> > > > >> there.
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> Gwen
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM,
> > > > Harsha <
> > > > > >> >> > > > > kafka@harsha.io
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > > > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> > > > > >> >> > > > > > > > > >> > > > >> > Can you look at
> > Parth's
> > > > last
> > > > > >> reply.
> > > > > >> >> > > Does
> > > > > >> >> > > > > it
> > > > > >> >> > > > > > > > answer
> > > > > >> >> > > > > > > > > >> your
> > > > > >> >> > > > > > > > > >> > > > >> > concerns.
> > > > > >> >> > > > > > > > > >> > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> > Thanks,
> > > > > >> >> > > > > > > > > >> > > > >> > Harsha
> > > > > >> >> > > > > > > > > >> > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25
> AM,
> > > > parth
> > > > > >> >> > > brahmbhatt
> > > > > >> >> > > > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The
> > > wiki
> > > > > >> already
> > > > > >> >> > has
> > > > > >> >> > > > > > details
> > > > > >> >> > > > > > > on
> > > > > >> >> > > > > > > > > >> token
> > > > > >> >> > > > > > > > > >> > > > >> >> expiration
> > > > > >> >> > > > > > > > > >> > > > >> >> under token acquisition
> process
> > > > > >> >> > > > > > > > > >> > > > >> >> <
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> > > > > >> >> > > > > > > > > >> > > > >> >.
> > > > > >> >> > > > > > > > > >> > > > >> >> Current proposal is that
> tokens
> > > will
> > > > > >> expire
> > > > > >> >> > > based
> > > > > >> >> > > > > on a
> > > > > >> >> > > > > > > > > server
> > > > > >> >> > > > > > > > > >> side
> > > > > >> >> > > > > > > > > >> > > > >> >> configuration (default 24
> hours)
> > > > > unless
> > > > > >> >> > renewed.
> > > > > >> >> > > > > > Renewal
> > > > > >> >> > > > > > > > is
> > > > > >> >> > > > > > > > > >> only
> > > > > >> >> > > > > > > > > >> > > > allowed
> > > > > >> >> > > > > > > > > >> > > > >> >> until the max life time of
> > token.
> > > > > >> >> > Alternatively
> > > > > >> >> > > we
> > > > > >> >> > > > > > could
> > > > > >> >> > > > > > > > > also
> > > > > >> >> > > > > > > > > >> make
> > > > > >> >> > > > > > > > > >> > > > that
> > > > > >> >> > > > > > > > > >> > > > >> >> an
> > > > > >> >> > > > > > > > > >> > > > >> >> optional param and the server
> > side
> > > > > >> default can
> > > > > >> >> > > > serve
> > > > > >> >> > > > > > as
> > > > > >> >> > > > > > > > the
> > > > > >> >> > > > > > > > > >> upper
> > > > > >> >> > > > > > > > > >> > > > bound.
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> To your second point it will
> be
> > > done
> > > > > >> exactly
> > > > > >> >> > the
> > > > > >> >> > > > > same
> > > > > >> >> > > > > > > way
> > > > > >> >> > > > > > > > we
> > > > > >> >> > > > > > > > > >> would
> > > > > >> >> > > > > > > > > >> > > > >> >> support
> > > > > >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling
> > > client
> > > > > >> will have
> > > > > >> >> > > to
> > > > > >> >> > > > > put
> > > > > >> >> > > > > > > the
> > > > > >> >> > > > > > > > > >> tokens it
> > > > > >> >> > > > > > > > > >> > > > >> wants
> > > > > >> >> > > > > > > > > >> > > > >> >> to use in the subject instance
> > and
> > > > > call
> > > > > >> >> > > > > > produce/consume
> > > > > >> >> > > > > > > > > inside
> > > > > >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will
> > > have
> > > > to
> > > > > >> keep
> > > > > >> >> > > track
> > > > > >> >> > > > of
> > > > > >> >> > > > > > its
> > > > > >> >> > > > > > > > own
> > > > > >> >> > > > > > > > > >> > > > subject. I
> > > > > >> >> > > > > > > > > >> > > > >> >> will have to look at the code
> to
> > > see
> > > > > if
> > > > > >> we
> > > > > >> >> > > support
> > > > > >> >> > > > > > this
> > > > > >> >> > > > > > > > > >> feature
> > > > > >> >> > > > > > > > > >> > > right
> > > > > >> >> > > > > > > > > >> > > > >> now
> > > > > >> >> > > > > > > > > >> > > > >> >> but my understanding is
> > delegation
> > > > > token
> > > > > >> >> > > shouldn't
> > > > > >> >> > > > > > need
> > > > > >> >> > > > > > > > any
> > > > > >> >> > > > > > > > > >> special
> > > > > >> >> > > > > > > > > >> > > > >> >> treatment as its just another
> > type
> > > > of
> > > > > >> >> > Credential
> > > > > >> >> > > > in
> > > > > >> >> > > > > > the
> > > > > >> >> > > > > > > > > >> subject.
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> I would also like to know what
> > is
> > > > your
> > > > > >> opinion
> > > > > >> >> > > > about
> > > > > >> >> > > > > > > > > infinite
> > > > > >> >> > > > > > > > > >> > > renewal
> > > > > >> >> > > > > > > > > >> > > > >> (my
> > > > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not
> support
> > > > > this),
> > > > > >> tokens
> > > > > >> >> > > > > > renewing
> > > > > >> >> > > > > > > > them
> > > > > >> >> > > > > > > > > >> > > self(my
> > > > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not
> support
> > > > this)
> > > > > >> and
> > > > > >> >> > most
> > > > > >> >> > > > > > > > importantly
> > > > > >> >> > > > > > > > > >> your
> > > > > >> >> > > > > > > > > >> > > > choice
> > > > > >> >> > > > > > > > > >> > > > >> >> between the alternatives
> listed
> > on
> > > > > this
> > > > > >> thread
> > > > > >> >> > > > > > > > > >> > > > >> >> <
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> > > > > >> >> > > > > > > > > >> > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> > > > > >> alternative-2 minus
> > > > > >> >> > > > > > > controller
> > > > > >> >> > > > > > > > > >> > > > distributing
> > > > > >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for
> > > reviewing.
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> Thanks
> > > > > >> >> > > > > > > > > >> > > > >> >> Parth
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17
> AM,
> > > Gwen
> > > > > >> Shapira <
> > > > > >> >> > > > > > > > > >> gwen@confluent.io>
> > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > Harsha,
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest
> > > Proxy.
> > > > I
> > > > > >> didn't
> > > > > >> >> > see
> > > > > >> >> > > > > your
> > > > > >> >> > > > > > > > design
> > > > > >> >> > > > > > > > > >> yet,
> > > > > >> >> > > > > > > > > >> > > > but in
> > > > > >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> > > > > >> producers, which
> > > > > >> >> > > > will
> > > > > >> >> > > > > > > serve
> > > > > >> >> > > > > > > > > >> multiple
> > > > > >> >> > > > > > > > > >> > > > users
> > > > > >> >> > > > > > > > > >> > > > >> >> > going through the proxy.
> Since
> > > > these
> > > > > >> users
> > > > > >> >> > > will
> > > > > >> >> > > > > have
> > > > > >> >> > > > > > > > > >> different
> > > > > >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> > > > > >> authenticate
> > > > > >> >> > > > > separately,
> > > > > >> >> > > > > > > and
> > > > > >> >> > > > > > > > > >> can't
> > > > > >> >> > > > > > > > > >> > > > share a
> > > > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > Gwen
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11
> > PM,
> > > > > >> Harsha <
> > > > > >> >> > > > > > > kafka@harsha.io
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> > > > > >> >> > > > > > > > > >> > > > >> >> > > On your second
> > > point.
> > > > > >> Can you
> > > > > >> >> > > > > describe
> > > > > >> >> > > > > > a
> > > > > >> >> > > > > > > > > >> usecase
> > > > > >> >> > > > > > > > > >> > > where
> > > > > >> >> > > > > > > > > >> > > > >> >> > > mutliple
> clients
> > > > ended
> > > > > up
> > > > > >> >> > > sharing a
> > > > > >> >> > > > > > > > producer
> > > > > >> >> > > > > > > > > >> and
> > > > > >> >> > > > > > > > > >> > > even
> > > > > >> >> > > > > > > > > >> > > > if
> > > > > >> >> > > > > > > > > >> > > > >> they
> > > > > >> >> > > > > > > > > >> > > > >> >> > > do why can't
> they
> > > not
> > > > > use
> > > > > >> >> > single
> > > > > >> >> > > > > token
> > > > > >> >> > > > > > > that
> > > > > >> >> > > > > > > > > >> producer
> > > > > >> >> > > > > > > > > >> > > > >> >> > > captures. Why
> > would
> > > > we
> > > > > >> need
> > > > > >> >> > > > multiple
> > > > > >> >> > > > > > > > clients
> > > > > >> >> > > > > > > > > >> with
> > > > > >> >> > > > > > > > > >> > > > >> different
> > > > > >> >> > > > > > > > > >> > > > >> >> > > tokens sharing
> a
> > > > single
> > > > > >> >> > instance
> > > > > >> >> > > of
> > > > > >> >> > > > > > > > producer.
> > > > > >> >> > > > > > > > > >> Also
> > > > > >> >> > > > > > > > > >> > > in
> > > > > >> >> > > > > > > > > >> > > > >> this
> > > > > >> >> > > > > > > > > >> > > > >> >> > > case other
> > clients
> > > > have
> > > > > >> access
> > > > > >> >> > > all
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > > > tokens
> > > > > >> >> > > > > > > > > >> no?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> > > > > >> >> > > > > > > > > >> > > > >> >> > > Harsha
> > > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at
> > 11:49
> > > > AM,
> > > > > >> Gwen
> > > > > >> >> > > Shapira
> > > > > >> >> > > > > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we
> > didn't
> > > > see
> > > > > >> in the
> > > > > >> >> > > wiki:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration
> > for
> > > > > >> delegation
> > > > > >> >> > > > > tokens?
> > > > > >> >> > > > > > > > > >> Renewal? How
> > > > > >> >> > > > > > > > > >> > > > do we
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use
> > > delegation
> > > > > >> tokens
> > > > > >> >> > for
> > > > > >> >> > > > > > "do-as"
> > > > > >> >> > > > > > > > > (say,
> > > > > >> >> > > > > > > > > >> > > submit
> > > > > >> >> > > > > > > > > >> > > > >> Storm
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will
> > > need a
> > > > > >> producer
> > > > > >> >> > > for
> > > > > >> >> > > > > > every
> > > > > >> >> > > > > > > > job
> > > > > >> >> > > > > > > > > >> (we
> > > > > >> >> > > > > > > > > >> > > can't
> > > > > >> >> > > > > > > > > >> > > > >> share
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> them between multiple
> jobs
> > > > > running
> > > > > >> on
> > > > > >> >> > same
> > > > > >> >> > > > > node),
> > > > > >> >> > > > > > > > since
> > > > > >> >> > > > > > > > > >> we
> > > > > >> >> > > > > > > > > >> > > only
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> authenticate when
> > connecting.
> > > > Is
> > > > > >> there a
> > > > > >> >> > > plan
> > > > > >> >> > > > > to
> > > > > >> >> > > > > > > > change
> > > > > >> >> > > > > > > > > >> this
> > > > > >> >> > > > > > > > > >> > > for
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in
> order
> > > to
> > > > > >> allow
> > > > > >> >> > > multiple
> > > > > >> >> > > > > > users
> > > > > >> >> > > > > > > > > with
> > > > > >> >> > > > > > > > > >> > > > different
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> > > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at
> 9:12
> > > AM,
> > > > > >> parth
> > > > > >> >> > > > > brahmbhatt
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> <
> > brahmbhatt.parth@gmail.com>
> > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one
> more
> > > > time,
> > > > > >> can
> > > > > >> >> > other
> > > > > >> >> > > > > > > committers
> > > > > >> >> > > > > > > > > >> review?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at
> > > 9:07
> > > > > AM,
> > > > > >> >> > Harsha <
> > > > > >> >> > > > > > > > > >> kafka@harsha.io>
> > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Overall
> > current
> > > > > >> design looks
> > > > > >> >> > > > good
> > > > > >> >> > > > > to
> > > > > >> >> > > > > > > > me. I
> > > > > >> >> > > > > > > > > >> am +1
> > > > > >> >> > > > > > > > > >> > > on
> > > > > >> >> > > > > > > > > >> > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > KIP.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you
> > review
> > > > this
> > > > > >> as
> > > > > >> >> > well.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016,
> at
> > > > 09:57
> > > > > >> AM,
> > > > > >> >> > parth
> > > > > >> >> > > > > > > > brahmbhatt
> > > > > >> >> > > > > > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review
> > > > Jitendra.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the
> idea
> > of
> > > > > >> infinite
> > > > > >> >> > > > lifetime
> > > > > >> >> > > > > > > but I
> > > > > >> >> > > > > > > > > >> see the
> > > > > >> >> > > > > > > > > >> > > > >> Streaming
> > > > > >> >> > > > > > > > > >> > > > >> >> > use
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for
> > Streaming
> > > > use
> > > > > >> case I
> > > > > >> >> > > was
> > > > > >> >> > > > > > hoping
> > > > > >> >> > > > > > > > > >> there will
> > > > > >> >> > > > > > > > > >> > > > be
> > > > > >> >> > > > > > > > > >> > > > >> some
> > > > > >> >> > > > > > > > > >> > > > >> >> > notion
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that
> can
> > > get
> > > > > new
> > > > > >> >> > > > delegation
> > > > > >> >> > > > > > > tokens
> > > > > >> >> > > > > > > > > at
> > > > > >> >> > > > > > > > > >> fixed
> > > > > >> >> > > > > > > > > >> > > > >> interval
> > > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to
> workers.
> > > If
> > > > > >> that is
> > > > > >> >> > not
> > > > > >> >> > > > the
> > > > > >> >> > > > > > case
> > > > > >> >> > > > > > > > for
> > > > > >> >> > > > > > > > > >> we can
> > > > > >> >> > > > > > > > > >> > > > >> discuss
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens
> > > renewing
> > > > > >> them self
> > > > > >> >> > > and
> > > > > >> >> > > > > the
> > > > > >> >> > > > > > > > > >> security
> > > > > >> >> > > > > > > > > >> > > > >> implications
> > > > > >> >> > > > > > > > > >> > > > >> >> > of the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want
> clients
> > > to
> > > > > >> fetch
> > > > > >> >> > tokens
> > > > > >> >> > > > > from
> > > > > >> >> > > > > > > > > >> zookeeper,
> > > > > >> >> > > > > > > > > >> > > > >> overall I
> > > > > >> >> > > > > > > > > >> > > > >> >> > think
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients
> > don't
> > > > rely
> > > > > >> on our
> > > > > >> >> > > > > metadata
> > > > > >> >> > > > > > > > store
> > > > > >> >> > > > > > > > > >> and I
> > > > > >> >> > > > > > > > > >> > > > >> think we
> > > > > >> >> > > > > > > > > >> > > > >> >> > are
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that
> > direction
> > > > > with
> > > > > >> all
> > > > > >> >> > the
> > > > > >> >> > > > > KIP-4
> > > > > >> >> > > > > > > > > >> improvements.
> > > > > >> >> > > > > > > > > >> > > > I
> > > > > >> >> > > > > > > > > >> > > > >> chose
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this
> > > case
> > > > > the
> > > > > >> client
> > > > > >> >> > > > will
> > > > > >> >> > > > > > > still
> > > > > >> >> > > > > > > > > >> just talk
> > > > > >> >> > > > > > > > > >> > > > to
> > > > > >> >> > > > > > > > > >> > > > >> >> > broker , its
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that
> will
> > > use
> > > > > >> zookeeper
> > > > > >> >> > > > which
> > > > > >> >> > > > > we
> > > > > >> >> > > > > > > > > >> already do
> > > > > >> >> > > > > > > > > >> > > > for a
> > > > > >> >> > > > > > > > > >> > > > >> lot
> > > > > >> >> > > > > > > > > >> > > > >> >> > of
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> > > > > >> development + and
> > > > > >> >> > > the
> > > > > >> >> > > > > > > ability
> > > > > >> >> > > > > > > > > so
> > > > > >> >> > > > > > > > > >> > > tokens
> > > > > >> >> > > > > > > > > >> > > > >> will
> > > > > >> >> > > > > > > > > >> > > > >> >> > survive
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> > > > > restart/cluster
> > > > > >> >> > > failure.
> > > > > >> >> > > > > if a
> > > > > >> >> > > > > > > > > >> majority
> > > > > >> >> > > > > > > > > >> > > > agrees
> > > > > >> >> > > > > > > > > >> > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > added
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> > > > > controller
> > > > > >> >> > > forwarding
> > > > > >> >> > > > > > keys
> > > > > >> >> > > > > > > to
> > > > > >> >> > > > > > > > > all
> > > > > >> >> > > > > > > > > >> > > > broker is
> > > > > >> >> > > > > > > > > >> > > > >> >> > justified
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter
> > > > security
> > > > > >> , I am
> > > > > >> >> > > fine
> > > > > >> >> > > > > > with
> > > > > >> >> > > > > > > > that
> > > > > >> >> > > > > > > > > >> option
> > > > > >> >> > > > > > > > > >> > > > too.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does
> > not
> > > > > >> support SSL
> > > > > >> >> > > we
> > > > > >> >> > > > > can
> > > > > >> >> > > > > > > not
> > > > > >> >> > > > > > > > > >> store
> > > > > >> >> > > > > > > > > >> > > > master
> > > > > >> >> > > > > > > > > >> > > > >> keys
> > > > > >> >> > > > > > > > > >> > > > >> >> > in
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master
> > keys
> > > > > will
> > > > > >> be
> > > > > >> >> > > exposed
> > > > > >> >> > > > > on
> > > > > >> >> > > > > > > > wire.
> > > > > >> >> > > > > > > > > To
> > > > > >> >> > > > > > > > > >> > > > support
> > > > > >> >> > > > > > > > > >> > > > >> >> > rotation
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting
> > > current
> > > > > >> clients is
> > > > > >> >> > > > > > > something I
> > > > > >> >> > > > > > > > > >> need to
> > > > > >> >> > > > > > > > > >> > > > put
> > > > > >> >> > > > > > > > > >> > > > >> more
> > > > > >> >> > > > > > > > > >> > > > >> >> > thought
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current
> > proposal
> > > > > >> assumes the
> > > > > >> >> > > > > rotation
> > > > > >> >> > > > > > > > will
> > > > > >> >> > > > > > > > > >> > > > invalidate
> > > > > >> >> > > > > > > > > >> > > > >> all
> > > > > >> >> > > > > > > > > >> > > > >> >> > current
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers
> > to
> > > > also
> > > > > >> review
> > > > > >> >> > > and
> > > > > >> >> > > > > post
> > > > > >> >> > > > > > > > their
> > > > > >> >> > > > > > > > > >> > > comments
> > > > > >> >> > > > > > > > > >> > > > >> so we
> > > > > >> >> > > > > > > > > >> > > > >> >> > can
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this
> KIP.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016
> > at
> > > > 8:39
> > > > > >> AM,
> > > > > >> >> > > Ashish
> > > > > >> >> > > > > > Singh
> > > > > >> >> > > > > > > <
> > > > > >> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> > > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18,
> 2016
> > > at
> > > > > >> 11:26 AM,
> > > > > >> >> > > > > Harsha <
> > > > > >> >> > > > > > > > > >> > > > kafka@harsha.io>
> > > > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> > > > > >> discussion
> > > > > >> >> > > threads
> > > > > >> >> > > > on
> > > > > >> >> > > > > > > this
> > > > > >> >> > > > > > > > > KIP.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the
> > response
> > > > > from
> > > > > >> >> > Jitendra
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a
> > > large
> > > > > >> number of
> > > > > >> >> > > > > clients
> > > > > >> >> > > > > > > that
> > > > > >> >> > > > > > > > > are
> > > > > >> >> > > > > > > > > >> > > > running
> > > > > >> >> > > > > > > > > >> > > > >> all
> > > > > >> >> > > > > > > > > >> > > > >> >> > over the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> > > > > authenticate
> > > > > >> with
> > > > > >> >> > > Kafka
> > > > > >> >> > > > > > > > brokers,
> > > > > >> >> > > > > > > > > >> is very
> > > > > >> >> > > > > > > > > >> > > > >> similar
> > > > > >> >> > > > > > > > > >> > > > >> >> > to the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case
> of
> > > > large
> > > > > >> number
> > > > > >> >> > of
> > > > > >> >> > > > > tasks
> > > > > >> >> > > > > > > > > running
> > > > > >> >> > > > > > > > > >> > > across
> > > > > >> >> > > > > > > > > >> > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > cluster
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> that
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need
> > authentication
> > > to
> > > > > >> Hdfs
> > > > > >> >> > > > Namenode.
> > > > > >> >> > > > > > > > > >> Therefore, the
> > > > > >> >> > > > > > > > > >> > > > >> >> > delegation token
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does
> seem
> > > > like
> > > > > a
> > > > > >> good
> > > > > >> >> > fit
> > > > > >> >> > > > for
> > > > > >> >> > > > > > > this
> > > > > >> >> > > > > > > > > use
> > > > > >> >> > > > > > > > > >> case
> > > > > >> >> > > > > > > > > >> > > > as we
> > > > > >> >> > > > > > > > > >> > > > >> >> > have seen
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> it
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large
> > > scale
> > > > > in
> > > > > >> HDFS
> > > > > >> >> > and
> > > > > >> >> > > > > YARN.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed
> > > design
> > > > is
> > > > > >> very
> > > > > >> >> > much
> > > > > >> >> > > > > > inline
> > > > > >> >> > > > > > > > with
> > > > > >> >> > > > > > > > > >> Hadoop
> > > > > >> >> > > > > > > > > >> > > > >> >> > approach. A few
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you
> guys
> > > > want
> > > > > >> to allow
> > > > > >> >> > > > > > infinite
> > > > > >> >> > > > > > > > > >> renewable
> > > > > >> >> > > > > > > > > >> > > > >> lifetime
> > > > > >> >> > > > > > > > > >> > > > >> >> > for a
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS
> > > restricts
> > > > a
> > > > > >> token
> > > > > >> >> > to a
> > > > > >> >> > > > max
> > > > > >> >> > > > > > > life
> > > > > >> >> > > > > > > > > time
> > > > > >> >> > > > > > > > > >> > > > (default
> > > > > >> >> > > > > > > > > >> > > > >> 7
> > > > > >> >> > > > > > > > > >> > > > >> >> > days). A
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's
> > > vulnerability
> > > > is
> > > > > >> >> > believed
> > > > > >> >> > > to
> > > > > >> >> > > > > > > > increase
> > > > > >> >> > > > > > > > > >> with
> > > > > >> >> > > > > > > > > >> > > > time.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that
> having
> > > > > infinite
> > > > > >> >> > > lifetime
> > > > > >> >> > > > > > might
> > > > > >> >> > > > > > > > not
> > > > > >> >> > > > > > > > > >> be the
> > > > > >> >> > > > > > > > > >> > > > best
> > > > > >> >> > > > > > > > > >> > > > >> idea.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I
> understand
> > > the
> > > > > >> tokens
> > > > > >> >> > are
> > > > > >> >> > > > > stored
> > > > > >> >> > > > > > > in
> > > > > >> >> > > > > > > > > >> zookeeper
> > > > > >> >> > > > > > > > > >> > > > as
> > > > > >> >> > > > > > > > > >> > > > >> well,
> > > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> can
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated
> there.
> > > This
> > > > > is
> > > > > >> clever
> > > > > >> >> > > as
> > > > > >> >> > > > it
> > > > > >> >> > > > > > can
> > > > > >> >> > > > > > > > > allow
> > > > > >> >> > > > > > > > > >> > > > >> replacing the
> > > > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run
> out
> > of
> > > > max
> > > > > >> life
> > > > > >> >> > > time,
> > > > > >> >> > > > > and
> > > > > >> >> > > > > > > > > clients
> > > > > >> >> > > > > > > > > >> can
> > > > > >> >> > > > > > > > > >> > > > >> download
> > > > > >> >> > > > > > > > > >> > > > >> >> > new
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper.
> It
> > > > > >> shouldn't be
> > > > > >> >> > a
> > > > > >> >> > > > big
> > > > > >> >> > > > > > load
> > > > > >> >> > > > > > > > on
> > > > > >> >> > > > > > > > > >> > > zookeeper
> > > > > >> >> > > > > > > > > >> > > > >> as a
> > > > > >> >> > > > > > > > > >> > > > >> >> > client
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a
> new
> > > > token
> > > > > >> once in
> > > > > >> >> > > > > several
> > > > > >> >> > > > > > > > days.
> > > > > >> >> > > > > > > > > >> In this
> > > > > >> >> > > > > > > > > >> > > > >> approach
> > > > > >> >> > > > > > > > > >> > > > >> >> > you
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite
> > > lifetime
> > > > > on
> > > > > >> the
> > > > > >> >> > > token
> > > > > >> >> > > > > even
> > > > > >> >> > > > > > > for
> > > > > >> >> > > > > > > > > >> long
> > > > > >> >> > > > > > > > > >> > > > running
> > > > > >> >> > > > > > > > > >> > > > >> >> > clients.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token
> > > password
> > > > > are
> > > > > >> >> > > generated
> > > > > >> >> > > > > > using
> > > > > >> >> > > > > > > a
> > > > > >> >> > > > > > > > > >> master
> > > > > >> >> > > > > > > > > >> > > key.
> > > > > >> >> > > > > > > > > >> > > > >> The
> > > > > >> >> > > > > > > > > >> > > > >> >> > master
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> key
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> > > > > >> periodically
> > > > > >> >> > > changed.
> > > > > >> >> > > > > In
> > > > > >> >> > > > > > > > > Hadoop,
> > > > > >> >> > > > > > > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> default
> > > > > >> >> > > > > > > > > >> > > > >> >> > renewal
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1
> day.?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will
> > > require
> > > > > >> brokers
> > > > > >> >> > > > > > maintaining
> > > > > >> >> > > > > > > a
> > > > > >> >> > > > > > > > > >> list of X
> > > > > >> >> > > > > > > > > >> > > > most
> > > > > >> >> > > > > > > > > >> > > > >> >> > recent
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> master
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list
> will
> > > > have
> > > > > >> to be
> > > > > >> >> > > > > persisted
> > > > > >> >> > > > > > > > > >> somewhere, as
> > > > > >> >> > > > > > > > > >> > > > if a
> > > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have
> to
> > > get
> > > > > >> that list
> > > > > >> >> > > > again
> > > > > >> >> > > > > > and
> > > > > >> >> > > > > > > > > >> storing
> > > > > >> >> > > > > > > > > >> > > > master
> > > > > >> >> > > > > > > > > >> > > > >> keys
> > > > > >> >> > > > > > > > > >> > > > >> >> > on ZK
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> is
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> > > > > However,
> > > > > >> if a
> > > > > >> >> > > > broker
> > > > > >> >> > > > > > goes
> > > > > >> >> > > > > > > > > down
> > > > > >> >> > > > > > > > > >> then
> > > > > >> >> > > > > > > > > >> > > we
> > > > > >> >> > > > > > > > > >> > > > >> have
> > > > > >> >> > > > > > > > > >> > > > >> >> > much
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with
> > and
> > > > > >> client can
> > > > > >> >> > > > always
> > > > > >> >> > > > > > > > > >> > > re-authenticate
> > > > > >> >> > > > > > > > > >> > > > is
> > > > > >> >> > > > > > > > > >> > > > >> such
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to
> > > take a
> > > > > >> look at
> > > > > >> >> > > other
> > > > > >> >> > > > > > > > > >> alternatives
> > > > > >> >> > > > > > > > > >> > > this
> > > > > >> >> > > > > > > > > >> > > > >> list has
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a
> > > thorough
> > > > > >> proposal,
> > > > > >> >> > > > great
> > > > > >> >> > > > > > > work!"
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7,
> > 2016,
> > > > at
> > > > > >> 10:28
> > > > > >> >> > PM,
> > > > > >> >> > > > Gwen
> > > > > >> >> > > > > > > > Shapira
> > > > > >> >> > > > > > > > > >> wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to
> > me.
> > > > > >> Thanks!
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7,
> > > 2016
> > > > at
> > > > > >> 9:25
> > > > > >> >> > PM,
> > > > > >> >> > > > > > Harsha <
> > > > > >> >> > > > > > > > > >> > > > kafka@harsha.io
> > > > > >> >> > > > > > > > > >> > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't
> > need
> > > > any
> > > > > >> release
> > > > > >> >> > > > > vehicle
> > > > > >> >> > > > > > > but
> > > > > >> >> > > > > > > > > >> still the
> > > > > >> >> > > > > > > > > >> > > > >> work can
> > > > > >> >> > > > > > > > > >> > > > >> >> > move
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> > > > > >> interested in
> > > > > >> >> > the
> > > > > >> >> > > > KIP
> > > > > >> >> > > > > > > > please
> > > > > >> >> > > > > > > > > >> do the
> > > > > >> >> > > > > > > > > >> > > > >> review and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar
> 7,
> > > > 2016,
> > > > > >> at
> > > > > >> >> > 04:59
> > > > > >> >> > > > PM,
> > > > > >> >> > > > > > > Ismael
> > > > > >> >> > > > > > > > > >> Juma
> > > > > >> >> > > > > > > > > >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree
> that
> > it
> > > > > >> would be
> > > > > >> >> > good
> > > > > >> >> > > > to
> > > > > >> >> > > > > > have
> > > > > >> >> > > > > > > > > more
> > > > > >> >> > > > > > > > > >> time
> > > > > >> >> > > > > > > > > >> > > to
> > > > > >> >> > > > > > > > > >> > > > >> review
> > > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar
> > 8,
> > > > 2016
> > > > > >> at
> > > > > >> >> > 12:55
> > > > > >> >> > > > AM,
> > > > > >> >> > > > > > Gwen
> > > > > >> >> > > > > > > > > >> Shapira <
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since
> > KIP-48
> > > > > >> depends on
> > > > > >> >> > > > KIP-43,
> > > > > >> >> > > > > > > which
> > > > > >> >> > > > > > > > > is
> > > > > >> >> > > > > > > > > >> > > > already a
> > > > > >> >> > > > > > > > > >> > > > >> bit
> > > > > >> >> > > > > > > > > >> > > > >> >> > of a
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next
> > > > release
> > > > > -
> > > > > >> any
> > > > > >> >> > > chance
> > > > > >> >> > > > > we
> > > > > >> >> > > > > > > can
> > > > > >> >> > > > > > > > > >> delay
> > > > > >> >> > > > > > > > > >> > > > >> delegation
> > > > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> > > > > community
> > > > > >> >> > working
> > > > > >> >> > > > on a
> > > > > >> >> > > > > > > > release
> > > > > >> >> > > > > > > > > >> every
> > > > > >> >> > > > > > > > > >> > > 3
> > > > > >> >> > > > > > > > > >> > > > >> month,
> > > > > >> >> > > > > > > > > >> > > > >> >> > this
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri,
> Feb
> > > 26,
> > > > > >> 2016 at
> > > > > >> >> > > 5:11
> > > > > >> >> > > > > PM,
> > > > > >> >> > > > > > > > Ashish
> > > > > >> >> > > > > > > > > >> Singh
> > > > > >> >> > > > > > > > > >> > > <
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> asingh@cloudera.com
> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks
> > > again
> > > > > for
> > > > > >> the
> > > > > >> >> > > > awesome
> > > > > >> >> > > > > > > write
> > > > > >> >> > > > > > > > > up.
> > > > > >> >> > > > > > > > > >> > > > Following
> > > > > >> >> > > > > > > > > >> > > > >> our
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I
> > > think
> > > > > it
> > > > > >> will
> > > > > >> >> > be
> > > > > >> >> > > > > easier
> > > > > >> >> > > > > > > to
> > > > > >> >> > > > > > > > > >> compare
> > > > > >> >> > > > > > > > > >> > > > >> various
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> > > > > together.
> > > > > >> I am
> > > > > >> >> > > > stating
> > > > > >> >> > > > > > > > below a
> > > > > >> >> > > > > > > > > >> few
> > > > > >> >> > > > > > > > > >> > > > >> >> > alternatives along
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> > > > > proposal.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> (Current
> > > > > >> proposal)
> > > > > >> >> > Store
> > > > > >> >> > > > > > > Delegation
> > > > > >> >> > > > > > > > > >> Token,
> > > > > >> >> > > > > > > > > >> > > DT,
> > > > > >> >> > > > > > > > > >> > > > >> on ZK.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > > Client
> > > > > >> >> > > authenticates
> > > > > >> >> > > > > > with a
> > > > > >> >> > > > > > > > > >> broker.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> > Once
> > > a
> > > > > >> client is
> > > > > >> >> > > > > > > > authenticated,
> > > > > >> >> > > > > > > > > >> it
> > > > > >> >> > > > > > > > > >> > > will
> > > > > >> >> > > > > > > > > >> > > > >> make a
> > > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> issue
> > a
> > > > > >> delegation
> > > > > >> >> > > > token.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> The
> > > > > broker
> > > > > >> >> > > generates
> > > > > >> >> > > > a
> > > > > >> >> > > > > > > shared
> > > > > >> >> > > > > > > > > >> secret
> > > > > >> >> > > > > > > > > >> > > > based
> > > > > >> >> > > > > > > > > >> > > > >> on
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> Password/Secret
> > > > > >> >> > shared
> > > > > >> >> > > > > > between
> > > > > >> >> > > > > > > > all
> > > > > >> >> > > > > > > > > >> > > brokers,
> > > > > >> >> > > > > > > > > >> > > > >> >> > randomly
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > > Broker
> > > > > >> stores
> > > > > >> >> > this
> > > > > >> >> > > > > token
> > > > > >> >> > > > > > in
> > > > > >> >> > > > > > > > its
> > > > > >> >> > > > > > > > > >> in
> > > > > >> >> > > > > > > > > >> > > > memory
> > > > > >> >> > > > > > > > > >> > > > >> cache.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > > > >> DelegationToken
> > > > > >> >> > > > > without
> > > > > >> >> > > > > > > the
> > > > > >> >> > > > > > > > > >> hmac in
> > > > > >> >> > > > > > > > > >> > > the
> > > > > >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5.
> All
> > > > > >> brokers will
> > > > > >> >> > > > have a
> > > > > >> >> > > > > > > cache
> > > > > >> >> > > > > > > > > >> backed
> > > > > >> >> > > > > > > > > >> > > by
> > > > > >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> they
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get
> > > > notified
> > > > > >> >> > whenever
> > > > > >> >> > > a
> > > > > >> >> > > > > new
> > > > > >> >> > > > > > > > token
> > > > > >> >> > > > > > > > > is
> > > > > >> >> > > > > > > > > >> > > > >> generated and
> > > > > >> >> > > > > > > > > >> > > > >> >> > they
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> local
> > > > cache
> > > > > >> whenever
> > > > > >> >> > > > token
> > > > > >> >> > > > > > > state
> > > > > >> >> > > > > > > > > >> changes.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > > Broker
> > > > > >> returns
> > > > > >> >> > the
> > > > > >> >> > > > > token
> > > > > >> >> > > > > > to
> > > > > >> >> > > > > > > > > >> Client.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> Probable
> > > > issues
> > > > > >> and
> > > > > >> >> > fixes
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > > > Probable
> > > > > >> race
> > > > > >> >> > > > > condition,
> > > > > >> >> > > > > > > > client
> > > > > >> >> > > > > > > > > >> tries
> > > > > >> >> > > > > > > > > >> > > to
> > > > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> with
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that
> > is
> > > > yet
> > > > > >> to be
> > > > > >> >> > > > updated
> > > > > >> >> > > > > > with
> > > > > >> >> > > > > > > > the
> > > > > >> >> > > > > > > > > >> newly
> > > > > >> >> > > > > > > > > >> > > > >> generated
> > > > > >> >> > > > > > > > > >> > > > >> >> > DT.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> This
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably
> be
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> dealt
> > > with
> > > > > >> making
> > > > > >> >> > > > > dtRequest
> > > > > >> >> > > > > > > > block
> > > > > >> >> > > > > > > > > >> until
> > > > > >> >> > > > > > > > > >> > > all
> > > > > >> >> > > > > > > > > >> > > > >> >> > brokers have
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> cache.
> > > Zk
> > > > > >> barrier or
> > > > > >> >> > > > > similar
> > > > > >> >> > > > > > > > > >> mechanism
> > > > > >> >> > > > > > > > > >> > > can
> > > > > >> >> > > > > > > > > >> > > > be
> > > > > >> >> > > > > > > > > >> > > > >> used.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > mechanisms
> > > > > >> will
> > > > > >> >> > > increase
> > > > > >> >> > > > > > > > > complexity.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> > > Using a
> > > > > >> static
> > > > > >> >> > > secret
> > > > > >> >> > > > > key
> > > > > >> >> > > > > > > > from
> > > > > >> >> > > > > > > > > >> config
> > > > > >> >> > > > > > > > > >> > > > >> file. Will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> config
> > > and
> > > > > >> uses a
> > > > > >> >> > > static
> > > > > >> >> > > > > > > secret
> > > > > >> >> > > > > > > > > >> key. It
> > > > > >> >> > > > > > > > > >> > > is
> > > > > >> >> > > > > > > > > >> > > > >> advised
> > > > > >> >> > > > > > > > > >> > > > >> >> > to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > periodically.
> > > > > >> This
> > > > > >> >> > can
> > > > > >> >> > > > be
> > > > > >> >> > > > > > > > avoided
> > > > > >> >> > > > > > > > > >> with
> > > > > >> >> > > > > > > > > >> > > > >> controller
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey
> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > passing
> > > to
> > > > > >> brokers
> > > > > >> >> > > > > > > periodically.
> > > > > >> >> > > > > > > > > >> However,
> > > > > >> >> > > > > > > > > >> > > > >> this will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > maintain
> > > > > >> certain
> > > > > >> >> > > counts
> > > > > >> >> > > > of
> > > > > >> >> > > > > > > > > >> secretKeys.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > (Alternative
> > > > 1)
> > > > > >> Have
> > > > > >> >> > > > > controller
> > > > > >> >> > > > > > > > > >> generate
> > > > > >> >> > > > > > > > > >> > > > >> delegation
> > > > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > > Client
> > > > > >> >> > > authenticates
> > > > > >> >> > > > > > with a
> > > > > >> >> > > > > > > > > >> broker.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> > Once
> > > a
> > > > > >> client is
> > > > > >> >> > > > > > > > authenticated,
> > > > > >> >> > > > > > > > > >> it
> > > > > >> >> > > > > > > > > >> > > will
> > > > > >> >> > > > > > > > > >> > > > >> make a
> > > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> issue
> > a
> > > > > >> delegation
> > > > > >> >> > > > token.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> > > Broker
> > > > > >> forwards
> > > > > >> >> > the
> > > > > >> >> > > > > > request
> > > > > >> >> > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > > controller.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > > > > Controller
> > > > > >> >> > > generates
> > > > > >> >> > > > a
> > > > > >> >> > > > > DT
> > > > > >> >> > > > > > > and
> > > > > >> >> > > > > > > > > >> > > broadcasts
> > > > > >> >> > > > > > > > > >> > > > >> to all
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5.
> > > Broker
> > > > > >> stores
> > > > > >> >> > this
> > > > > >> >> > > > > token
> > > > > >> >> > > > > > in
> > > > > >> >> > > > > > > > its
> > > > > >> >> > > > > > > > > >> memory
> > > > > >> >> > > > > > > > > >> > > > >> cache.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > > > > Controller
> > > > > >> >> > responds
> > > > > >> >> > > > to
> > > > > >> >> > > > > > > > broker’s
> > > > > >> >> > > > > > > > > >> DT
> > > > > >> >> > > > > > > > > >> > > req.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7.
> > > Broker
> > > > > >> returns
> > > > > >> >> > the
> > > > > >> >> > > > > token
> > > > > >> >> > > > > > to
> > > > > >> >> > > > > > > > > >> Client.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> Probable
> > > > issues
> > > > > >> and
> > > > > >> >> > fixes
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> We
> > > will
> > > > > >> have to
> > > > > >> >> > add
> > > > > >> >> > > > new
> > > > > >> >> > > > > > > APIs
> > > > > >> >> > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > support
> > > > > >> >> > > > > > > > > >> > > > >> >> > controller
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > brokers
> > > on
> > > > > >> top of
> > > > > >> >> > the
> > > > > >> >> > > > > > minimal
> > > > > >> >> > > > > > > > APIs
> > > > > >> >> > > > > > > > > >> that
> > > > > >> >> > > > > > > > > >> > > are
> > > > > >> >> > > > > > > > > >> > > > >> >> > currently
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> We
> > > will
> > > > > >> also have
> > > > > >> >> > > to
> > > > > >> >> > > > > add
> > > > > >> >> > > > > > > APIs
> > > > > >> >> > > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > support
> > > > > >> >> > > > > > > > > >> > > > >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> when a
> > > new
> > > > > >> broker
> > > > > >> >> > > comes
> > > > > >> >> > > > up
> > > > > >> >> > > > > > it
> > > > > >> >> > > > > > > > will
> > > > > >> >> > > > > > > > > >> have
> > > > > >> >> > > > > > > > > >> > > to
> > > > > >> >> > > > > > > > > >> > > > >> get all
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > > > >> controller.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> In
> > > > > >> catastrophic
> > > > > >> >> > > > > failures
> > > > > >> >> > > > > > > > where
> > > > > >> >> > > > > > > > > >> all
> > > > > >> >> > > > > > > > > >> > > > brokers
> > > > > >> >> > > > > > > > > >> > > > >> go
> > > > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> > lost
> > > > even
> > > > > >> if
> > > > > >> >> > > servers
> > > > > >> >> > > > > are
> > > > > >> >> > > > > > > > > >> restarted as
> > > > > >> >> > > > > > > > > >> > > > >> tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If
> > this
> > > > > >> happens,
> > > > > >> >> > then
> > > > > >> >> > > > > there
> > > > > >> >> > > > > > > are
> > > > > >> >> > > > > > > > > more
> > > > > >> >> > > > > > > > > >> > > > important
> > > > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> maybe
> > it
> > > > is
> > > > > >> better
> > > > > >> >> > to
> > > > > >> >> > > > > > > > > >> re-authenticate.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > (Alternative
> > > > 2)
> > > > > >> Do not
> > > > > >> >> > > > > > distribute
> > > > > >> >> > > > > > > > DT
> > > > > >> >> > > > > > > > > to
> > > > > >> >> > > > > > > > > >> > > other
> > > > > >> >> > > > > > > > > >> > > > >> brokers
> > > > > >> >> > > > > > > > > >> > > > >> >> > at
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > > Client
> > > > > >> >> > > authenticates
> > > > > >> >> > > > > > with a
> > > > > >> >> > > > > > > > > >> broker.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> > Once
> > > a
> > > > > >> client is
> > > > > >> >> > > > > > > > authenticated,
> > > > > >> >> > > > > > > > > >> it
> > > > > >> >> > > > > > > > > >> > > will
> > > > > >> >> > > > > > > > > >> > > > >> make a
> > > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> issue
> > a
> > > > > >> delegation
> > > > > >> >> > > > token.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> The
> > > > > broker
> > > > > >> >> > > generates
> > > > > >> >> > > > DT
> > > > > >> >> > > > > > of
> > > > > >> >> > > > > > > > > form,
> > > > > >> >> > > > > > > > > >> > > [hmac +
> > > > > >> >> > > > > > > > > >> > > > >> (owner,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > maxLifeTime,
> > > > > >> id,
> > > > > >> >> > hmac,
> > > > > >> >> > > > > > > > > >> expirationTime)]
> > > > > >> >> > > > > > > > > >> > > and
> > > > > >> >> > > > > > > > > >> > > > >> passes
> > > > > >> >> > > > > > > > > >> > > > >> >> > back
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> this
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac
> > is
> > > > > >> generated
> > > > > >> >> > via
> > > > > >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> > > > > >> >> > > > > > > > > >> > > > >> renewer,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> expirationTime)
> > > > > >> >> > using
> > > > > >> >> > > > > > > > SecretKey}.
> > > > > >> >> > > > > > > > > >> Note
> > > > > >> >> > > > > > > > > >> > > that
> > > > > >> >> > > > > > > > > >> > > > >> all
> > > > > >> >> > > > > > > > > >> > > > >> >> > brokers
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> have
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> SecretKey.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > > Client
> > > > > >> then goes
> > > > > >> >> > to
> > > > > >> >> > > > any
> > > > > >> >> > > > > > > > broker
> > > > > >> >> > > > > > > > > >> and to
> > > > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> Broker
> > > > > >> recalculates
> > > > > >> >> > > hmac
> > > > > >> >> > > > > > using
> > > > > >> >> > > > > > > > > >> (owner,
> > > > > >> >> > > > > > > > > >> > > > >> renewer,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> expirationTime) info
> > > > > >> >> > > > from
> > > > > >> >> > > > > DT
> > > > > >> >> > > > > > > and
> > > > > >> >> > > > > > > > > its
> > > > > >> >> > > > > > > > > >> > > > >> SecretKey. If
> > > > > >> >> > > > > > > > > >> > > > >> >> > it
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT,
> > > client
> > > > > is
> > > > > >> >> > > > > authenticated.
> > > > > >> >> > > > > > > > Yes,
> > > > > >> >> > > > > > > > > >> it will
> > > > > >> >> > > > > > > > > >> > > > do
> > > > > >> >> > > > > > > > > >> > > > >> other
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > timestamp
> > > > > >> expiry and
> > > > > >> >> > > > such.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note
> that
> > > > > secret
> > > > > >> key
> > > > > >> >> > will
> > > > > >> >> > > > be
> > > > > >> >> > > > > > > > > generated
> > > > > >> >> > > > > > > > > >> by
> > > > > >> >> > > > > > > > > >> > > > >> controller
> > > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > periodically.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> Probable
> > > > issues
> > > > > >> and
> > > > > >> >> > fixes
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> How
> > > to
> > > > > >> delete a
> > > > > >> >> > DT?
> > > > > >> >> > > > > Yes,
> > > > > >> >> > > > > > > that
> > > > > >> >> > > > > > > > > is
> > > > > >> >> > > > > > > > > >> a
> > > > > >> >> > > > > > > > > >> > > > downside
> > > > > >> >> > > > > > > > > >> > > > >> >> > here.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> > > handled
> > > > > >> with
> > > > > >> >> > > brokers
> > > > > >> >> > > > > > > > > maintaining
> > > > > >> >> > > > > > > > > >> a
> > > > > >> >> > > > > > > > > >> > > > >> blacklist of
> > > > > >> >> > > > > > > > > >> > > > >> >> > DTs,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can
> be
> > > > > >> removed after
> > > > > >> >> > > > > expiry.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> In
> > > > > >> catastrophic
> > > > > >> >> > > > > failures
> > > > > >> >> > > > > > > > where
> > > > > >> >> > > > > > > > > >> all
> > > > > >> >> > > > > > > > > >> > > > brokers
> > > > > >> >> > > > > > > > > >> > > > >> go
> > > > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> > lost
> > > > even
> > > > > >> if
> > > > > >> >> > > servers
> > > > > >> >> > > > > are
> > > > > >> >> > > > > > > > > >> restarted as
> > > > > >> >> > > > > > > > > >> > > > >> tokens
> > > > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If
> > this
> > > > > >> happens,
> > > > > >> >> > then
> > > > > >> >> > > > > there
> > > > > >> >> > > > > > > are
> > > > > >> >> > > > > > > > > more
> > > > > >> >> > > > > > > > > >> > > > important
> > > > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> maybe
> > it
> > > > is
> > > > > >> better
> > > > > >> >> > to
> > > > > >> >> > > > > > > > > >> re-authenticate.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri,
> > Feb
> > > > 26,
> > > > > >> 2016 at
> > > > > >> >> > > > 1:58
> > > > > >> >> > > > > > PM,
> > > > > >> >> > > > > > > > > Parth
> > > > > >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > pbrahmbhatt@hortonworks.com>
> > > > > >> >> > > > > > > > wrote:
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have
> > > filed
> > > > > >> KIP-48 so
> > > > > >> >> > > we
> > > > > >> >> > > > > can
> > > > > >> >> > > > > > > > offer
> > > > > >> >> > > > > > > > > >> hadoop
> > > > > >> >> > > > > > > > > >> > > > like
> > > > > >> >> > > > > > > > > >> > > > >> >> > delegation
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka.
> > You
> > > > can
> > > > > >> review
> > > > > >> >> > > the
> > > > > >> >> > > > > > design
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This
> KIP
> > > > > >> depends on
> > > > > >> >> > > KIP-43
> > > > > >> >> > > > > and
> > > > > >> >> > > > > > > we
> > > > > >> >> > > > > > > > > >> have also
> > > > > >> >> > > > > > > > > >> > > > >> >> > discussed an
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> proposed
> > > > > design
> > > > > >> here<
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> Regards,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > > >> >> > > > > > > > > >> > > > >>
> > > > > >> >> > > > > > > > > >> > > >
> > > > > >> >> > > > > > > > > >> > >
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > > >>
> > > > > >> >> > > > > > > > > >
> > > > > >> >> > > > > > > > >
> > > > > >> >> > > > > > > >
> > > > > >> >> > > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > --
> > > > > >> >> > > > > > Regards,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Rajini
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > > > --
> > > > > >> >> > > > Regards,
> > > > > >> >> > > >
> > > > > >> >> > > > Rajini
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >> Liquan Pei
> > > > > >> >> Software Engineer, Confluent Inc
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Parth,
Thanks for the reply.
It makes sense to only allow the renewal by users that authenticated using
*non* delegation token mechanism. Then, should we make the renewal a list?
For example, in the case of rest proxy, it will be useful for every
instance of rest proxy to be able to renew the tokens.
It would be clearer if we can document the request protocol like
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0)
.
It would also be useful to document the client APIs.
Thanks,
Jun
On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt <
brahmbhatt.parth@gmail.com> wrote:
> Hi,
>
> I am suggesting that we will only allow the renewal by users that
> authenticated using *non* delegation token mechanism. For example, If user
> Alice authenticated using kerberos and requested delegation tokens, only
> user Alice authenticated via non delegation token mechanism can renew.
> Clients that have access to delegation tokens can not issue renewal
> request for renewing their own token and this is primarily important to
> reduce the time window for which a compromised token will be valid.
>
> To clarify, Yes any authenticated user can request delegation tokens but
> even here I would recommend to avoid creating a chain where a client
> authenticated via delegation token request for more delegation tokens.
> Basically anyone can request delegation token, as long as they authenticate
> via a non delegation token mechanism.
>
> Aren't classes listed here
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-PublicInterfaces
> >
> sufficient?
>
> Thanks
> Parth
>
>
>
> On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Parth,
> >
> > Thanks for the reply. A couple of comments inline below.
> >
> > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> > brahmbhatt.parth@gmail.com> wrote:
> >
> > > 1. Who / how are tokens renewed? By original requester only? or using
> > > Kerberos
> > > auth only?
> > > My recommendation is to do this only using Kerberos auth and only threw
> > the
> > > renewer specified during the acquisition request.
> > >
> > >
> > Hmm, not sure that I follow this. Are you saying that any client
> > authenticated with the delegation token can renew, i.e. there is no
> renewer
> > needed?
> >
> > Also, just to be clear, any authenticated client (either through SASL or
> > SSL) can request a delegation token for the authenticated user, right?
> >
> >
> > > 2. Are tokens stored on each broker or in ZK?
> > > My recommendation is still to store in ZK or not store them at all. The
> > > whole controller based distribution is too much overhead with not much
> to
> > > achieve.
> > >
> > > 3. How are tokens invalidated / expired?
> > > Either by expiration time out or through an explicit request to
> > invalidate.
> > >
> > > 4. Which encryption algorithm is used?
> > > SCRAM
> > >
> > > 5. What is the impersonation proposal (it wasn't in the KIP but was
> > > discussed
> > > in this thread)?
> > > There is no imperonation proposal. I tried and explained how its a
> > > different problem and why its not really necessary to discuss that as
> > part
> > > of this KIP. This KIP will not support any impersonation, it will just
> > be
> > > another way to authenticate.
> > >
> > > 6. Do we need new ACLs, if so - for what actions?
> > > We do not need new ACLs.
> > >
> > >
> > Could we document the format of the new request/response and their
> > associated Resource and Operation for ACL?
> >
> >
> > > 7. How would the delegation token be configured in the client?
> > > Should be through config. I wasn't planning on supporting JAAS for
> > tokens.
> > > I don't believe hadoop does this either.
> > >
> > > Thanks
> > > Parth
> > >
> > >
> > >
> > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Harsha,
> > > >
> > > > Another question.
> > > >
> > > > 9. How would the delegation token be configured in the client? The
> > > standard
> > > > way is to do this through JAAS. However, we will need to think
> through
> > if
> > > > this is convenient in a shared environment. For example, when a new
> > task
> > > is
> > > > added to a Storm worker node, do we need to dynamically add a new
> > section
> > > > in the JAAS file? It may be more convenient if we can pass in the
> token
> > > > through the config directly w/o going through JAAS.
> > > >
> > > > Are you or Parth still actively working on this KIP?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > >
> > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Just to add on that list.
> > > > >
> > > > > 2. It would be good to document the format of the data stored in
> ZK.
> > > > > 7. Earlier, there was a discussion on whether the tokens should be
> > > > > propagated through ZK like config/acl/quota, or through the
> > controller.
> > > > > Currently, the controller is only designed for propagating topic
> > > > metadata,
> > > > > but not other data.
> > > > > 8. Should we use SCRAM to send the token instead of DIGEST-MD5
> since
> > > it's
> > > > > deprecated?
> > > > >
> > > > > Also, the images in the wiki seem broken.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io>
> > > > wrote:
> > > > >
> > > > >> From what I can see, remaining questions are:
> > > > >>
> > > > >> 1. Who / how are tokens renewed? By original requester only? or
> > using
> > > > >> Kerberos auth only?
> > > > >> 2. Are tokens stored on each broker or in ZK?
> > > > >> 3. How are tokens invalidated / expired?
> > > > >> 4. Which encryption algorithm is used?
> > > > >> 5. What is the impersonation proposal (it wasn't in the KIP but
> was
> > > > >> discussed in this thread)?
> > > > >> 6. Do we need new ACLs, if so - for what actions?
> > > > >>
> > > > >> Gwen
> > > > >>
> > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> > > > >> > Jun & Ismael,
> > > > >> > Unfortunately I couldn't attend the KIP
> > > > meeting
> > > > >> > when delegation tokens discussed.
> > > Appreciate
> > > > if
> > > > >> > you can update the thread if you have
> any
> > > > >> > further questions.
> > > > >> > Thanks,
> > > > >> > Harsha
> > > > >> >
> > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > > >> >> It seems that the links to images in the KIP are broken.
> > > > >> >>
> > > > >> >> Liquan
> > > > >> >>
> > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > > >> >>
> > > > >> >> > 110. What does getDelegationTokenAs mean?
> > > > >> >> > In the current proposal we only allow a user to get
> delegation
> > > > token
> > > > >> for
> > > > >> >> > the identity that it authenticated as using another
> mechanism,
> > > i.e.
> > > > >> A user
> > > > >> >> > that authenticate using a keytab for principal
> > user1@EXAMPLE.COM
> > > > >> will get
> > > > >> >> > delegation tokens for that user only. In future I think we
> will
> > > > have
> > > > >> to
> > > > >> >> > extend support such that we allow some set of users (
> > > > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to
> > > acquire
> > > > >> >> > delegation tokens on behalf of other users whose identity
> they
> > > have
> > > > >> >> > verified independently. Kafka brokers will have ACLs to
> > control
> > > > >> which
> > > > >> >> > users are allowed to impersonate other users and get tokens
> on
> > > > >> behalf of
> > > > >> >> > them. Overall Impersonation is a whole different problem in
> my
> > > > >> opinion and
> > > > >> >> > I think we can tackle it in separate KIP.
> > > > >> >> >
> > > > >> >> > 111. What's the typical rate of getting and renewing
> delegation
> > > > >> tokens?
> > > > >> >> > Typically this should be very very low, 1 request per minute
> > is a
> > > > >> >> > relatively high estimate. However it depends on the token
> > > > >> expiration. I am
> > > > >> >> > less worried about the extra load it puts on controller vs
> the
> > > > added
> > > > >> >> > complexity and the value it offers.
> > > > >> >> >
> > > > >> >> > Thanks
> > > > >> >> > Parth
> > > > >> >> >
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> > ismael@juma.me.uk>
> > > > >> wrote:
> > > > >> >> >
> > > > >> >> > > Thanks Rajini. It would probably require a separate KIP as
> it
> > > > will
> > > > >> >> > > introduce user visible changes. We could also update KIP-48
> > to
> > > > >> have this
> > > > >> >> > > information, but it seems cleaner to do it separately. We
> can
> > > > >> discuss
> > > > >> >> > that
> > > > >> >> > > in the KIP call today.
> > > > >> >> > >
> > > > >> >> > > Ismael
> > > > >> >> > >
> > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > > >> >> > >
> > > > >> >> > > > Ismael,
> > > > >> >> > > >
> > > > >> >> > > > I have created a JIRA (
> > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that need
> > another
> > > > >> KIP? If
> > > > >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA
> > that
> > > > gets
> > > > >> >> > > reviewed
> > > > >> >> > > > when the PR is ready?
> > > > >> >> > > >
> > > > >> >> > > > Thank you,
> > > > >> >> > > >
> > > > >> >> > > > Rajini
> > > > >> >> > > >
> > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > > ismael@juma.me.uk>
> > > > >> >> > wrote:
> > > > >> >> > > >
> > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > > >> >> > > > >
> > > > >> >> > > > > Gwen had independently mentioned this as a SASL
> mechanism
> > > > that
> > > > >> might
> > > > >> >> > be
> > > > >> >> > > > > useful for Kafka and I have been meaning to check it in
> > > more
> > > > >> detail.
> > > > >> >> > > Good
> > > > >> >> > > > > to know that you are willing to contribute an
> > > implementation.
> > > > >> Maybe
> > > > >> >> > we
> > > > >> >> > > > > should file a separate JIRA for this?
> > > > >> >> > > > >
> > > > >> >> > > > > Ismael
> > > > >> >> > > > >
> > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > > >> >> > > > >
> > > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> > > Mechanism)
> > > > >> is a
> > > > >> >> > > better
> > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> > > > built-in
> > > > >> SCRAM
> > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy to add
> > > > support
> > > > >> in
> > > > >> >> > Kafka
> > > > >> >> > > > > since
> > > > >> >> > > > > > it would be a useful mechanism to support anyway.
> > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the
> > > protocol
> > > > >> for
> > > > >> >> > > > > > SCRAM-SHA-256.
> > > > >> >> > > > > >
> > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > > jun@confluent.io
> > > > >
> > > > >> wrote:
> > > > >> >> > > > > >
> > > > >> >> > > > > > > Parth,
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > Thanks for the explanation. A couple of more
> > questions.
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > 111. What's the typical rate of getting and
> renewing
> > > > >> delegation
> > > > >> >> > > > tokens?
> > > > >> >> > > > > > > That may have an impact on whether they should be
> > > > directed
> > > > >> to the
> > > > >> >> > > > > > > controller.
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > Jun
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> > > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > > Hi Jun,
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > Thanks for reviewing.
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > * We could add a Cluster action to add acls on
> who
> > > can
> > > > >> request
> > > > >> >> > > > > > delegation
> > > > >> >> > > > > > > > tokens. I don't see the use case for that yet but
> > > down
> > > > >> the line
> > > > >> >> > > > when
> > > > >> >> > > > > we
> > > > >> >> > > > > > > > start supporting getDelegationTokenAs it will be
> > > > >> necessary.
> > > > >> >> > > > > > > > * Yes we recommend tokens to be only
> > used/distributed
> > > > >> over
> > > > >> >> > secure
> > > > >> >> > > > > > > channels.
> > > > >> >> > > > > > > > * Depending on what design we end up choosing
> > > > >> Invalidation will
> > > > >> >> > > be
> > > > >> >> > > > > > > > responsibility of every broker or controller.
> > > > >> >> > > > > > > > * I am not sure if I documented somewhere that
> > > > >> invalidation
> > > > >> >> > will
> > > > >> >> > > > > > directly
> > > > >> >> > > > > > > > go through zookeeper but that is not the intent.
> > > > >> Invalidation
> > > > >> >> > > will
> > > > >> >> > > > > > either
> > > > >> >> > > > > > > > be request based or due to expiration. No direct
> > > > >> zookeeper
> > > > >> >> > > > > interaction
> > > > >> >> > > > > > > from
> > > > >> >> > > > > > > > any client.
> > > > >> >> > > > > > > > * "Broker also stores the DelegationToken without
> > the
> > > > >> hmac in
> > > > >> >> > the
> > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
> > > > >> purpose of
> > > > >> >> > > > > zookeeper
> > > > >> >> > > > > > in
> > > > >> >> > > > > > > > this design is as distribution channel for tokens
> > > > >> between all
> > > > >> >> > > > brokers
> > > > >> >> > > > > > > and a
> > > > >> >> > > > > > > > layer that ensures only tokens that were
> generated
> > by
> > > > >> making a
> > > > >> >> > > > > request
> > > > >> >> > > > > > > to a
> > > > >> >> > > > > > > > broker will be accepted (more on this in second
> > > > >> paragraph). The
> > > > >> >> > > > token
> > > > >> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> > > > >> expiration,
> > > > >> >> > > hmac)
> > > > >> >> > > > ,
> > > > >> >> > > > > > one
> > > > >> >> > > > > > > of
> > > > >> >> > > > > > > > which is the finally generated hmac but hmac it
> > self
> > > is
> > > > >> >> > derivable
> > > > >> >> > > > if
> > > > >> >> > > > > > you
> > > > >> >> > > > > > > > have all the other elements of the token + secret
> > key
> > > > to
> > > > >> >> > generate
> > > > >> >> > > > > hmac.
> > > > >> >> > > > > > > > Given zookeeper does not provide SSL support we
> do
> > > not
> > > > >> want the
> > > > >> >> > > > > entire
> > > > >> >> > > > > > > > token to be wire transferred to zookeeper as that
> > > will
> > > > >> be an
> > > > >> >> > > > insecure
> > > > >> >> > > > > > > wire
> > > > >> >> > > > > > > > transfer. Instead we only store all the other
> > > elements
> > > > >> of a
> > > > >> >> > > > > delegation
> > > > >> >> > > > > > > > tokens. Brokers can read these elements and
> because
> > > > they
> > > > >> also
> > > > >> >> > > have
> > > > >> >> > > > > > access
> > > > >> >> > > > > > > > to secret key they will be able to generate hmac
> on
> > > > >> their end.
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > One of the alternative proposed is to avoid
> > zookeeper
> > > > >> >> > > altogether. A
> > > > >> >> > > > > > > Client
> > > > >> >> > > > > > > > will call broker with required information
> (owner,
> > > > >> renwer,
> > > > >> >> > > > > expiration)
> > > > >> >> > > > > > > and
> > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't store
> > this
> > > > in
> > > > >> >> > > zookeeper.
> > > > >> >> > > > > > From
> > > > >> >> > > > > > > > this point a client can contact any broker with
> all
> > > the
> > > > >> >> > > delegation
> > > > >> >> > > > > > token
> > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the
> > > borker
> > > > >> will
> > > > >> >> > > > > regenerate
> > > > >> >> > > > > > > the
> > > > >> >> > > > > > > > hmac and as long as it matches with hmac
> presented
> > by
> > > > >> client ,
> > > > >> >> > > > broker
> > > > >> >> > > > > > > will
> > > > >> >> > > > > > > > allow the request to authenticate. Only problem
> > with
> > > > >> this
> > > > >> >> > > approach
> > > > >> >> > > > > is
> > > > >> >> > > > > > if
> > > > >> >> > > > > > > > the secret key is compromised any client can now
> > > > generate
> > > > >> >> > random
> > > > >> >> > > > > tokens
> > > > >> >> > > > > > > and
> > > > >> >> > > > > > > > they will still be able to authenticate as any
> user
> > > > they
> > > > >> like.
> > > > >> >> > > with
> > > > >> >> > > > > > > > zookeeper we guarantee that only tokens acquired
> > via
> > > a
> > > > >> broker
> > > > >> >> > > > (using
> > > > >> >> > > > > > some
> > > > >> >> > > > > > > > auth scheme other than delegation token) will be
> > > > >> accepted. We
> > > > >> >> > > need
> > > > >> >> > > > to
> > > > >> >> > > > > > > > discuss which proposal makes more sense and we
> can
> > go
> > > > >> over it
> > > > >> >> > in
> > > > >> >> > > > > > > tomorrow's
> > > > >> >> > > > > > > > meeting.
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > Thanks
> > > > >> >> > > > > > > > Parth
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > > > >> jun@confluent.io>
> > > > >> >> > > > wrote:
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 100. This potentially can be useful for Kafka
> > > Connect
> > > > >> and
> > > > >> >> > Kafka
> > > > >> >> > > > > rest
> > > > >> >> > > > > > > > proxy
> > > > >> >> > > > > > > > > where a worker agent will need to run a task on
> > > > behalf
> > > > >> of a
> > > > >> >> > > > client.
> > > > >> >> > > > > > We
> > > > >> >> > > > > > > > will
> > > > >> >> > > > > > > > > likely need to change how those services use
> > Kafka
> > > > >> clients
> > > > >> >> > > > > > > > > (producer/consumer). Instead of a shared client
> > per
> > > > >> worker,
> > > > >> >> > we
> > > > >> >> > > > will
> > > > >> >> > > > > > > need
> > > > >> >> > > > > > > > a
> > > > >> >> > > > > > > > > client per user task since the authentication
> > > happens
> > > > >> at the
> > > > >> >> > > > > > connection
> > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer will be
> the
> > > > >> workers.
> > > > >> >> > So,
> > > > >> >> > > we
> > > > >> >> > > > > > > > probably
> > > > >> >> > > > > > > > > need to allow multiple renewers. For Kafka rest
> > > > proxy,
> > > > >> the
> > > > >> >> > > > renewer
> > > > >> >> > > > > > can
> > > > >> >> > > > > > > > > probably just be the creator of the token.
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 101. Do we need new acl on who can request
> > > delegation
> > > > >> tokens?
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 102. Do we recommend people to send delegation
> > > tokens
> > > > >> in an
> > > > >> >> > > > > encrypted
> > > > >> >> > > > > > > > > channel?
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 103. Who is responsible for expiring tokens,
> > every
> > > > >> broker?
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 104. For invalidating tokens, would it be
> better
> > to
> > > > do
> > > > >> it in
> > > > >> >> > a
> > > > >> >> > > > > > request
> > > > >> >> > > > > > > > > instead of going to ZK directly?
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 105. The terminology of client in the wiki
> > > sometimes
> > > > >> refers
> > > > >> >> > to
> > > > >> >> > > > the
> > > > >> >> > > > > > end
> > > > >> >> > > > > > > > > client and some other times refers to the
> client
> > > > using
> > > > >> the
> > > > >> >> > > > > delegation
> > > > >> >> > > > > > > > > tokens. It would be useful to distinguish
> between
> > > the
> > > > >> two.
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > 106. Could you explain the sentence "Broker
> also
> > > > >> stores the
> > > > >> >> > > > > > > > DelegationToken
> > > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit
> more? I
> > > > >> thought the
> > > > >> >> > > > > > > delegation
> > > > >> >> > > > > > > > > token is the hmac.
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > Thanks,
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > Jun
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> > > > >> jun@confluent.io>
> > > > >> >> > > > wrote:
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > > Hi, Harsha,
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can
> > > discuss
> > > > >> this in
> > > > >> >> > > the
> > > > >> >> > > > > > > meeting
> > > > >> >> > > > > > > > > > tomorrow.
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > > > Thanks,
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > > > Jun
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> > > > >> kafka@harsha.io>
> > > > >> >> > > > wrote:
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > > >> Hi All,
> > > > >> >> > > > > > > > > >> Can we have a KIP meeting around
> > > this.
> > > > >> The KIP
> > > > >> >> > is
> > > > >> >> > > > up
> > > > >> >> > > > > > for
> > > > >> >> > > > > > > > > >> sometime and if there are any
> > > questions
> > > > >> lets
> > > > >> >> > > > quickly
> > > > >> >> > > > > > hash
> > > > >> >> > > > > > > > out
> > > > >> >> > > > > > > > > >> details.
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > > >> Thanks,
> > > > >> >> > > > > > > > > >> Harsha
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> > > > brahmbhatt
> > > > >> wrote:
> > > > >> >> > > > > > > > > >> > That is what the hadoop echo system uses
> so
> > no
> > > > >> good
> > > > >> >> > reason
> > > > >> >> > > > > > really.
> > > > >> >> > > > > > > > We
> > > > >> >> > > > > > > > > >> > could
> > > > >> >> > > > > > > > > >> > change it to whatever is the newest
> > > recommended
> > > > >> standard
> > > > >> >> > > is.
> > > > >> >> > > > > > > > > >> >
> > > > >> >> > > > > > > > > >> > Thanks
> > > > >> >> > > > > > > > > >> > Parth
> > > > >> >> > > > > > > > > >> >
> > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael
> > Juma <
> > > > >> >> > > > > ismael@juma.me.uk
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > > > wrote:
> > > > >> >> > > > > > > > > >> >
> > > > >> >> > > > > > > > > >> > > Hi Parth,
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started
> > reviewing
> > > > >> this and
> > > > >> >> > > may
> > > > >> >> > > > > have
> > > > >> >> > > > > > > > > >> additional
> > > > >> >> > > > > > > > > >> > > questions later. The immediate question
> > that
> > > > >> came to
> > > > >> >> > > mind
> > > > >> >> > > > is
> > > > >> >> > > > > > our
> > > > >> >> > > > > > > > > >> choice of
> > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
> > > > >> OBSOLETE in
> > > > >> >> > the
> > > > >> >> > > > IANA
> > > > >> >> > > > > > > > > Registry
> > > > >> >> > > > > > > > > >> of
> > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC
> > (2831)
> > > > has
> > > > >> been
> > > > >> >> > > moved
> > > > >> >> > > > > to
> > > > >> >> > > > > > > > > Historic
> > > > >> >> > > > > > > > > >> > > status:
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > >
> > > > >>
> > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > What is the reasoning behind that
> choice?
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > Thanks,
> > > > >> >> > > > > > > > > >> > > Ismael
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> > > > Shapira <
> > > > >> >> > > > > > > gwen@confluent.io
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > >> wrote:
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that even
> though
> > > > >> delegation
> > > > >> >> > > > tokens
> > > > >> >> > > > > > > are a
> > > > >> >> > > > > > > > > >> Hadoop
> > > > >> >> > > > > > > > > >> > > > > innovation, I feel very strongly
> about
> > > not
> > > > >> adding
> > > > >> >> > > > > > dependency
> > > > >> >> > > > > > > > on
> > > > >> >> > > > > > > > > >> Hadoop
> > > > >> >> > > > > > > > > >> > > > > when implementing delegation tokens
> > for
> > > > >> Kafka. The
> > > > >> >> > > KIP
> > > > >> >> > > > > > > doesn't
> > > > >> >> > > > > > > > > >> imply
> > > > >> >> > > > > > > > > >> > > > > such dependency, but if you can
> > > clarify...
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no
> one
> > > will
> > > > >> read
> > > > >> >> > the
> > > > >> >> > > > KIP
> > > > >> >> > > > > > and
> > > > >> >> > > > > > > > > panic
> > > > >> >> > > > > > > > > >> > > > three weeks before the next release...
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > > * Can we get delegation token at any
> > > time
> > > > >> after
> > > > >> >> > > > > > > > authenticating?
> > > > >> >> > > > > > > > > >> only
> > > > >> >> > > > > > > > > >> > > > > immediately after?
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *As long as you are authenticated
> you
> > > can
> > > > >> get
> > > > >> >> > > > delegation
> > > > >> >> > > > > > > > tokens.
> > > > >> >> > > > > > > > > >> We
> > > > >> >> > > > > > > > > >> > > need
> > > > >> >> > > > > > > > > >> > > > to
> > > > >> >> > > > > > > > > >> > > > > discuss if a client authenticated
> > using
> > > > >> delegation
> > > > >> >> > > > > token,
> > > > >> >> > > > > > > can
> > > > >> >> > > > > > > > > also
> > > > >> >> > > > > > > > > >> > > > acquire
> > > > >> >> > > > > > > > > >> > > > > delegation token again or not. Also
> > > there
> > > > >> is the
> > > > >> >> > > > > question
> > > > >> >> > > > > > of
> > > > >> >> > > > > > > > do
> > > > >> >> > > > > > > > > we
> > > > >> >> > > > > > > > > >> > > allow
> > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation token
> or
> > we
> > > > >> want
> > > > >> >> > > specific
> > > > >> >> > > > > > ACLs
> > > > >> >> > > > > > > (I
> > > > >> >> > > > > > > > > >> think
> > > > >> >> > > > > > > > > >> > > its
> > > > >> >> > > > > > > > > >> > > > an
> > > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > I think we are debating two options:
> > > Either
> > > > >> require
> > > > >> >> > > > > Kerberos
> > > > >> >> > > > > > > > auth
> > > > >> >> > > > > > > > > >> for
> > > > >> >> > > > > > > > > >> > > > renewal or require non-owners to
> renew.
> > > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> > > > basically
> > > > >> >> > require
> > > > >> >> > > a
> > > > >> >> > > > > "job
> > > > >> >> > > > > > > > > master"
> > > > >> >> > > > > > > > > >> > > > to take responsibility for the
> renewal,
> > it
> > > > >> will have
> > > > >> >> > > its
> > > > >> >> > > > > own
> > > > >> >> > > > > > > > > >> identity
> > > > >> >> > > > > > > > > >> > > > anyway and I think this is the correct
> > > > design
> > > > >> >> > pattern
> > > > >> >> > > > > > anyway.
> > > > >> >> > > > > > > > For
> > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> > > > >> renewals?),
> > > > >> >> > but
> > > > >> >> > > > it
> > > > >> >> > > > > is
> > > > >> >> > > > > > > > hard
> > > > >> >> > > > > > > > > to
> > > > >> >> > > > > > > > > >> > > > debate simplicity without looking at
> the
> > > > code
> > > > >> >> > changes
> > > > >> >> > > > > > > required.
> > > > >> >> > > > > > > > If
> > > > >> >> > > > > > > > > >> you
> > > > >> >> > > > > > > > > >> > > > have a draft of how the "require
> > Kerberos"
> > > > >> will look
> > > > >> >> > > in
> > > > >> >> > > > > > Kafka
> > > > >> >> > > > > > > > > code,
> > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > > * My understanding is that tokens
> will
> > > > >> propagate
> > > > >> >> > via
> > > > >> >> > > > ZK
> > > > >> >> > > > > > but
> > > > >> >> > > > > > > > > >> without
> > > > >> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> > > > >> protocol,
> > > > >> >> > > > correct?
> > > > >> >> > > > > > > > Clients
> > > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth
> > > failure
> > > > >> (IIRC),
> > > > >> >> > > but
> > > > >> >> > > > > > since
> > > > >> >> > > > > > > > the
> > > > >> >> > > > > > > > > >> > > > > tokens propagate between brokers
> > asynch,
> > > > we
> > > > >> will
> > > > >> >> > > need
> > > > >> >> > > > to
> > > > >> >> > > > > > > > retry a
> > > > >> >> > > > > > > > > >> bit
> > > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth due to
> > > > timing
> > > > >> >> > issues.
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives
> right
> > > > now.
> > > > >> The
> > > > >> >> > > > current
> > > > >> >> > > > > > > > > documented
> > > > >> >> > > > > > > > > >> > > > approach
> > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does not
> > > require
> > > > >> any
> > > > >> >> > > changes
> > > > >> >> > > > > to
> > > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > > >> >> > > > > > > > > >> > > > > protocol. An alternative approach
> can
> > > > remove
> > > > >> >> > > zookeeper
> > > > >> >> > > > > > > > > dependency
> > > > >> >> > > > > > > > > >> as
> > > > >> >> > > > > > > > > >> > > well
> > > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> > > discussion
> > > > >> call.*
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want
> to
> > > > ping
> > > > >> Jun to
> > > > >> >> > > > > > arrange a
> > > > >> >> > > > > > > > > call?
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of
> > having
> > > > >> just the
> > > > >> >> > > > > > controller
> > > > >> >> > > > > > > > > issue
> > > > >> >> > > > > > > > > >> the
> > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing
> a
> > > > shared
> > > > >> >> > secret.
> > > > >> >> > > > Not
> > > > >> >> > > > > > > sure
> > > > >> >> > > > > > > > if
> > > > >> >> > > > > > > > > >> we
> > > > >> >> > > > > > > > > >> > > > > want to continue the discussion here
> > or
> > > on
> > > > >> the
> > > > >> >> > > wiki. I
> > > > >> >> > > > > > think
> > > > >> >> > > > > > > > > that
> > > > >> >> > > > > > > > > >> we
> > > > >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> > > > >> distribution"
> > > > >> >> > > from
> > > > >> >> > > > > > > "shared
> > > > >> >> > > > > > > > > >> secret
> > > > >> >> > > > > > > > > >> > > > > distribution" and use the controller
> > as
> > > > the
> > > > >> only
> > > > >> >> > > token
> > > > >> >> > > > > > > > generator
> > > > >> >> > > > > > > > > >> to
> > > > >> >> > > > > > > > > >> > > > > solve the second issue, while still
> > > using
> > > > >> ZK async
> > > > >> >> > > to
> > > > >> >> > > > > > > > distribute
> > > > >> >> > > > > > > > > >> > > > > tokens.
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous Email
> I
> > am
> > > > >> fine with
> > > > >> >> > > > that
> > > > >> >> > > > > > > > approach
> > > > >> >> > > > > > > > > >> as
> > > > >> >> > > > > > > > > >> > > long
> > > > >> >> > > > > > > > > >> > > > as
> > > > >> >> > > > > > > > > >> > > > > we agree that the extra complexity
> of
> > > > >> >> > > adding/updating
> > > > >> >> > > > > APIS
> > > > >> >> > > > > > > > adds
> > > > >> >> > > > > > > > > >> enough
> > > > >> >> > > > > > > > > >> > > > > value. The advantage with the
> > controller
> > > > >> approach
> > > > >> >> > is
> > > > >> >> > > > > > secret
> > > > >> >> > > > > > > > > >> rotation
> > > > >> >> > > > > > > > > >> > > can
> > > > >> >> > > > > > > > > >> > > > be
> > > > >> >> > > > > > > > > >> > > > > automated,frequent and would not
> > require
> > > > >> >> > > deployment. *
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Can you detail the extra complexity
> (or
> > > > point
> > > > >> me to
> > > > >> >> > > the
> > > > >> >> > > > > > email
> > > > >> >> > > > > > > I
> > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients can
> > already
> > > > >> find the
> > > > >> >> > > > > > controller
> > > > >> >> > > > > > > > from
> > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned
> about
> > > > >> controller
> > > > >> >> > > > load.
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> > > > kerberos
> > > > >> auth
> > > > >> >> > for
> > > > >> >> > > > > > > renwal, I
> > > > >> >> > > > > > > > > >> think
> > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer the the
> > > > request
> > > > >> >> > content
> > > > >> >> > > > in
> > > > >> >> > > > > a
> > > > >> >> > > > > > > > pretty
> > > > >> >> > > > > > > > > >> ugly
> > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to
> > > non-owner
> > > > >> is
> > > > >> >> > > better.
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil.
> > While
> > > > >> this will
> > > > >> >> > > make
> > > > >> >> > > > > the
> > > > >> >> > > > > > > > kafka
> > > > >> >> > > > > > > > > >> code
> > > > >> >> > > > > > > > > >> > > > > pretty straight forward , forcing
> > > renewer
> > > > >> to
> > > > >> >> > > > non-owner
> > > > >> >> > > > > > > pushes
> > > > >> >> > > > > > > > > >> the code
> > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes it even
> > > > harder
> > > > >> to
> > > > >> >> > > > > > integrate. *
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> > > > >> "renewal by
> > > > >> >> > > > other"
> > > > >> >> > > > > > > > approach
> > > > >> >> > > > > > > > > >> is
> > > > >> >> > > > > > > > > >> > > > that ugly for the clients we expect to
> > use
> > > > >> >> > delegation
> > > > >> >> > > > > tokens
> > > > >> >> > > > > > > > since
> > > > >> >> > > > > > > > > >> > > > they will have an app-master of some
> > sort
> > > > who
> > > > >> >> > > requested
> > > > >> >> > > > > the
> > > > >> >> > > > > > > > token
> > > > >> >> > > > > > > > > to
> > > > >> >> > > > > > > > > >> > > > begin with.
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > The response for my question on how
> > > > multiple
> > > > >> >> > > > identities
> > > > >> >> > > > > > will
> > > > >> >> > > > > > > > be
> > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to me -
> > > AFAIK,
> > > > >> we don't
> > > > >> >> > > > > > > > authenticate
> > > > >> >> > > > > > > > > >> each
> > > > >> >> > > > > > > > > >> > > > > request, we authenticate
> connections.
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > *We authenticate connections, and
> only
> > > > when
> > > > >> they
> > > > >> >> > are
> > > > >> >> > > > > being
> > > > >> >> > > > > > > > > >> established.
> > > > >> >> > > > > > > > > >> > > > Let
> > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a question,
> > in
> > > > >> absence of
> > > > >> >> > > > > > > delegation
> > > > >> >> > > > > > > > > >> tokens if
> > > > >> >> > > > > > > > > >> > > > we
> > > > >> >> > > > > > > > > >> > > > > had to support the use case using
> user
> > > > >> TGT's how
> > > > >> >> > > would
> > > > >> >> > > > > we
> > > > >> >> > > > > > do
> > > > >> >> > > > > > > > it?
> > > > >> >> > > > > > > > > >> My
> > > > >> >> > > > > > > > > >> > > point
> > > > >> >> > > > > > > > > >> > > > > was it would be no different with
> > > > delegation
> > > > >> >> > tokens.
> > > > >> >> > > > The
> > > > >> >> > > > > > use
> > > > >> >> > > > > > > > > case
> > > > >> >> > > > > > > > > >> you
> > > > >> >> > > > > > > > > >> > > are
> > > > >> >> > > > > > > > > >> > > > > describing seems more like
> > > impersonation.*
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the things
> > > that
> > > > >> >> > delegation
> > > > >> >> > > > > > tokens
> > > > >> >> > > > > > > > > >> handled.
> > > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > Gwen
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > > > > Thanks
> > > > >> >> > > > > > > > > >> > > > > Parth
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM,
> Gwen
> > > > >> Shapira <
> > > > >> >> > > > > > > > > gwen@confluent.io
> > > > >> >> > > > > > > > > >> >
> > > > >> >> > > > > > > > > >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >
> > > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> Few more comments:
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
> > > > >> doesn't seem
> > > > >> >> > to
> > > > >> >> > > > > have
> > > > >> >> > > > > > > good
> > > > >> >> > > > > > > > > >> > > > >> description of the changes to the
> > Kafka
> > > > >> Protocol.
> > > > >> >> > > > > Sounds
> > > > >> >> > > > > > > like
> > > > >> >> > > > > > > > > >> you are
> > > > >> >> > > > > > > > > >> > > > >> proposing new
> DelegationTokenRequest
> > > and
> > > > >> >> > > > > > RenewTokenRequest
> > > > >> >> > > > > > > > (and
> > > > >> >> > > > > > > > > >> > > > >> matching responses), without
> > detailing
> > > > the
> > > > >> >> > contents
> > > > >> >> > > > of
> > > > >> >> > > > > > the
> > > > >> >> > > > > > > > > >> requests
> > > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you show
> > the
> > > > >> class
> > > > >> >> > > > interface,
> > > > >> >> > > > > > but
> > > > >> >> > > > > > > > not
> > > > >> >> > > > > > > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> underlying protocol. This could be
> > seen
> > > > as
> > > > >> an
> > > > >> >> > > > > > > implementation
> > > > >> >> > > > > > > > > >> detail,
> > > > >> >> > > > > > > > > >> > > > >> but since the binary protocol is
> what
> > > we
> > > > >> provide
> > > > >> >> > to
> > > > >> >> > > > > > > non-Java
> > > > >> >> > > > > > > > > >> clients,
> > > > >> >> > > > > > > > > >> > > > >> we need to show the changes there.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> > > > >> >> > > > > > > > delegationTokenRequestHandler?
> > > > >> >> > > > > > > > > >> Is it
> > > > >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or
> > > > Client?
> > > > >> Its
> > > > >> >> > > > > unclear...
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even
> > though
> > > > >> delegation
> > > > >> >> > > > > tokens
> > > > >> >> > > > > > > are
> > > > >> >> > > > > > > > a
> > > > >> >> > > > > > > > > >> Hadoop
> > > > >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly
> > about
> > > > not
> > > > >> adding
> > > > >> >> > > > > > > dependency
> > > > >> >> > > > > > > > on
> > > > >> >> > > > > > > > > >> Hadoop
> > > > >> >> > > > > > > > > >> > > > >> when implementing delegation tokens
> > for
> > > > >> Kafka.
> > > > >> >> > The
> > > > >> >> > > > KIP
> > > > >> >> > > > > > > > doesn't
> > > > >> >> > > > > > > > > >> imply
> > > > >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> > > > clarify...
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token at
> any
> > > time
> > > > >> after
> > > > >> >> > > > > > > > authenticating?
> > > > >> >> > > > > > > > > >> only
> > > > >> >> > > > > > > > > >> > > > >> immediately after?
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * My understanding is that tokens
> > will
> > > > >> propagate
> > > > >> >> > > via
> > > > >> >> > > > ZK
> > > > >> >> > > > > > but
> > > > >> >> > > > > > > > > >> without
> > > > >> >> > > > > > > > > >> > > > >> additional changes to
> UpdateMetadata
> > > > >> protocol,
> > > > >> >> > > > correct?
> > > > >> >> > > > > > > > Clients
> > > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth
> > > > failure
> > > > >> >> > (IIRC),
> > > > >> >> > > > but
> > > > >> >> > > > > > > since
> > > > >> >> > > > > > > > > the
> > > > >> >> > > > > > > > > >> > > > >> tokens propagate between brokers
> > > asynch,
> > > > >> we will
> > > > >> >> > > need
> > > > >> >> > > > > to
> > > > >> >> > > > > > > > retry
> > > > >> >> > > > > > > > > a
> > > > >> >> > > > > > > > > >> bit
> > > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due
> to
> > > > timing
> > > > >> >> > issues.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
> > > > >> touching ZK
> > > > >> >> > > > directly
> > > > >> >> > > > > > :)
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of
> > having
> > > > >> just the
> > > > >> >> > > > > > controller
> > > > >> >> > > > > > > > > >> issue the
> > > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid
> syncing a
> > > > >> shared
> > > > >> >> > > secret.
> > > > >> >> > > > > Not
> > > > >> >> > > > > > > sure
> > > > >> >> > > > > > > > > if
> > > > >> >> > > > > > > > > >> we
> > > > >> >> > > > > > > > > >> > > > >> want to continue the discussion
> here
> > or
> > > > on
> > > > >> the
> > > > >> >> > > wiki.
> > > > >> >> > > > I
> > > > >> >> > > > > > > think
> > > > >> >> > > > > > > > > >> that we
> > > > >> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> > > > >> distribution"
> > > > >> >> > > from
> > > > >> >> > > > > > > "shared
> > > > >> >> > > > > > > > > >> secret
> > > > >> >> > > > > > > > > >> > > > >> distribution" and use the
> controller
> > as
> > > > >> the only
> > > > >> >> > > > token
> > > > >> >> > > > > > > > > generator
> > > > >> >> > > > > > > > > >> to
> > > > >> >> > > > > > > > > >> > > > >> solve the second issue, while still
> > > using
> > > > >> ZK
> > > > >> >> > async
> > > > >> >> > > to
> > > > >> >> > > > > > > > > distribute
> > > > >> >> > > > > > > > > >> > > > >> tokens.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with
> > infinite
> > > > >> lifetime
> > > > >> >> > of
> > > > >> >> > > > > > tokens
> > > > >> >> > > > > > > > (and
> > > > >> >> > > > > > > > > >> hoped
> > > > >> >> > > > > > > > > >> > > > >> to hear from others in the
> > community) -
> > > > but
> > > > >> >> > having
> > > > >> >> > > > the
> > > > >> >> > > > > > > option
> > > > >> >> > > > > > > > > and
> > > > >> >> > > > > > > > > >> > > > >> documenting it as less secure,
> allows
> > > > >> users to
> > > > >> >> > > > > configure
> > > > >> >> > > > > > > > their
> > > > >> >> > > > > > > > > >> system
> > > > >> >> > > > > > > > > >> > > > >> as the wish.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
> > > > >> kerberos auth
> > > > >> >> > > for
> > > > >> >> > > > > > > renwal,
> > > > >> >> > > > > > > > I
> > > > >> >> > > > > > > > > >> think
> > > > >> >> > > > > > > > > >> > > > >> it mixes the transport layer the
> the
> > > > >> request
> > > > >> >> > > content
> > > > >> >> > > > > in a
> > > > >> >> > > > > > > > > pretty
> > > > >> >> > > > > > > > > >> ugly
> > > > >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> > > > non-owner
> > > > >> is
> > > > >> >> > > better.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * More detailed explanation on what
> > we
> > > > >> plan to do
> > > > >> >> > > for
> > > > >> >> > > > > the
> > > > >> >> > > > > > > > java
> > > > >> >> > > > > > > > > >> clients
> > > > >> >> > > > > > > > > >> > > > >> specifically - new configuration?
> new
> > > > APIs?
> > > > >> >> > > > > > > > > >> > > > >> The response for my question on how
> > > > >> multiple
> > > > >> >> > > > identities
> > > > >> >> > > > > > > will
> > > > >> >> > > > > > > > be
> > > > >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me -
> > > AFAIK,
> > > > >> we
> > > > >> >> > don't
> > > > >> >> > > > > > > > authenticate
> > > > >> >> > > > > > > > > >> each
> > > > >> >> > > > > > > > > >> > > > >> request, we authenticate
> connections.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens
> are
> > > > only
> > > > >> used
> > > > >> >> > in
> > > > >> >> > > > the
> > > > >> >> > > > > > > Hadoop
> > > > >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there
> are
> > > > >> >> > alternatives
> > > > >> >> > > in
> > > > >> >> > > > > > other
> > > > >> >> > > > > > > > > >> ecosystems
> > > > >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and
> > > whether
> > > > >> there
> > > > >> >> > are
> > > > >> >> > > > some
> > > > >> >> > > > > > > > > >> advantages
> > > > >> >> > > > > > > > > >> > > > >> there.
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> Gwen
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM,
> > > Harsha <
> > > > >> >> > > > > kafka@harsha.io
> > > > >> >> > > > > > >
> > > > >> >> > > > > > > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> > > > >> >> > > > > > > > > >> > > > >> > Can you look at
> Parth's
> > > last
> > > > >> reply.
> > > > >> >> > > Does
> > > > >> >> > > > > it
> > > > >> >> > > > > > > > answer
> > > > >> >> > > > > > > > > >> your
> > > > >> >> > > > > > > > > >> > > > >> > concerns.
> > > > >> >> > > > > > > > > >> > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> > Thanks,
> > > > >> >> > > > > > > > > >> > > > >> > Harsha
> > > > >> >> > > > > > > > > >> > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM,
> > > parth
> > > > >> >> > > brahmbhatt
> > > > >> >> > > > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The
> > wiki
> > > > >> already
> > > > >> >> > has
> > > > >> >> > > > > > details
> > > > >> >> > > > > > > on
> > > > >> >> > > > > > > > > >> token
> > > > >> >> > > > > > > > > >> > > > >> >> expiration
> > > > >> >> > > > > > > > > >> > > > >> >> under token acquisition process
> > > > >> >> > > > > > > > > >> > > > >> >> <
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> > > > >> >> > > > > > > > > >> > > > >> >.
> > > > >> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens
> > will
> > > > >> expire
> > > > >> >> > > based
> > > > >> >> > > > > on a
> > > > >> >> > > > > > > > > server
> > > > >> >> > > > > > > > > >> side
> > > > >> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours)
> > > > unless
> > > > >> >> > renewed.
> > > > >> >> > > > > > Renewal
> > > > >> >> > > > > > > > is
> > > > >> >> > > > > > > > > >> only
> > > > >> >> > > > > > > > > >> > > > allowed
> > > > >> >> > > > > > > > > >> > > > >> >> until the max life time of
> token.
> > > > >> >> > Alternatively
> > > > >> >> > > we
> > > > >> >> > > > > > could
> > > > >> >> > > > > > > > > also
> > > > >> >> > > > > > > > > >> make
> > > > >> >> > > > > > > > > >> > > > that
> > > > >> >> > > > > > > > > >> > > > >> >> an
> > > > >> >> > > > > > > > > >> > > > >> >> optional param and the server
> side
> > > > >> default can
> > > > >> >> > > > serve
> > > > >> >> > > > > > as
> > > > >> >> > > > > > > > the
> > > > >> >> > > > > > > > > >> upper
> > > > >> >> > > > > > > > > >> > > > bound.
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> To your second point it will be
> > done
> > > > >> exactly
> > > > >> >> > the
> > > > >> >> > > > > same
> > > > >> >> > > > > > > way
> > > > >> >> > > > > > > > we
> > > > >> >> > > > > > > > > >> would
> > > > >> >> > > > > > > > > >> > > > >> >> support
> > > > >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling
> > client
> > > > >> will have
> > > > >> >> > > to
> > > > >> >> > > > > put
> > > > >> >> > > > > > > the
> > > > >> >> > > > > > > > > >> tokens it
> > > > >> >> > > > > > > > > >> > > > >> wants
> > > > >> >> > > > > > > > > >> > > > >> >> to use in the subject instance
> and
> > > > call
> > > > >> >> > > > > > produce/consume
> > > > >> >> > > > > > > > > inside
> > > > >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will
> > have
> > > to
> > > > >> keep
> > > > >> >> > > track
> > > > >> >> > > > of
> > > > >> >> > > > > > its
> > > > >> >> > > > > > > > own
> > > > >> >> > > > > > > > > >> > > > subject. I
> > > > >> >> > > > > > > > > >> > > > >> >> will have to look at the code to
> > see
> > > > if
> > > > >> we
> > > > >> >> > > support
> > > > >> >> > > > > > this
> > > > >> >> > > > > > > > > >> feature
> > > > >> >> > > > > > > > > >> > > right
> > > > >> >> > > > > > > > > >> > > > >> now
> > > > >> >> > > > > > > > > >> > > > >> >> but my understanding is
> delegation
> > > > token
> > > > >> >> > > shouldn't
> > > > >> >> > > > > > need
> > > > >> >> > > > > > > > any
> > > > >> >> > > > > > > > > >> special
> > > > >> >> > > > > > > > > >> > > > >> >> treatment as its just another
> type
> > > of
> > > > >> >> > Credential
> > > > >> >> > > > in
> > > > >> >> > > > > > the
> > > > >> >> > > > > > > > > >> subject.
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> I would also like to know what
> is
> > > your
> > > > >> opinion
> > > > >> >> > > > about
> > > > >> >> > > > > > > > > infinite
> > > > >> >> > > > > > > > > >> > > renewal
> > > > >> >> > > > > > > > > >> > > > >> (my
> > > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> > > > this),
> > > > >> tokens
> > > > >> >> > > > > > renewing
> > > > >> >> > > > > > > > them
> > > > >> >> > > > > > > > > >> > > self(my
> > > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> > > this)
> > > > >> and
> > > > >> >> > most
> > > > >> >> > > > > > > > importantly
> > > > >> >> > > > > > > > > >> your
> > > > >> >> > > > > > > > > >> > > > choice
> > > > >> >> > > > > > > > > >> > > > >> >> between the alternatives listed
> on
> > > > this
> > > > >> thread
> > > > >> >> > > > > > > > > >> > > > >> >> <
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >>
> > > >
> > >
> >
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> > > > >> >> > > > > > > > > >> > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> > > > >> alternative-2 minus
> > > > >> >> > > > > > > controller
> > > > >> >> > > > > > > > > >> > > > distributing
> > > > >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for
> > reviewing.
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> Thanks
> > > > >> >> > > > > > > > > >> > > > >> >> Parth
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM,
> > Gwen
> > > > >> Shapira <
> > > > >> >> > > > > > > > > >> gwen@confluent.io>
> > > > >> >> > > > > > > > > >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> > Harsha,
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest
> > Proxy.
> > > I
> > > > >> didn't
> > > > >> >> > see
> > > > >> >> > > > > your
> > > > >> >> > > > > > > > design
> > > > >> >> > > > > > > > > >> yet,
> > > > >> >> > > > > > > > > >> > > > but in
> > > > >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> > > > >> producers, which
> > > > >> >> > > > will
> > > > >> >> > > > > > > serve
> > > > >> >> > > > > > > > > >> multiple
> > > > >> >> > > > > > > > > >> > > > users
> > > > >> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since
> > > these
> > > > >> users
> > > > >> >> > > will
> > > > >> >> > > > > have
> > > > >> >> > > > > > > > > >> different
> > > > >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> > > > >> authenticate
> > > > >> >> > > > > separately,
> > > > >> >> > > > > > > and
> > > > >> >> > > > > > > > > >> can't
> > > > >> >> > > > > > > > > >> > > > share a
> > > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > Gwen
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11
> PM,
> > > > >> Harsha <
> > > > >> >> > > > > > > kafka@harsha.io
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > > > >> wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> > > > >> >> > > > > > > > > >> > > > >> >> > > On your second
> > point.
> > > > >> Can you
> > > > >> >> > > > > describe
> > > > >> >> > > > > > a
> > > > >> >> > > > > > > > > >> usecase
> > > > >> >> > > > > > > > > >> > > where
> > > > >> >> > > > > > > > > >> > > > >> >> > > mutliple clients
> > > ended
> > > > up
> > > > >> >> > > sharing a
> > > > >> >> > > > > > > > producer
> > > > >> >> > > > > > > > > >> and
> > > > >> >> > > > > > > > > >> > > even
> > > > >> >> > > > > > > > > >> > > > if
> > > > >> >> > > > > > > > > >> > > > >> they
> > > > >> >> > > > > > > > > >> > > > >> >> > > do why can't they
> > not
> > > > use
> > > > >> >> > single
> > > > >> >> > > > > token
> > > > >> >> > > > > > > that
> > > > >> >> > > > > > > > > >> producer
> > > > >> >> > > > > > > > > >> > > > >> >> > > captures. Why
> would
> > > we
> > > > >> need
> > > > >> >> > > > multiple
> > > > >> >> > > > > > > > clients
> > > > >> >> > > > > > > > > >> with
> > > > >> >> > > > > > > > > >> > > > >> different
> > > > >> >> > > > > > > > > >> > > > >> >> > > tokens sharing a
> > > single
> > > > >> >> > instance
> > > > >> >> > > of
> > > > >> >> > > > > > > > producer.
> > > > >> >> > > > > > > > > >> Also
> > > > >> >> > > > > > > > > >> > > in
> > > > >> >> > > > > > > > > >> > > > >> this
> > > > >> >> > > > > > > > > >> > > > >> >> > > case other
> clients
> > > have
> > > > >> access
> > > > >> >> > > all
> > > > >> >> > > > > the
> > > > >> >> > > > > > > > tokens
> > > > >> >> > > > > > > > > >> no?
> > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> > > > >> >> > > > > > > > > >> > > > >> >> > > Harsha
> > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at
> 11:49
> > > AM,
> > > > >> Gwen
> > > > >> >> > > Shapira
> > > > >> >> > > > > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we
> didn't
> > > see
> > > > >> in the
> > > > >> >> > > wiki:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration
> for
> > > > >> delegation
> > > > >> >> > > > > tokens?
> > > > >> >> > > > > > > > > >> Renewal? How
> > > > >> >> > > > > > > > > >> > > > do we
> > > > >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> > > > >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use
> > delegation
> > > > >> tokens
> > > > >> >> > for
> > > > >> >> > > > > > "do-as"
> > > > >> >> > > > > > > > > (say,
> > > > >> >> > > > > > > > > >> > > submit
> > > > >> >> > > > > > > > > >> > > > >> Storm
> > > > >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will
> > need a
> > > > >> producer
> > > > >> >> > > for
> > > > >> >> > > > > > every
> > > > >> >> > > > > > > > job
> > > > >> >> > > > > > > > > >> (we
> > > > >> >> > > > > > > > > >> > > can't
> > > > >> >> > > > > > > > > >> > > > >> share
> > > > >> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs
> > > > running
> > > > >> on
> > > > >> >> > same
> > > > >> >> > > > > node),
> > > > >> >> > > > > > > > since
> > > > >> >> > > > > > > > > >> we
> > > > >> >> > > > > > > > > >> > > only
> > > > >> >> > > > > > > > > >> > > > >> >> > >> authenticate when
> connecting.
> > > Is
> > > > >> there a
> > > > >> >> > > plan
> > > > >> >> > > > > to
> > > > >> >> > > > > > > > change
> > > > >> >> > > > > > > > > >> this
> > > > >> >> > > > > > > > > >> > > for
> > > > >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order
> > to
> > > > >> allow
> > > > >> >> > > multiple
> > > > >> >> > > > > > users
> > > > >> >> > > > > > > > > with
> > > > >> >> > > > > > > > > >> > > > different
> > > > >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> > > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12
> > AM,
> > > > >> parth
> > > > >> >> > > > > brahmbhatt
> > > > >> >> > > > > > > > > >> > > > >> >> > >> <
> brahmbhatt.parth@gmail.com>
> > > > >> wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more
> > > time,
> > > > >> can
> > > > >> >> > other
> > > > >> >> > > > > > > committers
> > > > >> >> > > > > > > > > >> review?
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> > > > >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at
> > 9:07
> > > > AM,
> > > > >> >> > Harsha <
> > > > >> >> > > > > > > > > >> kafka@harsha.io>
> > > > >> >> > > > > > > > > >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Overall
> current
> > > > >> design looks
> > > > >> >> > > > good
> > > > >> >> > > > > to
> > > > >> >> > > > > > > > me. I
> > > > >> >> > > > > > > > > >> am +1
> > > > >> >> > > > > > > > > >> > > on
> > > > >> >> > > > > > > > > >> > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > KIP.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you
> review
> > > this
> > > > >> as
> > > > >> >> > well.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at
> > > 09:57
> > > > >> AM,
> > > > >> >> > parth
> > > > >> >> > > > > > > > brahmbhatt
> > > > >> >> > > > > > > > > >> wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review
> > > Jitendra.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea
> of
> > > > >> infinite
> > > > >> >> > > > lifetime
> > > > >> >> > > > > > > but I
> > > > >> >> > > > > > > > > >> see the
> > > > >> >> > > > > > > > > >> > > > >> Streaming
> > > > >> >> > > > > > > > > >> > > > >> >> > use
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for
> Streaming
> > > use
> > > > >> case I
> > > > >> >> > > was
> > > > >> >> > > > > > hoping
> > > > >> >> > > > > > > > > >> there will
> > > > >> >> > > > > > > > > >> > > > be
> > > > >> >> > > > > > > > > >> > > > >> some
> > > > >> >> > > > > > > > > >> > > > >> >> > notion
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can
> > get
> > > > new
> > > > >> >> > > > delegation
> > > > >> >> > > > > > > tokens
> > > > >> >> > > > > > > > > at
> > > > >> >> > > > > > > > > >> fixed
> > > > >> >> > > > > > > > > >> > > > >> interval
> > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers.
> > If
> > > > >> that is
> > > > >> >> > not
> > > > >> >> > > > the
> > > > >> >> > > > > > case
> > > > >> >> > > > > > > > for
> > > > >> >> > > > > > > > > >> we can
> > > > >> >> > > > > > > > > >> > > > >> discuss
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens
> > renewing
> > > > >> them self
> > > > >> >> > > and
> > > > >> >> > > > > the
> > > > >> >> > > > > > > > > >> security
> > > > >> >> > > > > > > > > >> > > > >> implications
> > > > >> >> > > > > > > > > >> > > > >> >> > of the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients
> > to
> > > > >> fetch
> > > > >> >> > tokens
> > > > >> >> > > > > from
> > > > >> >> > > > > > > > > >> zookeeper,
> > > > >> >> > > > > > > > > >> > > > >> overall I
> > > > >> >> > > > > > > > > >> > > > >> >> > think
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients
> don't
> > > rely
> > > > >> on our
> > > > >> >> > > > > metadata
> > > > >> >> > > > > > > > store
> > > > >> >> > > > > > > > > >> and I
> > > > >> >> > > > > > > > > >> > > > >> think we
> > > > >> >> > > > > > > > > >> > > > >> >> > are
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that
> direction
> > > > with
> > > > >> all
> > > > >> >> > the
> > > > >> >> > > > > KIP-4
> > > > >> >> > > > > > > > > >> improvements.
> > > > >> >> > > > > > > > > >> > > > I
> > > > >> >> > > > > > > > > >> > > > >> chose
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this
> > case
> > > > the
> > > > >> client
> > > > >> >> > > > will
> > > > >> >> > > > > > > still
> > > > >> >> > > > > > > > > >> just talk
> > > > >> >> > > > > > > > > >> > > > to
> > > > >> >> > > > > > > > > >> > > > >> >> > broker , its
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will
> > use
> > > > >> zookeeper
> > > > >> >> > > > which
> > > > >> >> > > > > we
> > > > >> >> > > > > > > > > >> already do
> > > > >> >> > > > > > > > > >> > > > for a
> > > > >> >> > > > > > > > > >> > > > >> lot
> > > > >> >> > > > > > > > > >> > > > >> >> > of
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> > > > >> development + and
> > > > >> >> > > the
> > > > >> >> > > > > > > ability
> > > > >> >> > > > > > > > > so
> > > > >> >> > > > > > > > > >> > > tokens
> > > > >> >> > > > > > > > > >> > > > >> will
> > > > >> >> > > > > > > > > >> > > > >> >> > survive
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> > > > restart/cluster
> > > > >> >> > > failure.
> > > > >> >> > > > > if a
> > > > >> >> > > > > > > > > >> majority
> > > > >> >> > > > > > > > > >> > > > agrees
> > > > >> >> > > > > > > > > >> > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > added
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> > > > controller
> > > > >> >> > > forwarding
> > > > >> >> > > > > > keys
> > > > >> >> > > > > > > to
> > > > >> >> > > > > > > > > all
> > > > >> >> > > > > > > > > >> > > > broker is
> > > > >> >> > > > > > > > > >> > > > >> >> > justified
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter
> > > security
> > > > >> , I am
> > > > >> >> > > fine
> > > > >> >> > > > > > with
> > > > >> >> > > > > > > > that
> > > > >> >> > > > > > > > > >> option
> > > > >> >> > > > > > > > > >> > > > too.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does
> not
> > > > >> support SSL
> > > > >> >> > > we
> > > > >> >> > > > > can
> > > > >> >> > > > > > > not
> > > > >> >> > > > > > > > > >> store
> > > > >> >> > > > > > > > > >> > > > master
> > > > >> >> > > > > > > > > >> > > > >> keys
> > > > >> >> > > > > > > > > >> > > > >> >> > in
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master
> keys
> > > > will
> > > > >> be
> > > > >> >> > > exposed
> > > > >> >> > > > > on
> > > > >> >> > > > > > > > wire.
> > > > >> >> > > > > > > > > To
> > > > >> >> > > > > > > > > >> > > > support
> > > > >> >> > > > > > > > > >> > > > >> >> > rotation
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting
> > current
> > > > >> clients is
> > > > >> >> > > > > > > something I
> > > > >> >> > > > > > > > > >> need to
> > > > >> >> > > > > > > > > >> > > > put
> > > > >> >> > > > > > > > > >> > > > >> more
> > > > >> >> > > > > > > > > >> > > > >> >> > thought
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current
> proposal
> > > > >> assumes the
> > > > >> >> > > > > rotation
> > > > >> >> > > > > > > > will
> > > > >> >> > > > > > > > > >> > > > invalidate
> > > > >> >> > > > > > > > > >> > > > >> all
> > > > >> >> > > > > > > > > >> > > > >> >> > current
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers
> to
> > > also
> > > > >> review
> > > > >> >> > > and
> > > > >> >> > > > > post
> > > > >> >> > > > > > > > their
> > > > >> >> > > > > > > > > >> > > comments
> > > > >> >> > > > > > > > > >> > > > >> so we
> > > > >> >> > > > > > > > > >> > > > >> >> > can
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016
> at
> > > 8:39
> > > > >> AM,
> > > > >> >> > > Ashish
> > > > >> >> > > > > > Singh
> > > > >> >> > > > > > > <
> > > > >> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> > > > >> >> > > > > > > > > >> > > > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016
> > at
> > > > >> 11:26 AM,
> > > > >> >> > > > > Harsha <
> > > > >> >> > > > > > > > > >> > > > kafka@harsha.io>
> > > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> > > > >> discussion
> > > > >> >> > > threads
> > > > >> >> > > > on
> > > > >> >> > > > > > > this
> > > > >> >> > > > > > > > > KIP.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the
> response
> > > > from
> > > > >> >> > Jitendra
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a
> > large
> > > > >> number of
> > > > >> >> > > > > clients
> > > > >> >> > > > > > > that
> > > > >> >> > > > > > > > > are
> > > > >> >> > > > > > > > > >> > > > running
> > > > >> >> > > > > > > > > >> > > > >> all
> > > > >> >> > > > > > > > > >> > > > >> >> > over the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> > > > authenticate
> > > > >> with
> > > > >> >> > > Kafka
> > > > >> >> > > > > > > > brokers,
> > > > >> >> > > > > > > > > >> is very
> > > > >> >> > > > > > > > > >> > > > >> similar
> > > > >> >> > > > > > > > > >> > > > >> >> > to the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of
> > > large
> > > > >> number
> > > > >> >> > of
> > > > >> >> > > > > tasks
> > > > >> >> > > > > > > > > running
> > > > >> >> > > > > > > > > >> > > across
> > > > >> >> > > > > > > > > >> > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > cluster
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> that
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need
> authentication
> > to
> > > > >> Hdfs
> > > > >> >> > > > Namenode.
> > > > >> >> > > > > > > > > >> Therefore, the
> > > > >> >> > > > > > > > > >> > > > >> >> > delegation token
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem
> > > like
> > > > a
> > > > >> good
> > > > >> >> > fit
> > > > >> >> > > > for
> > > > >> >> > > > > > > this
> > > > >> >> > > > > > > > > use
> > > > >> >> > > > > > > > > >> case
> > > > >> >> > > > > > > > > >> > > > as we
> > > > >> >> > > > > > > > > >> > > > >> >> > have seen
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> it
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large
> > scale
> > > > in
> > > > >> HDFS
> > > > >> >> > and
> > > > >> >> > > > > YARN.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed
> > design
> > > is
> > > > >> very
> > > > >> >> > much
> > > > >> >> > > > > > inline
> > > > >> >> > > > > > > > with
> > > > >> >> > > > > > > > > >> Hadoop
> > > > >> >> > > > > > > > > >> > > > >> >> > approach. A few
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys
> > > want
> > > > >> to allow
> > > > >> >> > > > > > infinite
> > > > >> >> > > > > > > > > >> renewable
> > > > >> >> > > > > > > > > >> > > > >> lifetime
> > > > >> >> > > > > > > > > >> > > > >> >> > for a
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS
> > restricts
> > > a
> > > > >> token
> > > > >> >> > to a
> > > > >> >> > > > max
> > > > >> >> > > > > > > life
> > > > >> >> > > > > > > > > time
> > > > >> >> > > > > > > > > >> > > > (default
> > > > >> >> > > > > > > > > >> > > > >> 7
> > > > >> >> > > > > > > > > >> > > > >> >> > days). A
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's
> > vulnerability
> > > is
> > > > >> >> > believed
> > > > >> >> > > to
> > > > >> >> > > > > > > > increase
> > > > >> >> > > > > > > > > >> with
> > > > >> >> > > > > > > > > >> > > > time.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having
> > > > infinite
> > > > >> >> > > lifetime
> > > > >> >> > > > > > might
> > > > >> >> > > > > > > > not
> > > > >> >> > > > > > > > > >> be the
> > > > >> >> > > > > > > > > >> > > > best
> > > > >> >> > > > > > > > > >> > > > >> idea.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand
> > the
> > > > >> tokens
> > > > >> >> > are
> > > > >> >> > > > > stored
> > > > >> >> > > > > > > in
> > > > >> >> > > > > > > > > >> zookeeper
> > > > >> >> > > > > > > > > >> > > > as
> > > > >> >> > > > > > > > > >> > > > >> well,
> > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> can
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there.
> > This
> > > > is
> > > > >> clever
> > > > >> >> > > as
> > > > >> >> > > > it
> > > > >> >> > > > > > can
> > > > >> >> > > > > > > > > allow
> > > > >> >> > > > > > > > > >> > > > >> replacing the
> > > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out
> of
> > > max
> > > > >> life
> > > > >> >> > > time,
> > > > >> >> > > > > and
> > > > >> >> > > > > > > > > clients
> > > > >> >> > > > > > > > > >> can
> > > > >> >> > > > > > > > > >> > > > >> download
> > > > >> >> > > > > > > > > >> > > > >> >> > new
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> > > > >> shouldn't be
> > > > >> >> > a
> > > > >> >> > > > big
> > > > >> >> > > > > > load
> > > > >> >> > > > > > > > on
> > > > >> >> > > > > > > > > >> > > zookeeper
> > > > >> >> > > > > > > > > >> > > > >> as a
> > > > >> >> > > > > > > > > >> > > > >> >> > client
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new
> > > token
> > > > >> once in
> > > > >> >> > > > > several
> > > > >> >> > > > > > > > days.
> > > > >> >> > > > > > > > > >> In this
> > > > >> >> > > > > > > > > >> > > > >> approach
> > > > >> >> > > > > > > > > >> > > > >> >> > you
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite
> > lifetime
> > > > on
> > > > >> the
> > > > >> >> > > token
> > > > >> >> > > > > even
> > > > >> >> > > > > > > for
> > > > >> >> > > > > > > > > >> long
> > > > >> >> > > > > > > > > >> > > > running
> > > > >> >> > > > > > > > > >> > > > >> >> > clients.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token
> > password
> > > > are
> > > > >> >> > > generated
> > > > >> >> > > > > > using
> > > > >> >> > > > > > > a
> > > > >> >> > > > > > > > > >> master
> > > > >> >> > > > > > > > > >> > > key.
> > > > >> >> > > > > > > > > >> > > > >> The
> > > > >> >> > > > > > > > > >> > > > >> >> > master
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> key
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> > > > >> periodically
> > > > >> >> > > changed.
> > > > >> >> > > > > In
> > > > >> >> > > > > > > > > Hadoop,
> > > > >> >> > > > > > > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> default
> > > > >> >> > > > > > > > > >> > > > >> >> > renewal
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will
> > require
> > > > >> brokers
> > > > >> >> > > > > > maintaining
> > > > >> >> > > > > > > a
> > > > >> >> > > > > > > > > >> list of X
> > > > >> >> > > > > > > > > >> > > > most
> > > > >> >> > > > > > > > > >> > > > >> >> > recent
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> master
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will
> > > have
> > > > >> to be
> > > > >> >> > > > > persisted
> > > > >> >> > > > > > > > > >> somewhere, as
> > > > >> >> > > > > > > > > >> > > > if a
> > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to
> > get
> > > > >> that list
> > > > >> >> > > > again
> > > > >> >> > > > > > and
> > > > >> >> > > > > > > > > >> storing
> > > > >> >> > > > > > > > > >> > > > master
> > > > >> >> > > > > > > > > >> > > > >> keys
> > > > >> >> > > > > > > > > >> > > > >> >> > on ZK
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> is
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> > > > However,
> > > > >> if a
> > > > >> >> > > > broker
> > > > >> >> > > > > > goes
> > > > >> >> > > > > > > > > down
> > > > >> >> > > > > > > > > >> then
> > > > >> >> > > > > > > > > >> > > we
> > > > >> >> > > > > > > > > >> > > > >> have
> > > > >> >> > > > > > > > > >> > > > >> >> > much
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with
> and
> > > > >> client can
> > > > >> >> > > > always
> > > > >> >> > > > > > > > > >> > > re-authenticate
> > > > >> >> > > > > > > > > >> > > > is
> > > > >> >> > > > > > > > > >> > > > >> such
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to
> > take a
> > > > >> look at
> > > > >> >> > > other
> > > > >> >> > > > > > > > > >> alternatives
> > > > >> >> > > > > > > > > >> > > this
> > > > >> >> > > > > > > > > >> > > > >> list has
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a
> > thorough
> > > > >> proposal,
> > > > >> >> > > > great
> > > > >> >> > > > > > > work!"
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7,
> 2016,
> > > at
> > > > >> 10:28
> > > > >> >> > PM,
> > > > >> >> > > > Gwen
> > > > >> >> > > > > > > > Shapira
> > > > >> >> > > > > > > > > >> wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to
> me.
> > > > >> Thanks!
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7,
> > 2016
> > > at
> > > > >> 9:25
> > > > >> >> > PM,
> > > > >> >> > > > > > Harsha <
> > > > >> >> > > > > > > > > >> > > > kafka@harsha.io
> > > > >> >> > > > > > > > > >> > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't
> need
> > > any
> > > > >> release
> > > > >> >> > > > > vehicle
> > > > >> >> > > > > > > but
> > > > >> >> > > > > > > > > >> still the
> > > > >> >> > > > > > > > > >> > > > >> work can
> > > > >> >> > > > > > > > > >> > > > >> >> > move
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> > > > >> interested in
> > > > >> >> > the
> > > > >> >> > > > KIP
> > > > >> >> > > > > > > > please
> > > > >> >> > > > > > > > > >> do the
> > > > >> >> > > > > > > > > >> > > > >> review and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7,
> > > 2016,
> > > > >> at
> > > > >> >> > 04:59
> > > > >> >> > > > PM,
> > > > >> >> > > > > > > Ismael
> > > > >> >> > > > > > > > > >> Juma
> > > > >> >> > > > > > > > > >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that
> it
> > > > >> would be
> > > > >> >> > good
> > > > >> >> > > > to
> > > > >> >> > > > > > have
> > > > >> >> > > > > > > > > more
> > > > >> >> > > > > > > > > >> time
> > > > >> >> > > > > > > > > >> > > to
> > > > >> >> > > > > > > > > >> > > > >> review
> > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar
> 8,
> > > 2016
> > > > >> at
> > > > >> >> > 12:55
> > > > >> >> > > > AM,
> > > > >> >> > > > > > Gwen
> > > > >> >> > > > > > > > > >> Shapira <
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since
> KIP-48
> > > > >> depends on
> > > > >> >> > > > KIP-43,
> > > > >> >> > > > > > > which
> > > > >> >> > > > > > > > > is
> > > > >> >> > > > > > > > > >> > > > already a
> > > > >> >> > > > > > > > > >> > > > >> bit
> > > > >> >> > > > > > > > > >> > > > >> >> > of a
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next
> > > release
> > > > -
> > > > >> any
> > > > >> >> > > chance
> > > > >> >> > > > > we
> > > > >> >> > > > > > > can
> > > > >> >> > > > > > > > > >> delay
> > > > >> >> > > > > > > > > >> > > > >> delegation
> > > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> > > > community
> > > > >> >> > working
> > > > >> >> > > > on a
> > > > >> >> > > > > > > > release
> > > > >> >> > > > > > > > > >> every
> > > > >> >> > > > > > > > > >> > > 3
> > > > >> >> > > > > > > > > >> > > > >> month,
> > > > >> >> > > > > > > > > >> > > > >> >> > this
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb
> > 26,
> > > > >> 2016 at
> > > > >> >> > > 5:11
> > > > >> >> > > > > PM,
> > > > >> >> > > > > > > > Ashish
> > > > >> >> > > > > > > > > >> Singh
> > > > >> >> > > > > > > > > >> > > <
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com
> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks
> > again
> > > > for
> > > > >> the
> > > > >> >> > > > awesome
> > > > >> >> > > > > > > write
> > > > >> >> > > > > > > > > up.
> > > > >> >> > > > > > > > > >> > > > Following
> > > > >> >> > > > > > > > > >> > > > >> our
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I
> > think
> > > > it
> > > > >> will
> > > > >> >> > be
> > > > >> >> > > > > easier
> > > > >> >> > > > > > > to
> > > > >> >> > > > > > > > > >> compare
> > > > >> >> > > > > > > > > >> > > > >> various
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> > > > together.
> > > > >> I am
> > > > >> >> > > > stating
> > > > >> >> > > > > > > > below a
> > > > >> >> > > > > > > > > >> few
> > > > >> >> > > > > > > > > >> > > > >> >> > alternatives along
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> > > > proposal.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
> > > > >> proposal)
> > > > >> >> > Store
> > > > >> >> > > > > > > Delegation
> > > > >> >> > > > > > > > > >> Token,
> > > > >> >> > > > > > > > > >> > > DT,
> > > > >> >> > > > > > > > > >> > > > >> on ZK.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > Client
> > > > >> >> > > authenticates
> > > > >> >> > > > > > with a
> > > > >> >> > > > > > > > > >> broker.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> Once
> > a
> > > > >> client is
> > > > >> >> > > > > > > > authenticated,
> > > > >> >> > > > > > > > > >> it
> > > > >> >> > > > > > > > > >> > > will
> > > > >> >> > > > > > > > > >> > > > >> make a
> > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue
> a
> > > > >> delegation
> > > > >> >> > > > token.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > > > broker
> > > > >> >> > > generates
> > > > >> >> > > > a
> > > > >> >> > > > > > > shared
> > > > >> >> > > > > > > > > >> secret
> > > > >> >> > > > > > > > > >> > > > based
> > > > >> >> > > > > > > > > >> > > > >> on
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> Password/Secret
> > > > >> >> > shared
> > > > >> >> > > > > > between
> > > > >> >> > > > > > > > all
> > > > >> >> > > > > > > > > >> > > brokers,
> > > > >> >> > > > > > > > > >> > > > >> >> > randomly
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > Broker
> > > > >> stores
> > > > >> >> > this
> > > > >> >> > > > > token
> > > > >> >> > > > > > in
> > > > >> >> > > > > > > > its
> > > > >> >> > > > > > > > > >> in
> > > > >> >> > > > > > > > > >> > > > memory
> > > > >> >> > > > > > > > > >> > > > >> cache.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > > >> DelegationToken
> > > > >> >> > > > > without
> > > > >> >> > > > > > > the
> > > > >> >> > > > > > > > > >> hmac in
> > > > >> >> > > > > > > > > >> > > the
> > > > >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All
> > > > >> brokers will
> > > > >> >> > > > have a
> > > > >> >> > > > > > > cache
> > > > >> >> > > > > > > > > >> backed
> > > > >> >> > > > > > > > > >> > > by
> > > > >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> they
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get
> > > notified
> > > > >> >> > whenever
> > > > >> >> > > a
> > > > >> >> > > > > new
> > > > >> >> > > > > > > > token
> > > > >> >> > > > > > > > > is
> > > > >> >> > > > > > > > > >> > > > >> generated and
> > > > >> >> > > > > > > > > >> > > > >> >> > they
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local
> > > cache
> > > > >> whenever
> > > > >> >> > > > token
> > > > >> >> > > > > > > state
> > > > >> >> > > > > > > > > >> changes.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > Broker
> > > > >> returns
> > > > >> >> > the
> > > > >> >> > > > > token
> > > > >> >> > > > > > to
> > > > >> >> > > > > > > > > >> Client.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > > issues
> > > > >> and
> > > > >> >> > fixes
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > > Probable
> > > > >> race
> > > > >> >> > > > > condition,
> > > > >> >> > > > > > > > client
> > > > >> >> > > > > > > > > >> tries
> > > > >> >> > > > > > > > > >> > > to
> > > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> with
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that
> is
> > > yet
> > > > >> to be
> > > > >> >> > > > updated
> > > > >> >> > > > > > with
> > > > >> >> > > > > > > > the
> > > > >> >> > > > > > > > > >> newly
> > > > >> >> > > > > > > > > >> > > > >> generated
> > > > >> >> > > > > > > > > >> > > > >> >> > DT.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> This
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt
> > with
> > > > >> making
> > > > >> >> > > > > dtRequest
> > > > >> >> > > > > > > > block
> > > > >> >> > > > > > > > > >> until
> > > > >> >> > > > > > > > > >> > > all
> > > > >> >> > > > > > > > > >> > > > >> >> > brokers have
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache.
> > Zk
> > > > >> barrier or
> > > > >> >> > > > > similar
> > > > >> >> > > > > > > > > >> mechanism
> > > > >> >> > > > > > > > > >> > > can
> > > > >> >> > > > > > > > > >> > > > be
> > > > >> >> > > > > > > > > >> > > > >> used.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > mechanisms
> > > > >> will
> > > > >> >> > > increase
> > > > >> >> > > > > > > > > complexity.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> > Using a
> > > > >> static
> > > > >> >> > > secret
> > > > >> >> > > > > key
> > > > >> >> > > > > > > > from
> > > > >> >> > > > > > > > > >> config
> > > > >> >> > > > > > > > > >> > > > >> file. Will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config
> > and
> > > > >> uses a
> > > > >> >> > > static
> > > > >> >> > > > > > > secret
> > > > >> >> > > > > > > > > >> key. It
> > > > >> >> > > > > > > > > >> > > is
> > > > >> >> > > > > > > > > >> > > > >> advised
> > > > >> >> > > > > > > > > >> > > > >> >> > to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > periodically.
> > > > >> This
> > > > >> >> > can
> > > > >> >> > > > be
> > > > >> >> > > > > > > > avoided
> > > > >> >> > > > > > > > > >> with
> > > > >> >> > > > > > > > > >> > > > >> controller
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey
> and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> passing
> > to
> > > > >> brokers
> > > > >> >> > > > > > > periodically.
> > > > >> >> > > > > > > > > >> However,
> > > > >> >> > > > > > > > > >> > > > >> this will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> maintain
> > > > >> certain
> > > > >> >> > > counts
> > > > >> >> > > > of
> > > > >> >> > > > > > > > > >> secretKeys.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > (Alternative
> > > 1)
> > > > >> Have
> > > > >> >> > > > > controller
> > > > >> >> > > > > > > > > >> generate
> > > > >> >> > > > > > > > > >> > > > >> delegation
> > > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > Client
> > > > >> >> > > authenticates
> > > > >> >> > > > > > with a
> > > > >> >> > > > > > > > > >> broker.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> Once
> > a
> > > > >> client is
> > > > >> >> > > > > > > > authenticated,
> > > > >> >> > > > > > > > > >> it
> > > > >> >> > > > > > > > > >> > > will
> > > > >> >> > > > > > > > > >> > > > >> make a
> > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue
> a
> > > > >> delegation
> > > > >> >> > > > token.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> > Broker
> > > > >> forwards
> > > > >> >> > the
> > > > >> >> > > > > > request
> > > > >> >> > > > > > > > to
> > > > >> >> > > > > > > > > >> > > > controller.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > > > Controller
> > > > >> >> > > generates
> > > > >> >> > > > a
> > > > >> >> > > > > DT
> > > > >> >> > > > > > > and
> > > > >> >> > > > > > > > > >> > > broadcasts
> > > > >> >> > > > > > > > > >> > > > >> to all
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5.
> > Broker
> > > > >> stores
> > > > >> >> > this
> > > > >> >> > > > > token
> > > > >> >> > > > > > in
> > > > >> >> > > > > > > > its
> > > > >> >> > > > > > > > > >> memory
> > > > >> >> > > > > > > > > >> > > > >> cache.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > > > Controller
> > > > >> >> > responds
> > > > >> >> > > > to
> > > > >> >> > > > > > > > broker’s
> > > > >> >> > > > > > > > > >> DT
> > > > >> >> > > > > > > > > >> > > req.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7.
> > Broker
> > > > >> returns
> > > > >> >> > the
> > > > >> >> > > > > token
> > > > >> >> > > > > > to
> > > > >> >> > > > > > > > > >> Client.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > > issues
> > > > >> and
> > > > >> >> > fixes
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We
> > will
> > > > >> have to
> > > > >> >> > add
> > > > >> >> > > > new
> > > > >> >> > > > > > > APIs
> > > > >> >> > > > > > > > to
> > > > >> >> > > > > > > > > >> > > support
> > > > >> >> > > > > > > > > >> > > > >> >> > controller
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> brokers
> > on
> > > > >> top of
> > > > >> >> > the
> > > > >> >> > > > > > minimal
> > > > >> >> > > > > > > > APIs
> > > > >> >> > > > > > > > > >> that
> > > > >> >> > > > > > > > > >> > > are
> > > > >> >> > > > > > > > > >> > > > >> >> > currently
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We
> > will
> > > > >> also have
> > > > >> >> > > to
> > > > >> >> > > > > add
> > > > >> >> > > > > > > APIs
> > > > >> >> > > > > > > > > to
> > > > >> >> > > > > > > > > >> > > support
> > > > >> >> > > > > > > > > >> > > > >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a
> > new
> > > > >> broker
> > > > >> >> > > comes
> > > > >> >> > > > up
> > > > >> >> > > > > > it
> > > > >> >> > > > > > > > will
> > > > >> >> > > > > > > > > >> have
> > > > >> >> > > > > > > > > >> > > to
> > > > >> >> > > > > > > > > >> > > > >> get all
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > > >> controller.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
> > > > >> catastrophic
> > > > >> >> > > > > failures
> > > > >> >> > > > > > > > where
> > > > >> >> > > > > > > > > >> all
> > > > >> >> > > > > > > > > >> > > > brokers
> > > > >> >> > > > > > > > > >> > > > >> go
> > > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> lost
> > > even
> > > > >> if
> > > > >> >> > > servers
> > > > >> >> > > > > are
> > > > >> >> > > > > > > > > >> restarted as
> > > > >> >> > > > > > > > > >> > > > >> tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If
> this
> > > > >> happens,
> > > > >> >> > then
> > > > >> >> > > > > there
> > > > >> >> > > > > > > are
> > > > >> >> > > > > > > > > more
> > > > >> >> > > > > > > > > >> > > > important
> > > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe
> it
> > > is
> > > > >> better
> > > > >> >> > to
> > > > >> >> > > > > > > > > >> re-authenticate.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > (Alternative
> > > 2)
> > > > >> Do not
> > > > >> >> > > > > > distribute
> > > > >> >> > > > > > > > DT
> > > > >> >> > > > > > > > > to
> > > > >> >> > > > > > > > > >> > > other
> > > > >> >> > > > > > > > > >> > > > >> brokers
> > > > >> >> > > > > > > > > >> > > > >> >> > at
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > Client
> > > > >> >> > > authenticates
> > > > >> >> > > > > > with a
> > > > >> >> > > > > > > > > >> broker.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> Once
> > a
> > > > >> client is
> > > > >> >> > > > > > > > authenticated,
> > > > >> >> > > > > > > > > >> it
> > > > >> >> > > > > > > > > >> > > will
> > > > >> >> > > > > > > > > >> > > > >> make a
> > > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue
> a
> > > > >> delegation
> > > > >> >> > > > token.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > > > broker
> > > > >> >> > > generates
> > > > >> >> > > > DT
> > > > >> >> > > > > > of
> > > > >> >> > > > > > > > > form,
> > > > >> >> > > > > > > > > >> > > [hmac +
> > > > >> >> > > > > > > > > >> > > > >> (owner,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > maxLifeTime,
> > > > >> id,
> > > > >> >> > hmac,
> > > > >> >> > > > > > > > > >> expirationTime)]
> > > > >> >> > > > > > > > > >> > > and
> > > > >> >> > > > > > > > > >> > > > >> passes
> > > > >> >> > > > > > > > > >> > > > >> >> > back
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> this
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac
> is
> > > > >> generated
> > > > >> >> > via
> > > > >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> > > > >> >> > > > > > > > > >> > > > >> renewer,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> expirationTime)
> > > > >> >> > using
> > > > >> >> > > > > > > > SecretKey}.
> > > > >> >> > > > > > > > > >> Note
> > > > >> >> > > > > > > > > >> > > that
> > > > >> >> > > > > > > > > >> > > > >> all
> > > > >> >> > > > > > > > > >> > > > >> >> > brokers
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> have
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > Client
> > > > >> then goes
> > > > >> >> > to
> > > > >> >> > > > any
> > > > >> >> > > > > > > > broker
> > > > >> >> > > > > > > > > >> and to
> > > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
> > > > >> recalculates
> > > > >> >> > > hmac
> > > > >> >> > > > > > using
> > > > >> >> > > > > > > > > >> (owner,
> > > > >> >> > > > > > > > > >> > > > >> renewer,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> expirationTime) info
> > > > >> >> > > > from
> > > > >> >> > > > > DT
> > > > >> >> > > > > > > and
> > > > >> >> > > > > > > > > its
> > > > >> >> > > > > > > > > >> > > > >> SecretKey. If
> > > > >> >> > > > > > > > > >> > > > >> >> > it
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT,
> > client
> > > > is
> > > > >> >> > > > > authenticated.
> > > > >> >> > > > > > > > Yes,
> > > > >> >> > > > > > > > > >> it will
> > > > >> >> > > > > > > > > >> > > > do
> > > > >> >> > > > > > > > > >> > > > >> other
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > timestamp
> > > > >> expiry and
> > > > >> >> > > > such.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that
> > > > secret
> > > > >> key
> > > > >> >> > will
> > > > >> >> > > > be
> > > > >> >> > > > > > > > > generated
> > > > >> >> > > > > > > > > >> by
> > > > >> >> > > > > > > > > >> > > > >> controller
> > > > >> >> > > > > > > > > >> > > > >> >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > periodically.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > > issues
> > > > >> and
> > > > >> >> > fixes
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How
> > to
> > > > >> delete a
> > > > >> >> > DT?
> > > > >> >> > > > > Yes,
> > > > >> >> > > > > > > that
> > > > >> >> > > > > > > > > is
> > > > >> >> > > > > > > > > >> a
> > > > >> >> > > > > > > > > >> > > > downside
> > > > >> >> > > > > > > > > >> > > > >> >> > here.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> > handled
> > > > >> with
> > > > >> >> > > brokers
> > > > >> >> > > > > > > > > maintaining
> > > > >> >> > > > > > > > > >> a
> > > > >> >> > > > > > > > > >> > > > >> blacklist of
> > > > >> >> > > > > > > > > >> > > > >> >> > DTs,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be
> > > > >> removed after
> > > > >> >> > > > > expiry.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
> > > > >> catastrophic
> > > > >> >> > > > > failures
> > > > >> >> > > > > > > > where
> > > > >> >> > > > > > > > > >> all
> > > > >> >> > > > > > > > > >> > > > brokers
> > > > >> >> > > > > > > > > >> > > > >> go
> > > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> lost
> > > even
> > > > >> if
> > > > >> >> > > servers
> > > > >> >> > > > > are
> > > > >> >> > > > > > > > > >> restarted as
> > > > >> >> > > > > > > > > >> > > > >> tokens
> > > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If
> this
> > > > >> happens,
> > > > >> >> > then
> > > > >> >> > > > > there
> > > > >> >> > > > > > > are
> > > > >> >> > > > > > > > > more
> > > > >> >> > > > > > > > > >> > > > important
> > > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe
> it
> > > is
> > > > >> better
> > > > >> >> > to
> > > > >> >> > > > > > > > > >> re-authenticate.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri,
> Feb
> > > 26,
> > > > >> 2016 at
> > > > >> >> > > > 1:58
> > > > >> >> > > > > > PM,
> > > > >> >> > > > > > > > > Parth
> > > > >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > pbrahmbhatt@hortonworks.com>
> > > > >> >> > > > > > > > wrote:
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have
> > filed
> > > > >> KIP-48 so
> > > > >> >> > > we
> > > > >> >> > > > > can
> > > > >> >> > > > > > > > offer
> > > > >> >> > > > > > > > > >> hadoop
> > > > >> >> > > > > > > > > >> > > > like
> > > > >> >> > > > > > > > > >> > > > >> >> > delegation
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka.
> You
> > > can
> > > > >> review
> > > > >> >> > > the
> > > > >> >> > > > > > design
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
> > > > >> depends on
> > > > >> >> > > KIP-43
> > > > >> >> > > > > and
> > > > >> >> > > > > > > we
> > > > >> >> > > > > > > > > >> have also
> > > > >> >> > > > > > > > > >> > > > >> >> > discussed an
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed
> > > > design
> > > > >> here<
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > > >> >> > > > > > > > > >> > > > >> >> >
> > > > >> >> > > > > > > > > >> > > > >>
> > > > >> >> > > > > > > > > >> > > >
> > > > >> >> > > > > > > > > >> > >
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > > >>
> > > > >> >> > > > > > > > > >
> > > > >> >> > > > > > > > >
> > > > >> >> > > > > > > >
> > > > >> >> > > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > --
> > > > >> >> > > > > > Regards,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Rajini
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > > > --
> > > > >> >> > > > Regards,
> > > > >> >> > > >
> > > > >> >> > > > Rajini
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> Liquan Pei
> > > > >> >> Software Engineer, Confluent Inc
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by parth brahmbhatt <br...@gmail.com>.
Hi,
I am suggesting that we will only allow the renewal by users that
authenticated using *non* delegation token mechanism. For example, If user
Alice authenticated using kerberos and requested delegation tokens, only
user Alice authenticated via non delegation token mechanism can renew.
Clients that have access to delegation tokens can not issue renewal
request for renewing their own token and this is primarily important to
reduce the time window for which a compromised token will be valid.
To clarify, Yes any authenticated user can request delegation tokens but
even here I would recommend to avoid creating a chain where a client
authenticated via delegation token request for more delegation tokens.
Basically anyone can request delegation token, as long as they authenticate
via a non delegation token mechanism.
Aren't classes listed here
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-PublicInterfaces>
sufficient?
Thanks
Parth
On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <ju...@confluent.io> wrote:
> Parth,
>
> Thanks for the reply. A couple of comments inline below.
>
> On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
> brahmbhatt.parth@gmail.com> wrote:
>
> > 1. Who / how are tokens renewed? By original requester only? or using
> > Kerberos
> > auth only?
> > My recommendation is to do this only using Kerberos auth and only threw
> the
> > renewer specified during the acquisition request.
> >
> >
> Hmm, not sure that I follow this. Are you saying that any client
> authenticated with the delegation token can renew, i.e. there is no renewer
> needed?
>
> Also, just to be clear, any authenticated client (either through SASL or
> SSL) can request a delegation token for the authenticated user, right?
>
>
> > 2. Are tokens stored on each broker or in ZK?
> > My recommendation is still to store in ZK or not store them at all. The
> > whole controller based distribution is too much overhead with not much to
> > achieve.
> >
> > 3. How are tokens invalidated / expired?
> > Either by expiration time out or through an explicit request to
> invalidate.
> >
> > 4. Which encryption algorithm is used?
> > SCRAM
> >
> > 5. What is the impersonation proposal (it wasn't in the KIP but was
> > discussed
> > in this thread)?
> > There is no imperonation proposal. I tried and explained how its a
> > different problem and why its not really necessary to discuss that as
> part
> > of this KIP. This KIP will not support any impersonation, it will just
> be
> > another way to authenticate.
> >
> > 6. Do we need new ACLs, if so - for what actions?
> > We do not need new ACLs.
> >
> >
> Could we document the format of the new request/response and their
> associated Resource and Operation for ACL?
>
>
> > 7. How would the delegation token be configured in the client?
> > Should be through config. I wasn't planning on supporting JAAS for
> tokens.
> > I don't believe hadoop does this either.
> >
> > Thanks
> > Parth
> >
> >
> >
> > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Harsha,
> > >
> > > Another question.
> > >
> > > 9. How would the delegation token be configured in the client? The
> > standard
> > > way is to do this through JAAS. However, we will need to think through
> if
> > > this is convenient in a shared environment. For example, when a new
> task
> > is
> > > added to a Storm worker node, do we need to dynamically add a new
> section
> > > in the JAAS file? It may be more convenient if we can pass in the token
> > > through the config directly w/o going through JAAS.
> > >
> > > Are you or Parth still actively working on this KIP?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > >
> > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Just to add on that list.
> > > >
> > > > 2. It would be good to document the format of the data stored in ZK.
> > > > 7. Earlier, there was a discussion on whether the tokens should be
> > > > propagated through ZK like config/acl/quota, or through the
> controller.
> > > > Currently, the controller is only designed for propagating topic
> > > metadata,
> > > > but not other data.
> > > > 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since
> > it's
> > > > deprecated?
> > > >
> > > > Also, the images in the wiki seem broken.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io>
> > > wrote:
> > > >
> > > >> From what I can see, remaining questions are:
> > > >>
> > > >> 1. Who / how are tokens renewed? By original requester only? or
> using
> > > >> Kerberos auth only?
> > > >> 2. Are tokens stored on each broker or in ZK?
> > > >> 3. How are tokens invalidated / expired?
> > > >> 4. Which encryption algorithm is used?
> > > >> 5. What is the impersonation proposal (it wasn't in the KIP but was
> > > >> discussed in this thread)?
> > > >> 6. Do we need new ACLs, if so - for what actions?
> > > >>
> > > >> Gwen
> > > >>
> > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> > > >> > Jun & Ismael,
> > > >> > Unfortunately I couldn't attend the KIP
> > > meeting
> > > >> > when delegation tokens discussed.
> > Appreciate
> > > if
> > > >> > you can update the thread if you have any
> > > >> > further questions.
> > > >> > Thanks,
> > > >> > Harsha
> > > >> >
> > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > > >> >> It seems that the links to images in the KIP are broken.
> > > >> >>
> > > >> >> Liquan
> > > >> >>
> > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > > >> >> brahmbhatt.parth@gmail.com> wrote:
> > > >> >>
> > > >> >> > 110. What does getDelegationTokenAs mean?
> > > >> >> > In the current proposal we only allow a user to get delegation
> > > token
> > > >> for
> > > >> >> > the identity that it authenticated as using another mechanism,
> > i.e.
> > > >> A user
> > > >> >> > that authenticate using a keytab for principal
> user1@EXAMPLE.COM
> > > >> will get
> > > >> >> > delegation tokens for that user only. In future I think we will
> > > have
> > > >> to
> > > >> >> > extend support such that we allow some set of users (
> > > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to
> > acquire
> > > >> >> > delegation tokens on behalf of other users whose identity they
> > have
> > > >> >> > verified independently. Kafka brokers will have ACLs to
> control
> > > >> which
> > > >> >> > users are allowed to impersonate other users and get tokens on
> > > >> behalf of
> > > >> >> > them. Overall Impersonation is a whole different problem in my
> > > >> opinion and
> > > >> >> > I think we can tackle it in separate KIP.
> > > >> >> >
> > > >> >> > 111. What's the typical rate of getting and renewing delegation
> > > >> tokens?
> > > >> >> > Typically this should be very very low, 1 request per minute
> is a
> > > >> >> > relatively high estimate. However it depends on the token
> > > >> expiration. I am
> > > >> >> > less worried about the extra load it puts on controller vs the
> > > added
> > > >> >> > complexity and the value it offers.
> > > >> >> >
> > > >> >> > Thanks
> > > >> >> > Parth
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <
> ismael@juma.me.uk>
> > > >> wrote:
> > > >> >> >
> > > >> >> > > Thanks Rajini. It would probably require a separate KIP as it
> > > will
> > > >> >> > > introduce user visible changes. We could also update KIP-48
> to
> > > >> have this
> > > >> >> > > information, but it seems cleaner to do it separately. We can
> > > >> discuss
> > > >> >> > that
> > > >> >> > > in the KIP call today.
> > > >> >> > >
> > > >> >> > > Ismael
> > > >> >> > >
> > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > > >> >> > > rajinisivaram@googlemail.com> wrote:
> > > >> >> > >
> > > >> >> > > > Ismael,
> > > >> >> > > >
> > > >> >> > > > I have created a JIRA (
> > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that need
> another
> > > >> KIP? If
> > > >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA
> that
> > > gets
> > > >> >> > > reviewed
> > > >> >> > > > when the PR is ready?
> > > >> >> > > >
> > > >> >> > > > Thank you,
> > > >> >> > > >
> > > >> >> > > > Rajini
> > > >> >> > > >
> > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > > ismael@juma.me.uk>
> > > >> >> > wrote:
> > > >> >> > > >
> > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > > >> >> > > > >
> > > >> >> > > > > Gwen had independently mentioned this as a SASL mechanism
> > > that
> > > >> might
> > > >> >> > be
> > > >> >> > > > > useful for Kafka and I have been meaning to check it in
> > more
> > > >> detail.
> > > >> >> > > Good
> > > >> >> > > > > to know that you are willing to contribute an
> > implementation.
> > > >> Maybe
> > > >> >> > we
> > > >> >> > > > > should file a separate JIRA for this?
> > > >> >> > > > >
> > > >> >> > > > > Ismael
> > > >> >> > > > >
> > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > > >> >> > > > >
> > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> > Mechanism)
> > > >> is a
> > > >> >> > > better
> > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> > > built-in
> > > >> SCRAM
> > > >> >> > > > > > SaslServer or SaslClient, but I will be happy to add
> > > support
> > > >> in
> > > >> >> > Kafka
> > > >> >> > > > > since
> > > >> >> > > > > > it would be a useful mechanism to support anyway.
> > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the
> > protocol
> > > >> for
> > > >> >> > > > > > SCRAM-SHA-256.
> > > >> >> > > > > >
> > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> > jun@confluent.io
> > > >
> > > >> wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > > Parth,
> > > >> >> > > > > > >
> > > >> >> > > > > > > Thanks for the explanation. A couple of more
> questions.
> > > >> >> > > > > > >
> > > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > > >> >> > > > > > >
> > > >> >> > > > > > > 111. What's the typical rate of getting and renewing
> > > >> delegation
> > > >> >> > > > tokens?
> > > >> >> > > > > > > That may have an impact on whether they should be
> > > directed
> > > >> to the
> > > >> >> > > > > > > controller.
> > > >> >> > > > > > >
> > > >> >> > > > > > > Jun
> > > >> >> > > > > > >
> > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> > > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > > >> >> > > > > > >
> > > >> >> > > > > > > > Hi Jun,
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > Thanks for reviewing.
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > * We could add a Cluster action to add acls on who
> > can
> > > >> request
> > > >> >> > > > > > delegation
> > > >> >> > > > > > > > tokens. I don't see the use case for that yet but
> > down
> > > >> the line
> > > >> >> > > > when
> > > >> >> > > > > we
> > > >> >> > > > > > > > start supporting getDelegationTokenAs it will be
> > > >> necessary.
> > > >> >> > > > > > > > * Yes we recommend tokens to be only
> used/distributed
> > > >> over
> > > >> >> > secure
> > > >> >> > > > > > > channels.
> > > >> >> > > > > > > > * Depending on what design we end up choosing
> > > >> Invalidation will
> > > >> >> > > be
> > > >> >> > > > > > > > responsibility of every broker or controller.
> > > >> >> > > > > > > > * I am not sure if I documented somewhere that
> > > >> invalidation
> > > >> >> > will
> > > >> >> > > > > > directly
> > > >> >> > > > > > > > go through zookeeper but that is not the intent.
> > > >> Invalidation
> > > >> >> > > will
> > > >> >> > > > > > either
> > > >> >> > > > > > > > be request based or due to expiration. No direct
> > > >> zookeeper
> > > >> >> > > > > interaction
> > > >> >> > > > > > > from
> > > >> >> > > > > > > > any client.
> > > >> >> > > > > > > > * "Broker also stores the DelegationToken without
> the
> > > >> hmac in
> > > >> >> > the
> > > >> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
> > > >> purpose of
> > > >> >> > > > > zookeeper
> > > >> >> > > > > > in
> > > >> >> > > > > > > > this design is as distribution channel for tokens
> > > >> between all
> > > >> >> > > > brokers
> > > >> >> > > > > > > and a
> > > >> >> > > > > > > > layer that ensures only tokens that were generated
> by
> > > >> making a
> > > >> >> > > > > request
> > > >> >> > > > > > > to a
> > > >> >> > > > > > > > broker will be accepted (more on this in second
> > > >> paragraph). The
> > > >> >> > > > token
> > > >> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> > > >> expiration,
> > > >> >> > > hmac)
> > > >> >> > > > ,
> > > >> >> > > > > > one
> > > >> >> > > > > > > of
> > > >> >> > > > > > > > which is the finally generated hmac but hmac it
> self
> > is
> > > >> >> > derivable
> > > >> >> > > > if
> > > >> >> > > > > > you
> > > >> >> > > > > > > > have all the other elements of the token + secret
> key
> > > to
> > > >> >> > generate
> > > >> >> > > > > hmac.
> > > >> >> > > > > > > > Given zookeeper does not provide SSL support we do
> > not
> > > >> want the
> > > >> >> > > > > entire
> > > >> >> > > > > > > > token to be wire transferred to zookeeper as that
> > will
> > > >> be an
> > > >> >> > > > insecure
> > > >> >> > > > > > > wire
> > > >> >> > > > > > > > transfer. Instead we only store all the other
> > elements
> > > >> of a
> > > >> >> > > > > delegation
> > > >> >> > > > > > > > tokens. Brokers can read these elements and because
> > > they
> > > >> also
> > > >> >> > > have
> > > >> >> > > > > > access
> > > >> >> > > > > > > > to secret key they will be able to generate hmac on
> > > >> their end.
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > One of the alternative proposed is to avoid
> zookeeper
> > > >> >> > > altogether. A
> > > >> >> > > > > > > Client
> > > >> >> > > > > > > > will call broker with required information (owner,
> > > >> renwer,
> > > >> >> > > > > expiration)
> > > >> >> > > > > > > and
> > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't store
> this
> > > in
> > > >> >> > > zookeeper.
> > > >> >> > > > > > From
> > > >> >> > > > > > > > this point a client can contact any broker with all
> > the
> > > >> >> > > delegation
> > > >> >> > > > > > token
> > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the
> > borker
> > > >> will
> > > >> >> > > > > regenerate
> > > >> >> > > > > > > the
> > > >> >> > > > > > > > hmac and as long as it matches with hmac presented
> by
> > > >> client ,
> > > >> >> > > > broker
> > > >> >> > > > > > > will
> > > >> >> > > > > > > > allow the request to authenticate. Only problem
> with
> > > >> this
> > > >> >> > > approach
> > > >> >> > > > > is
> > > >> >> > > > > > if
> > > >> >> > > > > > > > the secret key is compromised any client can now
> > > generate
> > > >> >> > random
> > > >> >> > > > > tokens
> > > >> >> > > > > > > and
> > > >> >> > > > > > > > they will still be able to authenticate as any user
> > > they
> > > >> like.
> > > >> >> > > with
> > > >> >> > > > > > > > zookeeper we guarantee that only tokens acquired
> via
> > a
> > > >> broker
> > > >> >> > > > (using
> > > >> >> > > > > > some
> > > >> >> > > > > > > > auth scheme other than delegation token) will be
> > > >> accepted. We
> > > >> >> > > need
> > > >> >> > > > to
> > > >> >> > > > > > > > discuss which proposal makes more sense and we can
> go
> > > >> over it
> > > >> >> > in
> > > >> >> > > > > > > tomorrow's
> > > >> >> > > > > > > > meeting.
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > Also, can you forward the invite to me?
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > Thanks
> > > >> >> > > > > > > > Parth
> > > >> >> > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > > >> jun@confluent.io>
> > > >> >> > > > wrote:
> > > >> >> > > > > > > >
> > > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 100. This potentially can be useful for Kafka
> > Connect
> > > >> and
> > > >> >> > Kafka
> > > >> >> > > > > rest
> > > >> >> > > > > > > > proxy
> > > >> >> > > > > > > > > where a worker agent will need to run a task on
> > > behalf
> > > >> of a
> > > >> >> > > > client.
> > > >> >> > > > > > We
> > > >> >> > > > > > > > will
> > > >> >> > > > > > > > > likely need to change how those services use
> Kafka
> > > >> clients
> > > >> >> > > > > > > > > (producer/consumer). Instead of a shared client
> per
> > > >> worker,
> > > >> >> > we
> > > >> >> > > > will
> > > >> >> > > > > > > need
> > > >> >> > > > > > > > a
> > > >> >> > > > > > > > > client per user task since the authentication
> > happens
> > > >> at the
> > > >> >> > > > > > connection
> > > >> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
> > > >> workers.
> > > >> >> > So,
> > > >> >> > > we
> > > >> >> > > > > > > > probably
> > > >> >> > > > > > > > > need to allow multiple renewers. For Kafka rest
> > > proxy,
> > > >> the
> > > >> >> > > > renewer
> > > >> >> > > > > > can
> > > >> >> > > > > > > > > probably just be the creator of the token.
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 101. Do we need new acl on who can request
> > delegation
> > > >> tokens?
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 102. Do we recommend people to send delegation
> > tokens
> > > >> in an
> > > >> >> > > > > encrypted
> > > >> >> > > > > > > > > channel?
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 103. Who is responsible for expiring tokens,
> every
> > > >> broker?
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 104. For invalidating tokens, would it be better
> to
> > > do
> > > >> it in
> > > >> >> > a
> > > >> >> > > > > > request
> > > >> >> > > > > > > > > instead of going to ZK directly?
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 105. The terminology of client in the wiki
> > sometimes
> > > >> refers
> > > >> >> > to
> > > >> >> > > > the
> > > >> >> > > > > > end
> > > >> >> > > > > > > > > client and some other times refers to the client
> > > using
> > > >> the
> > > >> >> > > > > delegation
> > > >> >> > > > > > > > > tokens. It would be useful to distinguish between
> > the
> > > >> two.
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > 106. Could you explain the sentence "Broker also
> > > >> stores the
> > > >> >> > > > > > > > DelegationToken
> > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
> > > >> thought the
> > > >> >> > > > > > > delegation
> > > >> >> > > > > > > > > token is the hmac.
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > Thanks,
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > Jun
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> > > >> jun@confluent.io>
> > > >> >> > > > wrote:
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > > Hi, Harsha,
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can
> > discuss
> > > >> this in
> > > >> >> > > the
> > > >> >> > > > > > > meeting
> > > >> >> > > > > > > > > > tomorrow.
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > Thanks,
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > Jun
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> > > >> kafka@harsha.io>
> > > >> >> > > > wrote:
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > > >> Hi All,
> > > >> >> > > > > > > > > >> Can we have a KIP meeting around
> > this.
> > > >> The KIP
> > > >> >> > is
> > > >> >> > > > up
> > > >> >> > > > > > for
> > > >> >> > > > > > > > > >> sometime and if there are any
> > questions
> > > >> lets
> > > >> >> > > > quickly
> > > >> >> > > > > > hash
> > > >> >> > > > > > > > out
> > > >> >> > > > > > > > > >> details.
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > > >> Thanks,
> > > >> >> > > > > > > > > >> Harsha
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> > > brahmbhatt
> > > >> wrote:
> > > >> >> > > > > > > > > >> > That is what the hadoop echo system uses so
> no
> > > >> good
> > > >> >> > reason
> > > >> >> > > > > > really.
> > > >> >> > > > > > > > We
> > > >> >> > > > > > > > > >> > could
> > > >> >> > > > > > > > > >> > change it to whatever is the newest
> > recommended
> > > >> standard
> > > >> >> > > is.
> > > >> >> > > > > > > > > >> >
> > > >> >> > > > > > > > > >> > Thanks
> > > >> >> > > > > > > > > >> > Parth
> > > >> >> > > > > > > > > >> >
> > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael
> Juma <
> > > >> >> > > > > ismael@juma.me.uk
> > > >> >> > > > > > >
> > > >> >> > > > > > > > > wrote:
> > > >> >> > > > > > > > > >> >
> > > >> >> > > > > > > > > >> > > Hi Parth,
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started
> reviewing
> > > >> this and
> > > >> >> > > may
> > > >> >> > > > > have
> > > >> >> > > > > > > > > >> additional
> > > >> >> > > > > > > > > >> > > questions later. The immediate question
> that
> > > >> came to
> > > >> >> > > mind
> > > >> >> > > > is
> > > >> >> > > > > > our
> > > >> >> > > > > > > > > >> choice of
> > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
> > > >> OBSOLETE in
> > > >> >> > the
> > > >> >> > > > IANA
> > > >> >> > > > > > > > > Registry
> > > >> >> > > > > > > > > >> of
> > > >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC
> (2831)
> > > has
> > > >> been
> > > >> >> > > moved
> > > >> >> > > > > to
> > > >> >> > > > > > > > > Historic
> > > >> >> > > > > > > > > >> > > status:
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > >
> > > >> >> > >
> > > >>
> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > What is the reasoning behind that choice?
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > Thanks,
> > > >> >> > > > > > > > > >> > > Ismael
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> > > Shapira <
> > > >> >> > > > > > > gwen@confluent.io
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > >> wrote:
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >> > > > Also comments inline :)
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > > * I want to emphasize that even though
> > > >> delegation
> > > >> >> > > > tokens
> > > >> >> > > > > > > are a
> > > >> >> > > > > > > > > >> Hadoop
> > > >> >> > > > > > > > > >> > > > > innovation, I feel very strongly about
> > not
> > > >> adding
> > > >> >> > > > > > dependency
> > > >> >> > > > > > > > on
> > > >> >> > > > > > > > > >> Hadoop
> > > >> >> > > > > > > > > >> > > > > when implementing delegation tokens
> for
> > > >> Kafka. The
> > > >> >> > > KIP
> > > >> >> > > > > > > doesn't
> > > >> >> > > > > > > > > >> imply
> > > >> >> > > > > > > > > >> > > > > such dependency, but if you can
> > clarify...
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one
> > will
> > > >> read
> > > >> >> > the
> > > >> >> > > > KIP
> > > >> >> > > > > > and
> > > >> >> > > > > > > > > panic
> > > >> >> > > > > > > > > >> > > > three weeks before the next release...
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > > * Can we get delegation token at any
> > time
> > > >> after
> > > >> >> > > > > > > > authenticating?
> > > >> >> > > > > > > > > >> only
> > > >> >> > > > > > > > > >> > > > > immediately after?
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *As long as you are authenticated you
> > can
> > > >> get
> > > >> >> > > > delegation
> > > >> >> > > > > > > > tokens.
> > > >> >> > > > > > > > > >> We
> > > >> >> > > > > > > > > >> > > need
> > > >> >> > > > > > > > > >> > > > to
> > > >> >> > > > > > > > > >> > > > > discuss if a client authenticated
> using
> > > >> delegation
> > > >> >> > > > > token,
> > > >> >> > > > > > > can
> > > >> >> > > > > > > > > also
> > > >> >> > > > > > > > > >> > > > acquire
> > > >> >> > > > > > > > > >> > > > > delegation token again or not. Also
> > there
> > > >> is the
> > > >> >> > > > > question
> > > >> >> > > > > > of
> > > >> >> > > > > > > > do
> > > >> >> > > > > > > > > we
> > > >> >> > > > > > > > > >> > > allow
> > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation token or
> we
> > > >> want
> > > >> >> > > specific
> > > >> >> > > > > > ACLs
> > > >> >> > > > > > > (I
> > > >> >> > > > > > > > > >> think
> > > >> >> > > > > > > > > >> > > its
> > > >> >> > > > > > > > > >> > > > an
> > > >> >> > > > > > > > > >> > > > > overkill.)*
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > I think we are debating two options:
> > Either
> > > >> require
> > > >> >> > > > > Kerberos
> > > >> >> > > > > > > > auth
> > > >> >> > > > > > > > > >> for
> > > >> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
> > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> > > basically
> > > >> >> > require
> > > >> >> > > a
> > > >> >> > > > > "job
> > > >> >> > > > > > > > > master"
> > > >> >> > > > > > > > > >> > > > to take responsibility for the renewal,
> it
> > > >> will have
> > > >> >> > > its
> > > >> >> > > > > own
> > > >> >> > > > > > > > > >> identity
> > > >> >> > > > > > > > > >> > > > anyway and I think this is the correct
> > > design
> > > >> >> > pattern
> > > >> >> > > > > > anyway.
> > > >> >> > > > > > > > For
> > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> > > >> renewals?),
> > > >> >> > but
> > > >> >> > > > it
> > > >> >> > > > > is
> > > >> >> > > > > > > > hard
> > > >> >> > > > > > > > > to
> > > >> >> > > > > > > > > >> > > > debate simplicity without looking at the
> > > code
> > > >> >> > changes
> > > >> >> > > > > > > required.
> > > >> >> > > > > > > > If
> > > >> >> > > > > > > > > >> you
> > > >> >> > > > > > > > > >> > > > have a draft of how the "require
> Kerberos"
> > > >> will look
> > > >> >> > > in
> > > >> >> > > > > > Kafka
> > > >> >> > > > > > > > > code,
> > > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > > * My understanding is that tokens will
> > > >> propagate
> > > >> >> > via
> > > >> >> > > > ZK
> > > >> >> > > > > > but
> > > >> >> > > > > > > > > >> without
> > > >> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> > > >> protocol,
> > > >> >> > > > correct?
> > > >> >> > > > > > > > Clients
> > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth
> > failure
> > > >> (IIRC),
> > > >> >> > > but
> > > >> >> > > > > > since
> > > >> >> > > > > > > > the
> > > >> >> > > > > > > > > >> > > > > tokens propagate between brokers
> asynch,
> > > we
> > > >> will
> > > >> >> > > need
> > > >> >> > > > to
> > > >> >> > > > > > > > retry a
> > > >> >> > > > > > > > > >> bit
> > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth due to
> > > timing
> > > >> >> > issues.
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right
> > > now.
> > > >> The
> > > >> >> > > > current
> > > >> >> > > > > > > > > documented
> > > >> >> > > > > > > > > >> > > > approach
> > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does not
> > require
> > > >> any
> > > >> >> > > changes
> > > >> >> > > > > to
> > > >> >> > > > > > > > > >> > > UpdateMetadata
> > > >> >> > > > > > > > > >> > > > > protocol. An alternative approach can
> > > remove
> > > >> >> > > zookeeper
> > > >> >> > > > > > > > > dependency
> > > >> >> > > > > > > > > >> as
> > > >> >> > > > > > > > > >> > > well
> > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> > discussion
> > > >> call.*
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to
> > > ping
> > > >> Jun to
> > > >> >> > > > > > arrange a
> > > >> >> > > > > > > > > call?
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of
> having
> > > >> just the
> > > >> >> > > > > > controller
> > > >> >> > > > > > > > > issue
> > > >> >> > > > > > > > > >> the
> > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a
> > > shared
> > > >> >> > secret.
> > > >> >> > > > Not
> > > >> >> > > > > > > sure
> > > >> >> > > > > > > > if
> > > >> >> > > > > > > > > >> we
> > > >> >> > > > > > > > > >> > > > > want to continue the discussion here
> or
> > on
> > > >> the
> > > >> >> > > wiki. I
> > > >> >> > > > > > think
> > > >> >> > > > > > > > > that
> > > >> >> > > > > > > > > >> we
> > > >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> > > >> distribution"
> > > >> >> > > from
> > > >> >> > > > > > > "shared
> > > >> >> > > > > > > > > >> secret
> > > >> >> > > > > > > > > >> > > > > distribution" and use the controller
> as
> > > the
> > > >> only
> > > >> >> > > token
> > > >> >> > > > > > > > generator
> > > >> >> > > > > > > > > >> to
> > > >> >> > > > > > > > > >> > > > > solve the second issue, while still
> > using
> > > >> ZK async
> > > >> >> > > to
> > > >> >> > > > > > > > distribute
> > > >> >> > > > > > > > > >> > > > > tokens.
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I
> am
> > > >> fine with
> > > >> >> > > > that
> > > >> >> > > > > > > > approach
> > > >> >> > > > > > > > > >> as
> > > >> >> > > > > > > > > >> > > long
> > > >> >> > > > > > > > > >> > > > as
> > > >> >> > > > > > > > > >> > > > > we agree that the extra complexity of
> > > >> >> > > adding/updating
> > > >> >> > > > > APIS
> > > >> >> > > > > > > > adds
> > > >> >> > > > > > > > > >> enough
> > > >> >> > > > > > > > > >> > > > > value. The advantage with the
> controller
> > > >> approach
> > > >> >> > is
> > > >> >> > > > > > secret
> > > >> >> > > > > > > > > >> rotation
> > > >> >> > > > > > > > > >> > > can
> > > >> >> > > > > > > > > >> > > > be
> > > >> >> > > > > > > > > >> > > > > automated,frequent and would not
> require
> > > >> >> > > deployment. *
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Can you detail the extra complexity (or
> > > point
> > > >> me to
> > > >> >> > > the
> > > >> >> > > > > > email
> > > >> >> > > > > > > I
> > > >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> > > >> >> > > > > > > > > >> > > > As far as I can tell, clients can
> already
> > > >> find the
> > > >> >> > > > > > controller
> > > >> >> > > > > > > > from
> > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
> > > >> controller
> > > >> >> > > > load.
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> > > kerberos
> > > >> auth
> > > >> >> > for
> > > >> >> > > > > > > renwal, I
> > > >> >> > > > > > > > > >> think
> > > >> >> > > > > > > > > >> > > > > it mixes the transport layer the the
> > > request
> > > >> >> > content
> > > >> >> > > > in
> > > >> >> > > > > a
> > > >> >> > > > > > > > pretty
> > > >> >> > > > > > > > > >> ugly
> > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to
> > non-owner
> > > >> is
> > > >> >> > > better.
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil.
> While
> > > >> this will
> > > >> >> > > make
> > > >> >> > > > > the
> > > >> >> > > > > > > > kafka
> > > >> >> > > > > > > > > >> code
> > > >> >> > > > > > > > > >> > > > > pretty straight forward , forcing
> > renewer
> > > >> to
> > > >> >> > > > non-owner
> > > >> >> > > > > > > pushes
> > > >> >> > > > > > > > > >> the code
> > > >> >> > > > > > > > > >> > > > > ugliness to client and makes it even
> > > harder
> > > >> to
> > > >> >> > > > > > integrate. *
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> > > >> "renewal by
> > > >> >> > > > other"
> > > >> >> > > > > > > > approach
> > > >> >> > > > > > > > > >> is
> > > >> >> > > > > > > > > >> > > > that ugly for the clients we expect to
> use
> > > >> >> > delegation
> > > >> >> > > > > tokens
> > > >> >> > > > > > > > since
> > > >> >> > > > > > > > > >> > > > they will have an app-master of some
> sort
> > > who
> > > >> >> > > requested
> > > >> >> > > > > the
> > > >> >> > > > > > > > token
> > > >> >> > > > > > > > > to
> > > >> >> > > > > > > > > >> > > > begin with.
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > The response for my question on how
> > > multiple
> > > >> >> > > > identities
> > > >> >> > > > > > will
> > > >> >> > > > > > > > be
> > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to me -
> > AFAIK,
> > > >> we don't
> > > >> >> > > > > > > > authenticate
> > > >> >> > > > > > > > > >> each
> > > >> >> > > > > > > > > >> > > > > request, we authenticate connections.
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > *We authenticate connections, and only
> > > when
> > > >> they
> > > >> >> > are
> > > >> >> > > > > being
> > > >> >> > > > > > > > > >> established.
> > > >> >> > > > > > > > > >> > > > Let
> > > >> >> > > > > > > > > >> > > > > me try to phrase this as a question,
> in
> > > >> absence of
> > > >> >> > > > > > > delegation
> > > >> >> > > > > > > > > >> tokens if
> > > >> >> > > > > > > > > >> > > > we
> > > >> >> > > > > > > > > >> > > > > had to support the use case using user
> > > >> TGT's how
> > > >> >> > > would
> > > >> >> > > > > we
> > > >> >> > > > > > do
> > > >> >> > > > > > > > it?
> > > >> >> > > > > > > > > >> My
> > > >> >> > > > > > > > > >> > > point
> > > >> >> > > > > > > > > >> > > > > was it would be no different with
> > > delegation
> > > >> >> > tokens.
> > > >> >> > > > The
> > > >> >> > > > > > use
> > > >> >> > > > > > > > > case
> > > >> >> > > > > > > > > >> you
> > > >> >> > > > > > > > > >> > > are
> > > >> >> > > > > > > > > >> > > > > describing seems more like
> > impersonation.*
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the things
> > that
> > > >> >> > delegation
> > > >> >> > > > > > tokens
> > > >> >> > > > > > > > > >> handled.
> > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > Gwen
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > > > > Thanks
> > > >> >> > > > > > > > > >> > > > > Parth
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
> > > >> Shapira <
> > > >> >> > > > > > > > > gwen@confluent.io
> > > >> >> > > > > > > > > >> >
> > > >> >> > > > > > > > > >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >
> > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> Few more comments:
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
> > > >> doesn't seem
> > > >> >> > to
> > > >> >> > > > > have
> > > >> >> > > > > > > good
> > > >> >> > > > > > > > > >> > > > >> description of the changes to the
> Kafka
> > > >> Protocol.
> > > >> >> > > > > Sounds
> > > >> >> > > > > > > like
> > > >> >> > > > > > > > > >> you are
> > > >> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest
> > and
> > > >> >> > > > > > RenewTokenRequest
> > > >> >> > > > > > > > (and
> > > >> >> > > > > > > > > >> > > > >> matching responses), without
> detailing
> > > the
> > > >> >> > contents
> > > >> >> > > > of
> > > >> >> > > > > > the
> > > >> >> > > > > > > > > >> requests
> > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you show
> the
> > > >> class
> > > >> >> > > > interface,
> > > >> >> > > > > > but
> > > >> >> > > > > > > > not
> > > >> >> > > > > > > > > >> the
> > > >> >> > > > > > > > > >> > > > >> underlying protocol. This could be
> seen
> > > as
> > > >> an
> > > >> >> > > > > > > implementation
> > > >> >> > > > > > > > > >> detail,
> > > >> >> > > > > > > > > >> > > > >> but since the binary protocol is what
> > we
> > > >> provide
> > > >> >> > to
> > > >> >> > > > > > > non-Java
> > > >> >> > > > > > > > > >> clients,
> > > >> >> > > > > > > > > >> > > > >> we need to show the changes there.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> > > >> >> > > > > > > > delegationTokenRequestHandler?
> > > >> >> > > > > > > > > >> Is it
> > > >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or
> > > Client?
> > > >> Its
> > > >> >> > > > > unclear...
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even
> though
> > > >> delegation
> > > >> >> > > > > tokens
> > > >> >> > > > > > > are
> > > >> >> > > > > > > > a
> > > >> >> > > > > > > > > >> Hadoop
> > > >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly
> about
> > > not
> > > >> adding
> > > >> >> > > > > > > dependency
> > > >> >> > > > > > > > on
> > > >> >> > > > > > > > > >> Hadoop
> > > >> >> > > > > > > > > >> > > > >> when implementing delegation tokens
> for
> > > >> Kafka.
> > > >> >> > The
> > > >> >> > > > KIP
> > > >> >> > > > > > > > doesn't
> > > >> >> > > > > > > > > >> imply
> > > >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> > > clarify...
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token at any
> > time
> > > >> after
> > > >> >> > > > > > > > authenticating?
> > > >> >> > > > > > > > > >> only
> > > >> >> > > > > > > > > >> > > > >> immediately after?
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * My understanding is that tokens
> will
> > > >> propagate
> > > >> >> > > via
> > > >> >> > > > ZK
> > > >> >> > > > > > but
> > > >> >> > > > > > > > > >> without
> > > >> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
> > > >> protocol,
> > > >> >> > > > correct?
> > > >> >> > > > > > > > Clients
> > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth
> > > failure
> > > >> >> > (IIRC),
> > > >> >> > > > but
> > > >> >> > > > > > > since
> > > >> >> > > > > > > > > the
> > > >> >> > > > > > > > > >> > > > >> tokens propagate between brokers
> > asynch,
> > > >> we will
> > > >> >> > > need
> > > >> >> > > > > to
> > > >> >> > > > > > > > retry
> > > >> >> > > > > > > > > a
> > > >> >> > > > > > > > > >> bit
> > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to
> > > timing
> > > >> >> > issues.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
> > > >> touching ZK
> > > >> >> > > > directly
> > > >> >> > > > > > :)
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of
> having
> > > >> just the
> > > >> >> > > > > > controller
> > > >> >> > > > > > > > > >> issue the
> > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a
> > > >> shared
> > > >> >> > > secret.
> > > >> >> > > > > Not
> > > >> >> > > > > > > sure
> > > >> >> > > > > > > > > if
> > > >> >> > > > > > > > > >> we
> > > >> >> > > > > > > > > >> > > > >> want to continue the discussion here
> or
> > > on
> > > >> the
> > > >> >> > > wiki.
> > > >> >> > > > I
> > > >> >> > > > > > > think
> > > >> >> > > > > > > > > >> that we
> > > >> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> > > >> distribution"
> > > >> >> > > from
> > > >> >> > > > > > > "shared
> > > >> >> > > > > > > > > >> secret
> > > >> >> > > > > > > > > >> > > > >> distribution" and use the controller
> as
> > > >> the only
> > > >> >> > > > token
> > > >> >> > > > > > > > > generator
> > > >> >> > > > > > > > > >> to
> > > >> >> > > > > > > > > >> > > > >> solve the second issue, while still
> > using
> > > >> ZK
> > > >> >> > async
> > > >> >> > > to
> > > >> >> > > > > > > > > distribute
> > > >> >> > > > > > > > > >> > > > >> tokens.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with
> infinite
> > > >> lifetime
> > > >> >> > of
> > > >> >> > > > > > tokens
> > > >> >> > > > > > > > (and
> > > >> >> > > > > > > > > >> hoped
> > > >> >> > > > > > > > > >> > > > >> to hear from others in the
> community) -
> > > but
> > > >> >> > having
> > > >> >> > > > the
> > > >> >> > > > > > > option
> > > >> >> > > > > > > > > and
> > > >> >> > > > > > > > > >> > > > >> documenting it as less secure, allows
> > > >> users to
> > > >> >> > > > > configure
> > > >> >> > > > > > > > their
> > > >> >> > > > > > > > > >> system
> > > >> >> > > > > > > > > >> > > > >> as the wish.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
> > > >> kerberos auth
> > > >> >> > > for
> > > >> >> > > > > > > renwal,
> > > >> >> > > > > > > > I
> > > >> >> > > > > > > > > >> think
> > > >> >> > > > > > > > > >> > > > >> it mixes the transport layer the the
> > > >> request
> > > >> >> > > content
> > > >> >> > > > > in a
> > > >> >> > > > > > > > > pretty
> > > >> >> > > > > > > > > >> ugly
> > > >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> > > non-owner
> > > >> is
> > > >> >> > > better.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * More detailed explanation on what
> we
> > > >> plan to do
> > > >> >> > > for
> > > >> >> > > > > the
> > > >> >> > > > > > > > java
> > > >> >> > > > > > > > > >> clients
> > > >> >> > > > > > > > > >> > > > >> specifically - new configuration? new
> > > APIs?
> > > >> >> > > > > > > > > >> > > > >> The response for my question on how
> > > >> multiple
> > > >> >> > > > identities
> > > >> >> > > > > > > will
> > > >> >> > > > > > > > be
> > > >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me -
> > AFAIK,
> > > >> we
> > > >> >> > don't
> > > >> >> > > > > > > > authenticate
> > > >> >> > > > > > > > > >> each
> > > >> >> > > > > > > > > >> > > > >> request, we authenticate connections.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are
> > > only
> > > >> used
> > > >> >> > in
> > > >> >> > > > the
> > > >> >> > > > > > > Hadoop
> > > >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> > > >> >> > alternatives
> > > >> >> > > in
> > > >> >> > > > > > other
> > > >> >> > > > > > > > > >> ecosystems
> > > >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and
> > whether
> > > >> there
> > > >> >> > are
> > > >> >> > > > some
> > > >> >> > > > > > > > > >> advantages
> > > >> >> > > > > > > > > >> > > > >> there.
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> Gwen
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM,
> > Harsha <
> > > >> >> > > > > kafka@harsha.io
> > > >> >> > > > > > >
> > > >> >> > > > > > > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> > > >> >> > > > > > > > > >> > > > >> > Can you look at Parth's
> > last
> > > >> reply.
> > > >> >> > > Does
> > > >> >> > > > > it
> > > >> >> > > > > > > > answer
> > > >> >> > > > > > > > > >> your
> > > >> >> > > > > > > > > >> > > > >> > concerns.
> > > >> >> > > > > > > > > >> > > > >> >
> > > >> >> > > > > > > > > >> > > > >> > Thanks,
> > > >> >> > > > > > > > > >> > > > >> > Harsha
> > > >> >> > > > > > > > > >> > > > >> >
> > > >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM,
> > parth
> > > >> >> > > brahmbhatt
> > > >> >> > > > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The
> wiki
> > > >> already
> > > >> >> > has
> > > >> >> > > > > > details
> > > >> >> > > > > > > on
> > > >> >> > > > > > > > > >> token
> > > >> >> > > > > > > > > >> > > > >> >> expiration
> > > >> >> > > > > > > > > >> > > > >> >> under token acquisition process
> > > >> >> > > > > > > > > >> > > > >> >> <
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> > > >> >> > > > > > > > > >> > > > >> >.
> > > >> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens
> will
> > > >> expire
> > > >> >> > > based
> > > >> >> > > > > on a
> > > >> >> > > > > > > > > server
> > > >> >> > > > > > > > > >> side
> > > >> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours)
> > > unless
> > > >> >> > renewed.
> > > >> >> > > > > > Renewal
> > > >> >> > > > > > > > is
> > > >> >> > > > > > > > > >> only
> > > >> >> > > > > > > > > >> > > > allowed
> > > >> >> > > > > > > > > >> > > > >> >> until the max life time of token.
> > > >> >> > Alternatively
> > > >> >> > > we
> > > >> >> > > > > > could
> > > >> >> > > > > > > > > also
> > > >> >> > > > > > > > > >> make
> > > >> >> > > > > > > > > >> > > > that
> > > >> >> > > > > > > > > >> > > > >> >> an
> > > >> >> > > > > > > > > >> > > > >> >> optional param and the server side
> > > >> default can
> > > >> >> > > > serve
> > > >> >> > > > > > as
> > > >> >> > > > > > > > the
> > > >> >> > > > > > > > > >> upper
> > > >> >> > > > > > > > > >> > > > bound.
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> To your second point it will be
> done
> > > >> exactly
> > > >> >> > the
> > > >> >> > > > > same
> > > >> >> > > > > > > way
> > > >> >> > > > > > > > we
> > > >> >> > > > > > > > > >> would
> > > >> >> > > > > > > > > >> > > > >> >> support
> > > >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling
> client
> > > >> will have
> > > >> >> > > to
> > > >> >> > > > > put
> > > >> >> > > > > > > the
> > > >> >> > > > > > > > > >> tokens it
> > > >> >> > > > > > > > > >> > > > >> wants
> > > >> >> > > > > > > > > >> > > > >> >> to use in the subject instance and
> > > call
> > > >> >> > > > > > produce/consume
> > > >> >> > > > > > > > > inside
> > > >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will
> have
> > to
> > > >> keep
> > > >> >> > > track
> > > >> >> > > > of
> > > >> >> > > > > > its
> > > >> >> > > > > > > > own
> > > >> >> > > > > > > > > >> > > > subject. I
> > > >> >> > > > > > > > > >> > > > >> >> will have to look at the code to
> see
> > > if
> > > >> we
> > > >> >> > > support
> > > >> >> > > > > > this
> > > >> >> > > > > > > > > >> feature
> > > >> >> > > > > > > > > >> > > right
> > > >> >> > > > > > > > > >> > > > >> now
> > > >> >> > > > > > > > > >> > > > >> >> but my understanding is delegation
> > > token
> > > >> >> > > shouldn't
> > > >> >> > > > > > need
> > > >> >> > > > > > > > any
> > > >> >> > > > > > > > > >> special
> > > >> >> > > > > > > > > >> > > > >> >> treatment as its just another type
> > of
> > > >> >> > Credential
> > > >> >> > > > in
> > > >> >> > > > > > the
> > > >> >> > > > > > > > > >> subject.
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> I would also like to know what is
> > your
> > > >> opinion
> > > >> >> > > > about
> > > >> >> > > > > > > > > infinite
> > > >> >> > > > > > > > > >> > > renewal
> > > >> >> > > > > > > > > >> > > > >> (my
> > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> > > this),
> > > >> tokens
> > > >> >> > > > > > renewing
> > > >> >> > > > > > > > them
> > > >> >> > > > > > > > > >> > > self(my
> > > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> > this)
> > > >> and
> > > >> >> > most
> > > >> >> > > > > > > > importantly
> > > >> >> > > > > > > > > >> your
> > > >> >> > > > > > > > > >> > > > choice
> > > >> >> > > > > > > > > >> > > > >> >> between the alternatives listed on
> > > this
> > > >> thread
> > > >> >> > > > > > > > > >> > > > >> >> <
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >>
> > >
> >
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> > > >> >> > > > > > > > > >> > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> > > >> alternative-2 minus
> > > >> >> > > > > > > controller
> > > >> >> > > > > > > > > >> > > > distributing
> > > >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for
> reviewing.
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> Thanks
> > > >> >> > > > > > > > > >> > > > >> >> Parth
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM,
> Gwen
> > > >> Shapira <
> > > >> >> > > > > > > > > >> gwen@confluent.io>
> > > >> >> > > > > > > > > >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> > Harsha,
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest
> Proxy.
> > I
> > > >> didn't
> > > >> >> > see
> > > >> >> > > > > your
> > > >> >> > > > > > > > design
> > > >> >> > > > > > > > > >> yet,
> > > >> >> > > > > > > > > >> > > > but in
> > > >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> > > >> producers, which
> > > >> >> > > > will
> > > >> >> > > > > > > serve
> > > >> >> > > > > > > > > >> multiple
> > > >> >> > > > > > > > > >> > > > users
> > > >> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since
> > these
> > > >> users
> > > >> >> > > will
> > > >> >> > > > > have
> > > >> >> > > > > > > > > >> different
> > > >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> > > >> authenticate
> > > >> >> > > > > separately,
> > > >> >> > > > > > > and
> > > >> >> > > > > > > > > >> can't
> > > >> >> > > > > > > > > >> > > > share a
> > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > Gwen
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM,
> > > >> Harsha <
> > > >> >> > > > > > > kafka@harsha.io
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > > > >> wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> > > >> >> > > > > > > > > >> > > > >> >> > > On your second
> point.
> > > >> Can you
> > > >> >> > > > > describe
> > > >> >> > > > > > a
> > > >> >> > > > > > > > > >> usecase
> > > >> >> > > > > > > > > >> > > where
> > > >> >> > > > > > > > > >> > > > >> >> > > mutliple clients
> > ended
> > > up
> > > >> >> > > sharing a
> > > >> >> > > > > > > > producer
> > > >> >> > > > > > > > > >> and
> > > >> >> > > > > > > > > >> > > even
> > > >> >> > > > > > > > > >> > > > if
> > > >> >> > > > > > > > > >> > > > >> they
> > > >> >> > > > > > > > > >> > > > >> >> > > do why can't they
> not
> > > use
> > > >> >> > single
> > > >> >> > > > > token
> > > >> >> > > > > > > that
> > > >> >> > > > > > > > > >> producer
> > > >> >> > > > > > > > > >> > > > >> >> > > captures. Why would
> > we
> > > >> need
> > > >> >> > > > multiple
> > > >> >> > > > > > > > clients
> > > >> >> > > > > > > > > >> with
> > > >> >> > > > > > > > > >> > > > >> different
> > > >> >> > > > > > > > > >> > > > >> >> > > tokens sharing a
> > single
> > > >> >> > instance
> > > >> >> > > of
> > > >> >> > > > > > > > producer.
> > > >> >> > > > > > > > > >> Also
> > > >> >> > > > > > > > > >> > > in
> > > >> >> > > > > > > > > >> > > > >> this
> > > >> >> > > > > > > > > >> > > > >> >> > > case other clients
> > have
> > > >> access
> > > >> >> > > all
> > > >> >> > > > > the
> > > >> >> > > > > > > > tokens
> > > >> >> > > > > > > > > >> no?
> > > >> >> > > > > > > > > >> > > > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> > > >> >> > > > > > > > > >> > > > >> >> > > Harsha
> > > >> >> > > > > > > > > >> > > > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49
> > AM,
> > > >> Gwen
> > > >> >> > > Shapira
> > > >> >> > > > > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't
> > see
> > > >> in the
> > > >> >> > > wiki:
> > > >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
> > > >> delegation
> > > >> >> > > > > tokens?
> > > >> >> > > > > > > > > >> Renewal? How
> > > >> >> > > > > > > > > >> > > > do we
> > > >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> > > >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use
> delegation
> > > >> tokens
> > > >> >> > for
> > > >> >> > > > > > "do-as"
> > > >> >> > > > > > > > > (say,
> > > >> >> > > > > > > > > >> > > submit
> > > >> >> > > > > > > > > >> > > > >> Storm
> > > >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will
> need a
> > > >> producer
> > > >> >> > > for
> > > >> >> > > > > > every
> > > >> >> > > > > > > > job
> > > >> >> > > > > > > > > >> (we
> > > >> >> > > > > > > > > >> > > can't
> > > >> >> > > > > > > > > >> > > > >> share
> > > >> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs
> > > running
> > > >> on
> > > >> >> > same
> > > >> >> > > > > node),
> > > >> >> > > > > > > > since
> > > >> >> > > > > > > > > >> we
> > > >> >> > > > > > > > > >> > > only
> > > >> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting.
> > Is
> > > >> there a
> > > >> >> > > plan
> > > >> >> > > > > to
> > > >> >> > > > > > > > change
> > > >> >> > > > > > > > > >> this
> > > >> >> > > > > > > > > >> > > for
> > > >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order
> to
> > > >> allow
> > > >> >> > > multiple
> > > >> >> > > > > > users
> > > >> >> > > > > > > > > with
> > > >> >> > > > > > > > > >> > > > different
> > > >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> > > >> >> > > > > > > > > >> > > > >> >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12
> AM,
> > > >> parth
> > > >> >> > > > > brahmbhatt
> > > >> >> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com>
> > > >> wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more
> > time,
> > > >> can
> > > >> >> > other
> > > >> >> > > > > > > committers
> > > >> >> > > > > > > > > >> review?
> > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> > > >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at
> 9:07
> > > AM,
> > > >> >> > Harsha <
> > > >> >> > > > > > > > > >> kafka@harsha.io>
> > > >> >> > > > > > > > > >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> Overall current
> > > >> design looks
> > > >> >> > > > good
> > > >> >> > > > > to
> > > >> >> > > > > > > > me. I
> > > >> >> > > > > > > > > >> am +1
> > > >> >> > > > > > > > > >> > > on
> > > >> >> > > > > > > > > >> > > > >> the
> > > >> >> > > > > > > > > >> > > > >> >> > KIP.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review
> > this
> > > >> as
> > > >> >> > well.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at
> > 09:57
> > > >> AM,
> > > >> >> > parth
> > > >> >> > > > > > > > brahmbhatt
> > > >> >> > > > > > > > > >> wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review
> > Jitendra.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
> > > >> infinite
> > > >> >> > > > lifetime
> > > >> >> > > > > > > but I
> > > >> >> > > > > > > > > >> see the
> > > >> >> > > > > > > > > >> > > > >> Streaming
> > > >> >> > > > > > > > > >> > > > >> >> > use
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming
> > use
> > > >> case I
> > > >> >> > > was
> > > >> >> > > > > > hoping
> > > >> >> > > > > > > > > >> there will
> > > >> >> > > > > > > > > >> > > > be
> > > >> >> > > > > > > > > >> > > > >> some
> > > >> >> > > > > > > > > >> > > > >> >> > notion
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can
> get
> > > new
> > > >> >> > > > delegation
> > > >> >> > > > > > > tokens
> > > >> >> > > > > > > > > at
> > > >> >> > > > > > > > > >> fixed
> > > >> >> > > > > > > > > >> > > > >> interval
> > > >> >> > > > > > > > > >> > > > >> >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers.
> If
> > > >> that is
> > > >> >> > not
> > > >> >> > > > the
> > > >> >> > > > > > case
> > > >> >> > > > > > > > for
> > > >> >> > > > > > > > > >> we can
> > > >> >> > > > > > > > > >> > > > >> discuss
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens
> renewing
> > > >> them self
> > > >> >> > > and
> > > >> >> > > > > the
> > > >> >> > > > > > > > > >> security
> > > >> >> > > > > > > > > >> > > > >> implications
> > > >> >> > > > > > > > > >> > > > >> >> > of the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients
> to
> > > >> fetch
> > > >> >> > tokens
> > > >> >> > > > > from
> > > >> >> > > > > > > > > >> zookeeper,
> > > >> >> > > > > > > > > >> > > > >> overall I
> > > >> >> > > > > > > > > >> > > > >> >> > think
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't
> > rely
> > > >> on our
> > > >> >> > > > > metadata
> > > >> >> > > > > > > > store
> > > >> >> > > > > > > > > >> and I
> > > >> >> > > > > > > > > >> > > > >> think we
> > > >> >> > > > > > > > > >> > > > >> >> > are
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction
> > > with
> > > >> all
> > > >> >> > the
> > > >> >> > > > > KIP-4
> > > >> >> > > > > > > > > >> improvements.
> > > >> >> > > > > > > > > >> > > > I
> > > >> >> > > > > > > > > >> > > > >> chose
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this
> case
> > > the
> > > >> client
> > > >> >> > > > will
> > > >> >> > > > > > > still
> > > >> >> > > > > > > > > >> just talk
> > > >> >> > > > > > > > > >> > > > to
> > > >> >> > > > > > > > > >> > > > >> >> > broker , its
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will
> use
> > > >> zookeeper
> > > >> >> > > > which
> > > >> >> > > > > we
> > > >> >> > > > > > > > > >> already do
> > > >> >> > > > > > > > > >> > > > for a
> > > >> >> > > > > > > > > >> > > > >> lot
> > > >> >> > > > > > > > > >> > > > >> >> > of
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> > > >> development + and
> > > >> >> > > the
> > > >> >> > > > > > > ability
> > > >> >> > > > > > > > > so
> > > >> >> > > > > > > > > >> > > tokens
> > > >> >> > > > > > > > > >> > > > >> will
> > > >> >> > > > > > > > > >> > > > >> >> > survive
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> > > restart/cluster
> > > >> >> > > failure.
> > > >> >> > > > > if a
> > > >> >> > > > > > > > > >> majority
> > > >> >> > > > > > > > > >> > > > agrees
> > > >> >> > > > > > > > > >> > > > >> the
> > > >> >> > > > > > > > > >> > > > >> >> > added
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> > > controller
> > > >> >> > > forwarding
> > > >> >> > > > > > keys
> > > >> >> > > > > > > to
> > > >> >> > > > > > > > > all
> > > >> >> > > > > > > > > >> > > > broker is
> > > >> >> > > > > > > > > >> > > > >> >> > justified
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter
> > security
> > > >> , I am
> > > >> >> > > fine
> > > >> >> > > > > > with
> > > >> >> > > > > > > > that
> > > >> >> > > > > > > > > >> option
> > > >> >> > > > > > > > > >> > > > too.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
> > > >> support SSL
> > > >> >> > > we
> > > >> >> > > > > can
> > > >> >> > > > > > > not
> > > >> >> > > > > > > > > >> store
> > > >> >> > > > > > > > > >> > > > master
> > > >> >> > > > > > > > > >> > > > >> keys
> > > >> >> > > > > > > > > >> > > > >> >> > in
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys
> > > will
> > > >> be
> > > >> >> > > exposed
> > > >> >> > > > > on
> > > >> >> > > > > > > > wire.
> > > >> >> > > > > > > > > To
> > > >> >> > > > > > > > > >> > > > support
> > > >> >> > > > > > > > > >> > > > >> >> > rotation
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting
> current
> > > >> clients is
> > > >> >> > > > > > > something I
> > > >> >> > > > > > > > > >> need to
> > > >> >> > > > > > > > > >> > > > put
> > > >> >> > > > > > > > > >> > > > >> more
> > > >> >> > > > > > > > > >> > > > >> >> > thought
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
> > > >> assumes the
> > > >> >> > > > > rotation
> > > >> >> > > > > > > > will
> > > >> >> > > > > > > > > >> > > > invalidate
> > > >> >> > > > > > > > > >> > > > >> all
> > > >> >> > > > > > > > > >> > > > >> >> > current
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to
> > also
> > > >> review
> > > >> >> > > and
> > > >> >> > > > > post
> > > >> >> > > > > > > > their
> > > >> >> > > > > > > > > >> > > comments
> > > >> >> > > > > > > > > >> > > > >> so we
> > > >> >> > > > > > > > > >> > > > >> >> > can
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at
> > 8:39
> > > >> AM,
> > > >> >> > > Ashish
> > > >> >> > > > > > Singh
> > > >> >> > > > > > > <
> > > >> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> > > >> >> > > > > > > > > >> > > > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016
> at
> > > >> 11:26 AM,
> > > >> >> > > > > Harsha <
> > > >> >> > > > > > > > > >> > > > kafka@harsha.io>
> > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> > > >> discussion
> > > >> >> > > threads
> > > >> >> > > > on
> > > >> >> > > > > > > this
> > > >> >> > > > > > > > > KIP.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response
> > > from
> > > >> >> > Jitendra
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a
> large
> > > >> number of
> > > >> >> > > > > clients
> > > >> >> > > > > > > that
> > > >> >> > > > > > > > > are
> > > >> >> > > > > > > > > >> > > > running
> > > >> >> > > > > > > > > >> > > > >> all
> > > >> >> > > > > > > > > >> > > > >> >> > over the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> > > authenticate
> > > >> with
> > > >> >> > > Kafka
> > > >> >> > > > > > > > brokers,
> > > >> >> > > > > > > > > >> is very
> > > >> >> > > > > > > > > >> > > > >> similar
> > > >> >> > > > > > > > > >> > > > >> >> > to the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of
> > large
> > > >> number
> > > >> >> > of
> > > >> >> > > > > tasks
> > > >> >> > > > > > > > > running
> > > >> >> > > > > > > > > >> > > across
> > > >> >> > > > > > > > > >> > > > >> the
> > > >> >> > > > > > > > > >> > > > >> >> > cluster
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> that
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication
> to
> > > >> Hdfs
> > > >> >> > > > Namenode.
> > > >> >> > > > > > > > > >> Therefore, the
> > > >> >> > > > > > > > > >> > > > >> >> > delegation token
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem
> > like
> > > a
> > > >> good
> > > >> >> > fit
> > > >> >> > > > for
> > > >> >> > > > > > > this
> > > >> >> > > > > > > > > use
> > > >> >> > > > > > > > > >> case
> > > >> >> > > > > > > > > >> > > > as we
> > > >> >> > > > > > > > > >> > > > >> >> > have seen
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> it
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large
> scale
> > > in
> > > >> HDFS
> > > >> >> > and
> > > >> >> > > > > YARN.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed
> design
> > is
> > > >> very
> > > >> >> > much
> > > >> >> > > > > > inline
> > > >> >> > > > > > > > with
> > > >> >> > > > > > > > > >> Hadoop
> > > >> >> > > > > > > > > >> > > > >> >> > approach. A few
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys
> > want
> > > >> to allow
> > > >> >> > > > > > infinite
> > > >> >> > > > > > > > > >> renewable
> > > >> >> > > > > > > > > >> > > > >> lifetime
> > > >> >> > > > > > > > > >> > > > >> >> > for a
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS
> restricts
> > a
> > > >> token
> > > >> >> > to a
> > > >> >> > > > max
> > > >> >> > > > > > > life
> > > >> >> > > > > > > > > time
> > > >> >> > > > > > > > > >> > > > (default
> > > >> >> > > > > > > > > >> > > > >> 7
> > > >> >> > > > > > > > > >> > > > >> >> > days). A
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's
> vulnerability
> > is
> > > >> >> > believed
> > > >> >> > > to
> > > >> >> > > > > > > > increase
> > > >> >> > > > > > > > > >> with
> > > >> >> > > > > > > > > >> > > > time.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having
> > > infinite
> > > >> >> > > lifetime
> > > >> >> > > > > > might
> > > >> >> > > > > > > > not
> > > >> >> > > > > > > > > >> be the
> > > >> >> > > > > > > > > >> > > > best
> > > >> >> > > > > > > > > >> > > > >> idea.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand
> the
> > > >> tokens
> > > >> >> > are
> > > >> >> > > > > stored
> > > >> >> > > > > > > in
> > > >> >> > > > > > > > > >> zookeeper
> > > >> >> > > > > > > > > >> > > > as
> > > >> >> > > > > > > > > >> > > > >> well,
> > > >> >> > > > > > > > > >> > > > >> >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> can
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there.
> This
> > > is
> > > >> clever
> > > >> >> > > as
> > > >> >> > > > it
> > > >> >> > > > > > can
> > > >> >> > > > > > > > > allow
> > > >> >> > > > > > > > > >> > > > >> replacing the
> > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of
> > max
> > > >> life
> > > >> >> > > time,
> > > >> >> > > > > and
> > > >> >> > > > > > > > > clients
> > > >> >> > > > > > > > > >> can
> > > >> >> > > > > > > > > >> > > > >> download
> > > >> >> > > > > > > > > >> > > > >> >> > new
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> > > >> shouldn't be
> > > >> >> > a
> > > >> >> > > > big
> > > >> >> > > > > > load
> > > >> >> > > > > > > > on
> > > >> >> > > > > > > > > >> > > zookeeper
> > > >> >> > > > > > > > > >> > > > >> as a
> > > >> >> > > > > > > > > >> > > > >> >> > client
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new
> > token
> > > >> once in
> > > >> >> > > > > several
> > > >> >> > > > > > > > days.
> > > >> >> > > > > > > > > >> In this
> > > >> >> > > > > > > > > >> > > > >> approach
> > > >> >> > > > > > > > > >> > > > >> >> > you
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite
> lifetime
> > > on
> > > >> the
> > > >> >> > > token
> > > >> >> > > > > even
> > > >> >> > > > > > > for
> > > >> >> > > > > > > > > >> long
> > > >> >> > > > > > > > > >> > > > running
> > > >> >> > > > > > > > > >> > > > >> >> > clients.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token
> password
> > > are
> > > >> >> > > generated
> > > >> >> > > > > > using
> > > >> >> > > > > > > a
> > > >> >> > > > > > > > > >> master
> > > >> >> > > > > > > > > >> > > key.
> > > >> >> > > > > > > > > >> > > > >> The
> > > >> >> > > > > > > > > >> > > > >> >> > master
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> key
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> > > >> periodically
> > > >> >> > > changed.
> > > >> >> > > > > In
> > > >> >> > > > > > > > > Hadoop,
> > > >> >> > > > > > > > > >> the
> > > >> >> > > > > > > > > >> > > > >> default
> > > >> >> > > > > > > > > >> > > > >> >> > renewal
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will
> require
> > > >> brokers
> > > >> >> > > > > > maintaining
> > > >> >> > > > > > > a
> > > >> >> > > > > > > > > >> list of X
> > > >> >> > > > > > > > > >> > > > most
> > > >> >> > > > > > > > > >> > > > >> >> > recent
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> master
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will
> > have
> > > >> to be
> > > >> >> > > > > persisted
> > > >> >> > > > > > > > > >> somewhere, as
> > > >> >> > > > > > > > > >> > > > if a
> > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to
> get
> > > >> that list
> > > >> >> > > > again
> > > >> >> > > > > > and
> > > >> >> > > > > > > > > >> storing
> > > >> >> > > > > > > > > >> > > > master
> > > >> >> > > > > > > > > >> > > > >> keys
> > > >> >> > > > > > > > > >> > > > >> >> > on ZK
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> is
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> > > However,
> > > >> if a
> > > >> >> > > > broker
> > > >> >> > > > > > goes
> > > >> >> > > > > > > > > down
> > > >> >> > > > > > > > > >> then
> > > >> >> > > > > > > > > >> > > we
> > > >> >> > > > > > > > > >> > > > >> have
> > > >> >> > > > > > > > > >> > > > >> >> > much
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
> > > >> client can
> > > >> >> > > > always
> > > >> >> > > > > > > > > >> > > re-authenticate
> > > >> >> > > > > > > > > >> > > > is
> > > >> >> > > > > > > > > >> > > > >> such
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to
> take a
> > > >> look at
> > > >> >> > > other
> > > >> >> > > > > > > > > >> alternatives
> > > >> >> > > > > > > > > >> > > this
> > > >> >> > > > > > > > > >> > > > >> list has
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a
> thorough
> > > >> proposal,
> > > >> >> > > > great
> > > >> >> > > > > > > work!"
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016,
> > at
> > > >> 10:28
> > > >> >> > PM,
> > > >> >> > > > Gwen
> > > >> >> > > > > > > > Shapira
> > > >> >> > > > > > > > > >> wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
> > > >> Thanks!
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7,
> 2016
> > at
> > > >> 9:25
> > > >> >> > PM,
> > > >> >> > > > > > Harsha <
> > > >> >> > > > > > > > > >> > > > kafka@harsha.io
> > > >> >> > > > > > > > > >> > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need
> > any
> > > >> release
> > > >> >> > > > > vehicle
> > > >> >> > > > > > > but
> > > >> >> > > > > > > > > >> still the
> > > >> >> > > > > > > > > >> > > > >> work can
> > > >> >> > > > > > > > > >> > > > >> >> > move
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> > > >> interested in
> > > >> >> > the
> > > >> >> > > > KIP
> > > >> >> > > > > > > > please
> > > >> >> > > > > > > > > >> do the
> > > >> >> > > > > > > > > >> > > > >> review and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7,
> > 2016,
> > > >> at
> > > >> >> > 04:59
> > > >> >> > > > PM,
> > > >> >> > > > > > > Ismael
> > > >> >> > > > > > > > > >> Juma
> > > >> >> > > > > > > > > >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it
> > > >> would be
> > > >> >> > good
> > > >> >> > > > to
> > > >> >> > > > > > have
> > > >> >> > > > > > > > > more
> > > >> >> > > > > > > > > >> time
> > > >> >> > > > > > > > > >> > > to
> > > >> >> > > > > > > > > >> > > > >> review
> > > >> >> > > > > > > > > >> > > > >> >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8,
> > 2016
> > > >> at
> > > >> >> > 12:55
> > > >> >> > > > AM,
> > > >> >> > > > > > Gwen
> > > >> >> > > > > > > > > >> Shapira <
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
> > > >> depends on
> > > >> >> > > > KIP-43,
> > > >> >> > > > > > > which
> > > >> >> > > > > > > > > is
> > > >> >> > > > > > > > > >> > > > already a
> > > >> >> > > > > > > > > >> > > > >> bit
> > > >> >> > > > > > > > > >> > > > >> >> > of a
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next
> > release
> > > -
> > > >> any
> > > >> >> > > chance
> > > >> >> > > > > we
> > > >> >> > > > > > > can
> > > >> >> > > > > > > > > >> delay
> > > >> >> > > > > > > > > >> > > > >> delegation
> > > >> >> > > > > > > > > >> > > > >> >> > tokens
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> > > community
> > > >> >> > working
> > > >> >> > > > on a
> > > >> >> > > > > > > > release
> > > >> >> > > > > > > > > >> every
> > > >> >> > > > > > > > > >> > > 3
> > > >> >> > > > > > > > > >> > > > >> month,
> > > >> >> > > > > > > > > >> > > > >> >> > this
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb
> 26,
> > > >> 2016 at
> > > >> >> > > 5:11
> > > >> >> > > > > PM,
> > > >> >> > > > > > > > Ashish
> > > >> >> > > > > > > > > >> Singh
> > > >> >> > > > > > > > > >> > > <
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks
> again
> > > for
> > > >> the
> > > >> >> > > > awesome
> > > >> >> > > > > > > write
> > > >> >> > > > > > > > > up.
> > > >> >> > > > > > > > > >> > > > Following
> > > >> >> > > > > > > > > >> > > > >> our
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I
> think
> > > it
> > > >> will
> > > >> >> > be
> > > >> >> > > > > easier
> > > >> >> > > > > > > to
> > > >> >> > > > > > > > > >> compare
> > > >> >> > > > > > > > > >> > > > >> various
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> > > together.
> > > >> I am
> > > >> >> > > > stating
> > > >> >> > > > > > > > below a
> > > >> >> > > > > > > > > >> few
> > > >> >> > > > > > > > > >> > > > >> >> > alternatives along
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> > > proposal.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
> > > >> proposal)
> > > >> >> > Store
> > > >> >> > > > > > > Delegation
> > > >> >> > > > > > > > > >> Token,
> > > >> >> > > > > > > > > >> > > DT,
> > > >> >> > > > > > > > > >> > > > >> on ZK.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> Client
> > > >> >> > > authenticates
> > > >> >> > > > > > with a
> > > >> >> > > > > > > > > >> broker.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once
> a
> > > >> client is
> > > >> >> > > > > > > > authenticated,
> > > >> >> > > > > > > > > >> it
> > > >> >> > > > > > > > > >> > > will
> > > >> >> > > > > > > > > >> > > > >> make a
> > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > > >> delegation
> > > >> >> > > > token.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > > broker
> > > >> >> > > generates
> > > >> >> > > > a
> > > >> >> > > > > > > shared
> > > >> >> > > > > > > > > >> secret
> > > >> >> > > > > > > > > >> > > > based
> > > >> >> > > > > > > > > >> > > > >> on
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> Password/Secret
> > > >> >> > shared
> > > >> >> > > > > > between
> > > >> >> > > > > > > > all
> > > >> >> > > > > > > > > >> > > brokers,
> > > >> >> > > > > > > > > >> > > > >> >> > randomly
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> Broker
> > > >> stores
> > > >> >> > this
> > > >> >> > > > > token
> > > >> >> > > > > > in
> > > >> >> > > > > > > > its
> > > >> >> > > > > > > > > >> in
> > > >> >> > > > > > > > > >> > > > memory
> > > >> >> > > > > > > > > >> > > > >> cache.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > >> DelegationToken
> > > >> >> > > > > without
> > > >> >> > > > > > > the
> > > >> >> > > > > > > > > >> hmac in
> > > >> >> > > > > > > > > >> > > the
> > > >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All
> > > >> brokers will
> > > >> >> > > > have a
> > > >> >> > > > > > > cache
> > > >> >> > > > > > > > > >> backed
> > > >> >> > > > > > > > > >> > > by
> > > >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> they
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get
> > notified
> > > >> >> > whenever
> > > >> >> > > a
> > > >> >> > > > > new
> > > >> >> > > > > > > > token
> > > >> >> > > > > > > > > is
> > > >> >> > > > > > > > > >> > > > >> generated and
> > > >> >> > > > > > > > > >> > > > >> >> > they
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local
> > cache
> > > >> whenever
> > > >> >> > > > token
> > > >> >> > > > > > > state
> > > >> >> > > > > > > > > >> changes.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> Broker
> > > >> returns
> > > >> >> > the
> > > >> >> > > > > token
> > > >> >> > > > > > to
> > > >> >> > > > > > > > > >> Client.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > issues
> > > >> and
> > > >> >> > fixes
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> > Probable
> > > >> race
> > > >> >> > > > > condition,
> > > >> >> > > > > > > > client
> > > >> >> > > > > > > > > >> tries
> > > >> >> > > > > > > > > >> > > to
> > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> with
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is
> > yet
> > > >> to be
> > > >> >> > > > updated
> > > >> >> > > > > > with
> > > >> >> > > > > > > > the
> > > >> >> > > > > > > > > >> newly
> > > >> >> > > > > > > > > >> > > > >> generated
> > > >> >> > > > > > > > > >> > > > >> >> > DT.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> This
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt
> with
> > > >> making
> > > >> >> > > > > dtRequest
> > > >> >> > > > > > > > block
> > > >> >> > > > > > > > > >> until
> > > >> >> > > > > > > > > >> > > all
> > > >> >> > > > > > > > > >> > > > >> >> > brokers have
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache.
> Zk
> > > >> barrier or
> > > >> >> > > > > similar
> > > >> >> > > > > > > > > >> mechanism
> > > >> >> > > > > > > > > >> > > can
> > > >> >> > > > > > > > > >> > > > be
> > > >> >> > > > > > > > > >> > > > >> used.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> mechanisms
> > > >> will
> > > >> >> > > increase
> > > >> >> > > > > > > > > complexity.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2.
> Using a
> > > >> static
> > > >> >> > > secret
> > > >> >> > > > > key
> > > >> >> > > > > > > > from
> > > >> >> > > > > > > > > >> config
> > > >> >> > > > > > > > > >> > > > >> file. Will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config
> and
> > > >> uses a
> > > >> >> > > static
> > > >> >> > > > > > > secret
> > > >> >> > > > > > > > > >> key. It
> > > >> >> > > > > > > > > >> > > is
> > > >> >> > > > > > > > > >> > > > >> advised
> > > >> >> > > > > > > > > >> > > > >> >> > to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > periodically.
> > > >> This
> > > >> >> > can
> > > >> >> > > > be
> > > >> >> > > > > > > > avoided
> > > >> >> > > > > > > > > >> with
> > > >> >> > > > > > > > > >> > > > >> controller
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing
> to
> > > >> brokers
> > > >> >> > > > > > > periodically.
> > > >> >> > > > > > > > > >> However,
> > > >> >> > > > > > > > > >> > > > >> this will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain
> > > >> certain
> > > >> >> > > counts
> > > >> >> > > > of
> > > >> >> > > > > > > > > >> secretKeys.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> (Alternative
> > 1)
> > > >> Have
> > > >> >> > > > > controller
> > > >> >> > > > > > > > > >> generate
> > > >> >> > > > > > > > > >> > > > >> delegation
> > > >> >> > > > > > > > > >> > > > >> >> > token.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> Client
> > > >> >> > > authenticates
> > > >> >> > > > > > with a
> > > >> >> > > > > > > > > >> broker.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once
> a
> > > >> client is
> > > >> >> > > > > > > > authenticated,
> > > >> >> > > > > > > > > >> it
> > > >> >> > > > > > > > > >> > > will
> > > >> >> > > > > > > > > >> > > > >> make a
> > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > > >> delegation
> > > >> >> > > > token.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3.
> Broker
> > > >> forwards
> > > >> >> > the
> > > >> >> > > > > > request
> > > >> >> > > > > > > > to
> > > >> >> > > > > > > > > >> > > > controller.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > > Controller
> > > >> >> > > generates
> > > >> >> > > > a
> > > >> >> > > > > DT
> > > >> >> > > > > > > and
> > > >> >> > > > > > > > > >> > > broadcasts
> > > >> >> > > > > > > > > >> > > > >> to all
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5.
> Broker
> > > >> stores
> > > >> >> > this
> > > >> >> > > > > token
> > > >> >> > > > > > in
> > > >> >> > > > > > > > its
> > > >> >> > > > > > > > > >> memory
> > > >> >> > > > > > > > > >> > > > >> cache.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > > Controller
> > > >> >> > responds
> > > >> >> > > > to
> > > >> >> > > > > > > > broker’s
> > > >> >> > > > > > > > > >> DT
> > > >> >> > > > > > > > > >> > > req.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7.
> Broker
> > > >> returns
> > > >> >> > the
> > > >> >> > > > > token
> > > >> >> > > > > > to
> > > >> >> > > > > > > > > >> Client.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > issues
> > > >> and
> > > >> >> > fixes
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We
> will
> > > >> have to
> > > >> >> > add
> > > >> >> > > > new
> > > >> >> > > > > > > APIs
> > > >> >> > > > > > > > to
> > > >> >> > > > > > > > > >> > > support
> > > >> >> > > > > > > > > >> > > > >> >> > controller
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers
> on
> > > >> top of
> > > >> >> > the
> > > >> >> > > > > > minimal
> > > >> >> > > > > > > > APIs
> > > >> >> > > > > > > > > >> that
> > > >> >> > > > > > > > > >> > > are
> > > >> >> > > > > > > > > >> > > > >> >> > currently
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We
> will
> > > >> also have
> > > >> >> > > to
> > > >> >> > > > > add
> > > >> >> > > > > > > APIs
> > > >> >> > > > > > > > > to
> > > >> >> > > > > > > > > >> > > support
> > > >> >> > > > > > > > > >> > > > >> the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a
> new
> > > >> broker
> > > >> >> > > comes
> > > >> >> > > > up
> > > >> >> > > > > > it
> > > >> >> > > > > > > > will
> > > >> >> > > > > > > > > >> have
> > > >> >> > > > > > > > > >> > > to
> > > >> >> > > > > > > > > >> > > > >> get all
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > > >> controller.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
> > > >> catastrophic
> > > >> >> > > > > failures
> > > >> >> > > > > > > > where
> > > >> >> > > > > > > > > >> all
> > > >> >> > > > > > > > > >> > > > brokers
> > > >> >> > > > > > > > > >> > > > >> go
> > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost
> > even
> > > >> if
> > > >> >> > > servers
> > > >> >> > > > > are
> > > >> >> > > > > > > > > >> restarted as
> > > >> >> > > > > > > > > >> > > > >> tokens
> > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> > > >> happens,
> > > >> >> > then
> > > >> >> > > > > there
> > > >> >> > > > > > > are
> > > >> >> > > > > > > > > more
> > > >> >> > > > > > > > > >> > > > important
> > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it
> > is
> > > >> better
> > > >> >> > to
> > > >> >> > > > > > > > > >> re-authenticate.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> (Alternative
> > 2)
> > > >> Do not
> > > >> >> > > > > > distribute
> > > >> >> > > > > > > > DT
> > > >> >> > > > > > > > > to
> > > >> >> > > > > > > > > >> > > other
> > > >> >> > > > > > > > > >> > > > >> brokers
> > > >> >> > > > > > > > > >> > > > >> >> > at
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> Client
> > > >> >> > > authenticates
> > > >> >> > > > > > with a
> > > >> >> > > > > > > > > >> broker.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once
> a
> > > >> client is
> > > >> >> > > > > > > > authenticated,
> > > >> >> > > > > > > > > >> it
> > > >> >> > > > > > > > > >> > > will
> > > >> >> > > > > > > > > >> > > > >> make a
> > > >> >> > > > > > > > > >> > > > >> >> > broker
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > > >> delegation
> > > >> >> > > > token.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > > broker
> > > >> >> > > generates
> > > >> >> > > > DT
> > > >> >> > > > > > of
> > > >> >> > > > > > > > > form,
> > > >> >> > > > > > > > > >> > > [hmac +
> > > >> >> > > > > > > > > >> > > > >> (owner,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > maxLifeTime,
> > > >> id,
> > > >> >> > hmac,
> > > >> >> > > > > > > > > >> expirationTime)]
> > > >> >> > > > > > > > > >> > > and
> > > >> >> > > > > > > > > >> > > > >> passes
> > > >> >> > > > > > > > > >> > > > >> >> > back
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> this
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is
> > > >> generated
> > > >> >> > via
> > > >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> > > >> >> > > > > > > > > >> > > > >> renewer,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> expirationTime)
> > > >> >> > using
> > > >> >> > > > > > > > SecretKey}.
> > > >> >> > > > > > > > > >> Note
> > > >> >> > > > > > > > > >> > > that
> > > >> >> > > > > > > > > >> > > > >> all
> > > >> >> > > > > > > > > >> > > > >> >> > brokers
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> have
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> Client
> > > >> then goes
> > > >> >> > to
> > > >> >> > > > any
> > > >> >> > > > > > > > broker
> > > >> >> > > > > > > > > >> and to
> > > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
> > > >> recalculates
> > > >> >> > > hmac
> > > >> >> > > > > > using
> > > >> >> > > > > > > > > >> (owner,
> > > >> >> > > > > > > > > >> > > > >> renewer,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> expirationTime) info
> > > >> >> > > > from
> > > >> >> > > > > DT
> > > >> >> > > > > > > and
> > > >> >> > > > > > > > > its
> > > >> >> > > > > > > > > >> > > > >> SecretKey. If
> > > >> >> > > > > > > > > >> > > > >> >> > it
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT,
> client
> > > is
> > > >> >> > > > > authenticated.
> > > >> >> > > > > > > > Yes,
> > > >> >> > > > > > > > > >> it will
> > > >> >> > > > > > > > > >> > > > do
> > > >> >> > > > > > > > > >> > > > >> other
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> timestamp
> > > >> expiry and
> > > >> >> > > > such.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that
> > > secret
> > > >> key
> > > >> >> > will
> > > >> >> > > > be
> > > >> >> > > > > > > > > generated
> > > >> >> > > > > > > > > >> by
> > > >> >> > > > > > > > > >> > > > >> controller
> > > >> >> > > > > > > > > >> > > > >> >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> periodically.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> > issues
> > > >> and
> > > >> >> > fixes
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How
> to
> > > >> delete a
> > > >> >> > DT?
> > > >> >> > > > > Yes,
> > > >> >> > > > > > > that
> > > >> >> > > > > > > > > is
> > > >> >> > > > > > > > > >> a
> > > >> >> > > > > > > > > >> > > > downside
> > > >> >> > > > > > > > > >> > > > >> >> > here.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be
> handled
> > > >> with
> > > >> >> > > brokers
> > > >> >> > > > > > > > > maintaining
> > > >> >> > > > > > > > > >> a
> > > >> >> > > > > > > > > >> > > > >> blacklist of
> > > >> >> > > > > > > > > >> > > > >> >> > DTs,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be
> > > >> removed after
> > > >> >> > > > > expiry.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
> > > >> catastrophic
> > > >> >> > > > > failures
> > > >> >> > > > > > > > where
> > > >> >> > > > > > > > > >> all
> > > >> >> > > > > > > > > >> > > > brokers
> > > >> >> > > > > > > > > >> > > > >> go
> > > >> >> > > > > > > > > >> > > > >> >> > down,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost
> > even
> > > >> if
> > > >> >> > > servers
> > > >> >> > > > > are
> > > >> >> > > > > > > > > >> restarted as
> > > >> >> > > > > > > > > >> > > > >> tokens
> > > >> >> > > > > > > > > >> > > > >> >> > are not
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> > > >> happens,
> > > >> >> > then
> > > >> >> > > > > there
> > > >> >> > > > > > > are
> > > >> >> > > > > > > > > more
> > > >> >> > > > > > > > > >> > > > important
> > > >> >> > > > > > > > > >> > > > >> >> > things to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it
> > is
> > > >> better
> > > >> >> > to
> > > >> >> > > > > > > > > >> re-authenticate.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb
> > 26,
> > > >> 2016 at
> > > >> >> > > > 1:58
> > > >> >> > > > > > PM,
> > > >> >> > > > > > > > > Parth
> > > >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > pbrahmbhatt@hortonworks.com>
> > > >> >> > > > > > > > wrote:
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have
> filed
> > > >> KIP-48 so
> > > >> >> > > we
> > > >> >> > > > > can
> > > >> >> > > > > > > > offer
> > > >> >> > > > > > > > > >> hadoop
> > > >> >> > > > > > > > > >> > > > like
> > > >> >> > > > > > > > > >> > > > >> >> > delegation
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You
> > can
> > > >> review
> > > >> >> > > the
> > > >> >> > > > > > design
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
> > > >> depends on
> > > >> >> > > KIP-43
> > > >> >> > > > > and
> > > >> >> > > > > > > we
> > > >> >> > > > > > > > > >> have also
> > > >> >> > > > > > > > > >> > > > >> >> > discussed an
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed
> > > design
> > > >> here<
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> > > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > > >> >> > > > > > > > > >> > > > >> >> >
> > > >> >> > > > > > > > > >> > > > >>
> > > >> >> > > > > > > > > >> > > >
> > > >> >> > > > > > > > > >> > >
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > > >>
> > > >> >> > > > > > > > > >
> > > >> >> > > > > > > > >
> > > >> >> > > > > > > >
> > > >> >> > > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > --
> > > >> >> > > > > > Regards,
> > > >> >> > > > > >
> > > >> >> > > > > > Rajini
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > --
> > > >> >> > > > Regards,
> > > >> >> > > >
> > > >> >> > > > Rajini
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >> Liquan Pei
> > > >> >> Software Engineer, Confluent Inc
> > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Parth,
Thanks for the reply. A couple of comments inline below.
On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt <
brahmbhatt.parth@gmail.com> wrote:
> 1. Who / how are tokens renewed? By original requester only? or using
> Kerberos
> auth only?
> My recommendation is to do this only using Kerberos auth and only threw the
> renewer specified during the acquisition request.
>
>
Hmm, not sure that I follow this. Are you saying that any client
authenticated with the delegation token can renew, i.e. there is no renewer
needed?
Also, just to be clear, any authenticated client (either through SASL or
SSL) can request a delegation token for the authenticated user, right?
> 2. Are tokens stored on each broker or in ZK?
> My recommendation is still to store in ZK or not store them at all. The
> whole controller based distribution is too much overhead with not much to
> achieve.
>
> 3. How are tokens invalidated / expired?
> Either by expiration time out or through an explicit request to invalidate.
>
> 4. Which encryption algorithm is used?
> SCRAM
>
> 5. What is the impersonation proposal (it wasn't in the KIP but was
> discussed
> in this thread)?
> There is no imperonation proposal. I tried and explained how its a
> different problem and why its not really necessary to discuss that as part
> of this KIP. This KIP will not support any impersonation, it will just be
> another way to authenticate.
>
> 6. Do we need new ACLs, if so - for what actions?
> We do not need new ACLs.
>
>
Could we document the format of the new request/response and their
associated Resource and Operation for ACL?
> 7. How would the delegation token be configured in the client?
> Should be through config. I wasn't planning on supporting JAAS for tokens.
> I don't believe hadoop does this either.
>
> Thanks
> Parth
>
>
>
> On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Harsha,
> >
> > Another question.
> >
> > 9. How would the delegation token be configured in the client? The
> standard
> > way is to do this through JAAS. However, we will need to think through if
> > this is convenient in a shared environment. For example, when a new task
> is
> > added to a Storm worker node, do we need to dynamically add a new section
> > in the JAAS file? It may be more convenient if we can pass in the token
> > through the config directly w/o going through JAAS.
> >
> > Are you or Parth still actively working on this KIP?
> >
> > Thanks,
> >
> > Jun
> >
> >
> >
> > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Just to add on that list.
> > >
> > > 2. It would be good to document the format of the data stored in ZK.
> > > 7. Earlier, there was a discussion on whether the tokens should be
> > > propagated through ZK like config/acl/quota, or through the controller.
> > > Currently, the controller is only designed for propagating topic
> > metadata,
> > > but not other data.
> > > 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since
> it's
> > > deprecated?
> > >
> > > Also, the images in the wiki seem broken.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > >
> > >> From what I can see, remaining questions are:
> > >>
> > >> 1. Who / how are tokens renewed? By original requester only? or using
> > >> Kerberos auth only?
> > >> 2. Are tokens stored on each broker or in ZK?
> > >> 3. How are tokens invalidated / expired?
> > >> 4. Which encryption algorithm is used?
> > >> 5. What is the impersonation proposal (it wasn't in the KIP but was
> > >> discussed in this thread)?
> > >> 6. Do we need new ACLs, if so - for what actions?
> > >>
> > >> Gwen
> > >>
> > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> > >> > Jun & Ismael,
> > >> > Unfortunately I couldn't attend the KIP
> > meeting
> > >> > when delegation tokens discussed.
> Appreciate
> > if
> > >> > you can update the thread if you have any
> > >> > further questions.
> > >> > Thanks,
> > >> > Harsha
> > >> >
> > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> > >> >> It seems that the links to images in the KIP are broken.
> > >> >>
> > >> >> Liquan
> > >> >>
> > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> > >> >> brahmbhatt.parth@gmail.com> wrote:
> > >> >>
> > >> >> > 110. What does getDelegationTokenAs mean?
> > >> >> > In the current proposal we only allow a user to get delegation
> > token
> > >> for
> > >> >> > the identity that it authenticated as using another mechanism,
> i.e.
> > >> A user
> > >> >> > that authenticate using a keytab for principal user1@EXAMPLE.COM
> > >> will get
> > >> >> > delegation tokens for that user only. In future I think we will
> > have
> > >> to
> > >> >> > extend support such that we allow some set of users (
> > >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to
> acquire
> > >> >> > delegation tokens on behalf of other users whose identity they
> have
> > >> >> > verified independently. Kafka brokers will have ACLs to control
> > >> which
> > >> >> > users are allowed to impersonate other users and get tokens on
> > >> behalf of
> > >> >> > them. Overall Impersonation is a whole different problem in my
> > >> opinion and
> > >> >> > I think we can tackle it in separate KIP.
> > >> >> >
> > >> >> > 111. What's the typical rate of getting and renewing delegation
> > >> tokens?
> > >> >> > Typically this should be very very low, 1 request per minute is a
> > >> >> > relatively high estimate. However it depends on the token
> > >> expiration. I am
> > >> >> > less worried about the extra load it puts on controller vs the
> > added
> > >> >> > complexity and the value it offers.
> > >> >> >
> > >> >> > Thanks
> > >> >> > Parth
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk>
> > >> wrote:
> > >> >> >
> > >> >> > > Thanks Rajini. It would probably require a separate KIP as it
> > will
> > >> >> > > introduce user visible changes. We could also update KIP-48 to
> > >> have this
> > >> >> > > information, but it seems cleaner to do it separately. We can
> > >> discuss
> > >> >> > that
> > >> >> > > in the KIP call today.
> > >> >> > >
> > >> >> > > Ismael
> > >> >> > >
> > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> > >> >> > > rajinisivaram@googlemail.com> wrote:
> > >> >> > >
> > >> >> > > > Ismael,
> > >> >> > > >
> > >> >> > > > I have created a JIRA (
> > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> > >> >> > > > for adding SCRAM as a SASL mechanism. Would that need another
> > >> KIP? If
> > >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA that
> > gets
> > >> >> > > reviewed
> > >> >> > > > when the PR is ready?
> > >> >> > > >
> > >> >> > > > Thank you,
> > >> >> > > >
> > >> >> > > > Rajini
> > >> >> > > >
> > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> > ismael@juma.me.uk>
> > >> >> > wrote:
> > >> >> > > >
> > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> > >> >> > > > >
> > >> >> > > > > Gwen had independently mentioned this as a SASL mechanism
> > that
> > >> might
> > >> >> > be
> > >> >> > > > > useful for Kafka and I have been meaning to check it in
> more
> > >> detail.
> > >> >> > > Good
> > >> >> > > > > to know that you are willing to contribute an
> implementation.
> > >> Maybe
> > >> >> > we
> > >> >> > > > > should file a separate JIRA for this?
> > >> >> > > > >
> > >> >> > > > > Ismael
> > >> >> > > > >
> > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> > >> >> > > > > rajinisivaram@googlemail.com> wrote:
> > >> >> > > > >
> > >> >> > > > > > SCRAM (Salted Challenge Response Authentication
> Mechanism)
> > >> is a
> > >> >> > > better
> > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> > built-in
> > >> SCRAM
> > >> >> > > > > > SaslServer or SaslClient, but I will be happy to add
> > support
> > >> in
> > >> >> > Kafka
> > >> >> > > > > since
> > >> >> > > > > > it would be a useful mechanism to support anyway.
> > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the
> protocol
> > >> for
> > >> >> > > > > > SCRAM-SHA-256.
> > >> >> > > > > >
> > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <
> jun@confluent.io
> > >
> > >> wrote:
> > >> >> > > > > >
> > >> >> > > > > > > Parth,
> > >> >> > > > > > >
> > >> >> > > > > > > Thanks for the explanation. A couple of more questions.
> > >> >> > > > > > >
> > >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> > >> >> > > > > > >
> > >> >> > > > > > > 111. What's the typical rate of getting and renewing
> > >> delegation
> > >> >> > > > tokens?
> > >> >> > > > > > > That may have an impact on whether they should be
> > directed
> > >> to the
> > >> >> > > > > > > controller.
> > >> >> > > > > > >
> > >> >> > > > > > > Jun
> > >> >> > > > > > >
> > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> > >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> > >> >> > > > > > >
> > >> >> > > > > > > > Hi Jun,
> > >> >> > > > > > > >
> > >> >> > > > > > > > Thanks for reviewing.
> > >> >> > > > > > > >
> > >> >> > > > > > > > * We could add a Cluster action to add acls on who
> can
> > >> request
> > >> >> > > > > > delegation
> > >> >> > > > > > > > tokens. I don't see the use case for that yet but
> down
> > >> the line
> > >> >> > > > when
> > >> >> > > > > we
> > >> >> > > > > > > > start supporting getDelegationTokenAs it will be
> > >> necessary.
> > >> >> > > > > > > > * Yes we recommend tokens to be only used/distributed
> > >> over
> > >> >> > secure
> > >> >> > > > > > > channels.
> > >> >> > > > > > > > * Depending on what design we end up choosing
> > >> Invalidation will
> > >> >> > > be
> > >> >> > > > > > > > responsibility of every broker or controller.
> > >> >> > > > > > > > * I am not sure if I documented somewhere that
> > >> invalidation
> > >> >> > will
> > >> >> > > > > > directly
> > >> >> > > > > > > > go through zookeeper but that is not the intent.
> > >> Invalidation
> > >> >> > > will
> > >> >> > > > > > either
> > >> >> > > > > > > > be request based or due to expiration. No direct
> > >> zookeeper
> > >> >> > > > > interaction
> > >> >> > > > > > > from
> > >> >> > > > > > > > any client.
> > >> >> > > > > > > > * "Broker also stores the DelegationToken without the
> > >> hmac in
> > >> >> > the
> > >> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
> > >> purpose of
> > >> >> > > > > zookeeper
> > >> >> > > > > > in
> > >> >> > > > > > > > this design is as distribution channel for tokens
> > >> between all
> > >> >> > > > brokers
> > >> >> > > > > > > and a
> > >> >> > > > > > > > layer that ensures only tokens that were generated by
> > >> making a
> > >> >> > > > > request
> > >> >> > > > > > > to a
> > >> >> > > > > > > > broker will be accepted (more on this in second
> > >> paragraph). The
> > >> >> > > > token
> > >> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> > >> expiration,
> > >> >> > > hmac)
> > >> >> > > > ,
> > >> >> > > > > > one
> > >> >> > > > > > > of
> > >> >> > > > > > > > which is the finally generated hmac but hmac it self
> is
> > >> >> > derivable
> > >> >> > > > if
> > >> >> > > > > > you
> > >> >> > > > > > > > have all the other elements of the token + secret key
> > to
> > >> >> > generate
> > >> >> > > > > hmac.
> > >> >> > > > > > > > Given zookeeper does not provide SSL support we do
> not
> > >> want the
> > >> >> > > > > entire
> > >> >> > > > > > > > token to be wire transferred to zookeeper as that
> will
> > >> be an
> > >> >> > > > insecure
> > >> >> > > > > > > wire
> > >> >> > > > > > > > transfer. Instead we only store all the other
> elements
> > >> of a
> > >> >> > > > > delegation
> > >> >> > > > > > > > tokens. Brokers can read these elements and because
> > they
> > >> also
> > >> >> > > have
> > >> >> > > > > > access
> > >> >> > > > > > > > to secret key they will be able to generate hmac on
> > >> their end.
> > >> >> > > > > > > >
> > >> >> > > > > > > > One of the alternative proposed is to avoid zookeeper
> > >> >> > > altogether. A
> > >> >> > > > > > > Client
> > >> >> > > > > > > > will call broker with required information (owner,
> > >> renwer,
> > >> >> > > > > expiration)
> > >> >> > > > > > > and
> > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't store this
> > in
> > >> >> > > zookeeper.
> > >> >> > > > > > From
> > >> >> > > > > > > > this point a client can contact any broker with all
> the
> > >> >> > > delegation
> > >> >> > > > > > token
> > >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the
> borker
> > >> will
> > >> >> > > > > regenerate
> > >> >> > > > > > > the
> > >> >> > > > > > > > hmac and as long as it matches with hmac presented by
> > >> client ,
> > >> >> > > > broker
> > >> >> > > > > > > will
> > >> >> > > > > > > > allow the request to authenticate. Only problem with
> > >> this
> > >> >> > > approach
> > >> >> > > > > is
> > >> >> > > > > > if
> > >> >> > > > > > > > the secret key is compromised any client can now
> > generate
> > >> >> > random
> > >> >> > > > > tokens
> > >> >> > > > > > > and
> > >> >> > > > > > > > they will still be able to authenticate as any user
> > they
> > >> like.
> > >> >> > > with
> > >> >> > > > > > > > zookeeper we guarantee that only tokens acquired via
> a
> > >> broker
> > >> >> > > > (using
> > >> >> > > > > > some
> > >> >> > > > > > > > auth scheme other than delegation token) will be
> > >> accepted. We
> > >> >> > > need
> > >> >> > > > to
> > >> >> > > > > > > > discuss which proposal makes more sense and we can go
> > >> over it
> > >> >> > in
> > >> >> > > > > > > tomorrow's
> > >> >> > > > > > > > meeting.
> > >> >> > > > > > > >
> > >> >> > > > > > > > Also, can you forward the invite to me?
> > >> >> > > > > > > >
> > >> >> > > > > > > > Thanks
> > >> >> > > > > > > > Parth
> > >> >> > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> > >> jun@confluent.io>
> > >> >> > > > wrote:
> > >> >> > > > > > > >
> > >> >> > > > > > > > > Thanks for the KIP. A few comments.
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 100. This potentially can be useful for Kafka
> Connect
> > >> and
> > >> >> > Kafka
> > >> >> > > > > rest
> > >> >> > > > > > > > proxy
> > >> >> > > > > > > > > where a worker agent will need to run a task on
> > behalf
> > >> of a
> > >> >> > > > client.
> > >> >> > > > > > We
> > >> >> > > > > > > > will
> > >> >> > > > > > > > > likely need to change how those services use Kafka
> > >> clients
> > >> >> > > > > > > > > (producer/consumer). Instead of a shared client per
> > >> worker,
> > >> >> > we
> > >> >> > > > will
> > >> >> > > > > > > need
> > >> >> > > > > > > > a
> > >> >> > > > > > > > > client per user task since the authentication
> happens
> > >> at the
> > >> >> > > > > > connection
> > >> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
> > >> workers.
> > >> >> > So,
> > >> >> > > we
> > >> >> > > > > > > > probably
> > >> >> > > > > > > > > need to allow multiple renewers. For Kafka rest
> > proxy,
> > >> the
> > >> >> > > > renewer
> > >> >> > > > > > can
> > >> >> > > > > > > > > probably just be the creator of the token.
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 101. Do we need new acl on who can request
> delegation
> > >> tokens?
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 102. Do we recommend people to send delegation
> tokens
> > >> in an
> > >> >> > > > > encrypted
> > >> >> > > > > > > > > channel?
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 103. Who is responsible for expiring tokens, every
> > >> broker?
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 104. For invalidating tokens, would it be better to
> > do
> > >> it in
> > >> >> > a
> > >> >> > > > > > request
> > >> >> > > > > > > > > instead of going to ZK directly?
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 105. The terminology of client in the wiki
> sometimes
> > >> refers
> > >> >> > to
> > >> >> > > > the
> > >> >> > > > > > end
> > >> >> > > > > > > > > client and some other times refers to the client
> > using
> > >> the
> > >> >> > > > > delegation
> > >> >> > > > > > > > > tokens. It would be useful to distinguish between
> the
> > >> two.
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > 106. Could you explain the sentence "Broker also
> > >> stores the
> > >> >> > > > > > > > DelegationToken
> > >> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
> > >> thought the
> > >> >> > > > > > > delegation
> > >> >> > > > > > > > > token is the hmac.
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > Thanks,
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > Jun
> > >> >> > > > > > > > >
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> > >> jun@confluent.io>
> > >> >> > > > wrote:
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > > Hi, Harsha,
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can
> discuss
> > >> this in
> > >> >> > > the
> > >> >> > > > > > > meeting
> > >> >> > > > > > > > > > tomorrow.
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > Thanks,
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > Jun
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> > >> kafka@harsha.io>
> > >> >> > > > wrote:
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > > >> Hi All,
> > >> >> > > > > > > > > >> Can we have a KIP meeting around
> this.
> > >> The KIP
> > >> >> > is
> > >> >> > > > up
> > >> >> > > > > > for
> > >> >> > > > > > > > > >> sometime and if there are any
> questions
> > >> lets
> > >> >> > > > quickly
> > >> >> > > > > > hash
> > >> >> > > > > > > > out
> > >> >> > > > > > > > > >> details.
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > > >> Thanks,
> > >> >> > > > > > > > > >> Harsha
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> > brahmbhatt
> > >> wrote:
> > >> >> > > > > > > > > >> > That is what the hadoop echo system uses so no
> > >> good
> > >> >> > reason
> > >> >> > > > > > really.
> > >> >> > > > > > > > We
> > >> >> > > > > > > > > >> > could
> > >> >> > > > > > > > > >> > change it to whatever is the newest
> recommended
> > >> standard
> > >> >> > > is.
> > >> >> > > > > > > > > >> >
> > >> >> > > > > > > > > >> > Thanks
> > >> >> > > > > > > > > >> > Parth
> > >> >> > > > > > > > > >> >
> > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
> > >> >> > > > > ismael@juma.me.uk
> > >> >> > > > > > >
> > >> >> > > > > > > > > wrote:
> > >> >> > > > > > > > > >> >
> > >> >> > > > > > > > > >> > > Hi Parth,
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing
> > >> this and
> > >> >> > > may
> > >> >> > > > > have
> > >> >> > > > > > > > > >> additional
> > >> >> > > > > > > > > >> > > questions later. The immediate question that
> > >> came to
> > >> >> > > mind
> > >> >> > > > is
> > >> >> > > > > > our
> > >> >> > > > > > > > > >> choice of
> > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
> > >> OBSOLETE in
> > >> >> > the
> > >> >> > > > IANA
> > >> >> > > > > > > > > Registry
> > >> >> > > > > > > > > >> of
> > >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831)
> > has
> > >> been
> > >> >> > > moved
> > >> >> > > > > to
> > >> >> > > > > > > > > Historic
> > >> >> > > > > > > > > >> > > status:
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > >
> > >> >> > > > > >
> > >> >> > >
> > >> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > What is the reasoning behind that choice?
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > Thanks,
> > >> >> > > > > > > > > >> > > Ismael
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> > Shapira <
> > >> >> > > > > > > gwen@confluent.io
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > >> wrote:
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >> > > > Also comments inline :)
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > > * I want to emphasize that even though
> > >> delegation
> > >> >> > > > tokens
> > >> >> > > > > > > are a
> > >> >> > > > > > > > > >> Hadoop
> > >> >> > > > > > > > > >> > > > > innovation, I feel very strongly about
> not
> > >> adding
> > >> >> > > > > > dependency
> > >> >> > > > > > > > on
> > >> >> > > > > > > > > >> Hadoop
> > >> >> > > > > > > > > >> > > > > when implementing delegation tokens for
> > >> Kafka. The
> > >> >> > > KIP
> > >> >> > > > > > > doesn't
> > >> >> > > > > > > > > >> imply
> > >> >> > > > > > > > > >> > > > > such dependency, but if you can
> clarify...
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one
> will
> > >> read
> > >> >> > the
> > >> >> > > > KIP
> > >> >> > > > > > and
> > >> >> > > > > > > > > panic
> > >> >> > > > > > > > > >> > > > three weeks before the next release...
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > > * Can we get delegation token at any
> time
> > >> after
> > >> >> > > > > > > > authenticating?
> > >> >> > > > > > > > > >> only
> > >> >> > > > > > > > > >> > > > > immediately after?
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *As long as you are authenticated you
> can
> > >> get
> > >> >> > > > delegation
> > >> >> > > > > > > > tokens.
> > >> >> > > > > > > > > >> We
> > >> >> > > > > > > > > >> > > need
> > >> >> > > > > > > > > >> > > > to
> > >> >> > > > > > > > > >> > > > > discuss if a client authenticated using
> > >> delegation
> > >> >> > > > > token,
> > >> >> > > > > > > can
> > >> >> > > > > > > > > also
> > >> >> > > > > > > > > >> > > > acquire
> > >> >> > > > > > > > > >> > > > > delegation token again or not. Also
> there
> > >> is the
> > >> >> > > > > question
> > >> >> > > > > > of
> > >> >> > > > > > > > do
> > >> >> > > > > > > > > we
> > >> >> > > > > > > > > >> > > allow
> > >> >> > > > > > > > > >> > > > > anyone to acquire delegation token or we
> > >> want
> > >> >> > > specific
> > >> >> > > > > > ACLs
> > >> >> > > > > > > (I
> > >> >> > > > > > > > > >> think
> > >> >> > > > > > > > > >> > > its
> > >> >> > > > > > > > > >> > > > an
> > >> >> > > > > > > > > >> > > > > overkill.)*
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > I think we are debating two options:
> Either
> > >> require
> > >> >> > > > > Kerberos
> > >> >> > > > > > > > auth
> > >> >> > > > > > > > > >> for
> > >> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
> > >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> > basically
> > >> >> > require
> > >> >> > > a
> > >> >> > > > > "job
> > >> >> > > > > > > > > master"
> > >> >> > > > > > > > > >> > > > to take responsibility for the renewal, it
> > >> will have
> > >> >> > > its
> > >> >> > > > > own
> > >> >> > > > > > > > > >> identity
> > >> >> > > > > > > > > >> > > > anyway and I think this is the correct
> > design
> > >> >> > pattern
> > >> >> > > > > > anyway.
> > >> >> > > > > > > > For
> > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> > >> renewals?),
> > >> >> > but
> > >> >> > > > it
> > >> >> > > > > is
> > >> >> > > > > > > > hard
> > >> >> > > > > > > > > to
> > >> >> > > > > > > > > >> > > > debate simplicity without looking at the
> > code
> > >> >> > changes
> > >> >> > > > > > > required.
> > >> >> > > > > > > > If
> > >> >> > > > > > > > > >> you
> > >> >> > > > > > > > > >> > > > have a draft of how the "require Kerberos"
> > >> will look
> > >> >> > > in
> > >> >> > > > > > Kafka
> > >> >> > > > > > > > > code,
> > >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > > * My understanding is that tokens will
> > >> propagate
> > >> >> > via
> > >> >> > > > ZK
> > >> >> > > > > > but
> > >> >> > > > > > > > > >> without
> > >> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> > >> protocol,
> > >> >> > > > correct?
> > >> >> > > > > > > > Clients
> > >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth
> failure
> > >> (IIRC),
> > >> >> > > but
> > >> >> > > > > > since
> > >> >> > > > > > > > the
> > >> >> > > > > > > > > >> > > > > tokens propagate between brokers asynch,
> > we
> > >> will
> > >> >> > > need
> > >> >> > > > to
> > >> >> > > > > > > > retry a
> > >> >> > > > > > > > > >> bit
> > >> >> > > > > > > > > >> > > > > to avoid clients failing auth due to
> > timing
> > >> >> > issues.
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right
> > now.
> > >> The
> > >> >> > > > current
> > >> >> > > > > > > > > documented
> > >> >> > > > > > > > > >> > > > approach
> > >> >> > > > > > > > > >> > > > > is zookeeper based and it does not
> require
> > >> any
> > >> >> > > changes
> > >> >> > > > > to
> > >> >> > > > > > > > > >> > > UpdateMetadata
> > >> >> > > > > > > > > >> > > > > protocol. An alternative approach can
> > remove
> > >> >> > > zookeeper
> > >> >> > > > > > > > > dependency
> > >> >> > > > > > > > > >> as
> > >> >> > > > > > > > > >> > > well
> > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP
> discussion
> > >> call.*
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to
> > ping
> > >> Jun to
> > >> >> > > > > > arrange a
> > >> >> > > > > > > > > call?
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having
> > >> just the
> > >> >> > > > > > controller
> > >> >> > > > > > > > > issue
> > >> >> > > > > > > > > >> the
> > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a
> > shared
> > >> >> > secret.
> > >> >> > > > Not
> > >> >> > > > > > > sure
> > >> >> > > > > > > > if
> > >> >> > > > > > > > > >> we
> > >> >> > > > > > > > > >> > > > > want to continue the discussion here or
> on
> > >> the
> > >> >> > > wiki. I
> > >> >> > > > > > think
> > >> >> > > > > > > > > that
> > >> >> > > > > > > > > >> we
> > >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> > >> distribution"
> > >> >> > > from
> > >> >> > > > > > > "shared
> > >> >> > > > > > > > > >> secret
> > >> >> > > > > > > > > >> > > > > distribution" and use the controller as
> > the
> > >> only
> > >> >> > > token
> > >> >> > > > > > > > generator
> > >> >> > > > > > > > > >> to
> > >> >> > > > > > > > > >> > > > > solve the second issue, while still
> using
> > >> ZK async
> > >> >> > > to
> > >> >> > > > > > > > distribute
> > >> >> > > > > > > > > >> > > > > tokens.
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am
> > >> fine with
> > >> >> > > > that
> > >> >> > > > > > > > approach
> > >> >> > > > > > > > > >> as
> > >> >> > > > > > > > > >> > > long
> > >> >> > > > > > > > > >> > > > as
> > >> >> > > > > > > > > >> > > > > we agree that the extra complexity of
> > >> >> > > adding/updating
> > >> >> > > > > APIS
> > >> >> > > > > > > > adds
> > >> >> > > > > > > > > >> enough
> > >> >> > > > > > > > > >> > > > > value. The advantage with the controller
> > >> approach
> > >> >> > is
> > >> >> > > > > > secret
> > >> >> > > > > > > > > >> rotation
> > >> >> > > > > > > > > >> > > can
> > >> >> > > > > > > > > >> > > > be
> > >> >> > > > > > > > > >> > > > > automated,frequent and would not require
> > >> >> > > deployment. *
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Can you detail the extra complexity (or
> > point
> > >> me to
> > >> >> > > the
> > >> >> > > > > > email
> > >> >> > > > > > > I
> > >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> > >> >> > > > > > > > > >> > > > As far as I can tell, clients can already
> > >> find the
> > >> >> > > > > > controller
> > >> >> > > > > > > > from
> > >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
> > >> controller
> > >> >> > > > load.
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> > kerberos
> > >> auth
> > >> >> > for
> > >> >> > > > > > > renwal, I
> > >> >> > > > > > > > > >> think
> > >> >> > > > > > > > > >> > > > > it mixes the transport layer the the
> > request
> > >> >> > content
> > >> >> > > > in
> > >> >> > > > > a
> > >> >> > > > > > > > pretty
> > >> >> > > > > > > > > >> ugly
> > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to
> non-owner
> > >> is
> > >> >> > > better.
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While
> > >> this will
> > >> >> > > make
> > >> >> > > > > the
> > >> >> > > > > > > > kafka
> > >> >> > > > > > > > > >> code
> > >> >> > > > > > > > > >> > > > > pretty straight forward , forcing
> renewer
> > >> to
> > >> >> > > > non-owner
> > >> >> > > > > > > pushes
> > >> >> > > > > > > > > >> the code
> > >> >> > > > > > > > > >> > > > > ugliness to client and makes it even
> > harder
> > >> to
> > >> >> > > > > > integrate. *
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> > >> "renewal by
> > >> >> > > > other"
> > >> >> > > > > > > > approach
> > >> >> > > > > > > > > >> is
> > >> >> > > > > > > > > >> > > > that ugly for the clients we expect to use
> > >> >> > delegation
> > >> >> > > > > tokens
> > >> >> > > > > > > > since
> > >> >> > > > > > > > > >> > > > they will have an app-master of some sort
> > who
> > >> >> > > requested
> > >> >> > > > > the
> > >> >> > > > > > > > token
> > >> >> > > > > > > > > to
> > >> >> > > > > > > > > >> > > > begin with.
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > The response for my question on how
> > multiple
> > >> >> > > > identities
> > >> >> > > > > > will
> > >> >> > > > > > > > be
> > >> >> > > > > > > > > >> > > > > handled wasn't super clear to me -
> AFAIK,
> > >> we don't
> > >> >> > > > > > > > authenticate
> > >> >> > > > > > > > > >> each
> > >> >> > > > > > > > > >> > > > > request, we authenticate connections.
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > *We authenticate connections, and only
> > when
> > >> they
> > >> >> > are
> > >> >> > > > > being
> > >> >> > > > > > > > > >> established.
> > >> >> > > > > > > > > >> > > > Let
> > >> >> > > > > > > > > >> > > > > me try to phrase this as a question, in
> > >> absence of
> > >> >> > > > > > > delegation
> > >> >> > > > > > > > > >> tokens if
> > >> >> > > > > > > > > >> > > > we
> > >> >> > > > > > > > > >> > > > > had to support the use case using user
> > >> TGT's how
> > >> >> > > would
> > >> >> > > > > we
> > >> >> > > > > > do
> > >> >> > > > > > > > it?
> > >> >> > > > > > > > > >> My
> > >> >> > > > > > > > > >> > > point
> > >> >> > > > > > > > > >> > > > > was it would be no different with
> > delegation
> > >> >> > tokens.
> > >> >> > > > The
> > >> >> > > > > > use
> > >> >> > > > > > > > > case
> > >> >> > > > > > > > > >> you
> > >> >> > > > > > > > > >> > > are
> > >> >> > > > > > > > > >> > > > > describing seems more like
> impersonation.*
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the things
> that
> > >> >> > delegation
> > >> >> > > > > > tokens
> > >> >> > > > > > > > > >> handled.
> > >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > Gwen
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > > > > Thanks
> > >> >> > > > > > > > > >> > > > > Parth
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
> > >> Shapira <
> > >> >> > > > > > > > > gwen@confluent.io
> > >> >> > > > > > > > > >> >
> > >> >> > > > > > > > > >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >
> > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> Few more comments:
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
> > >> doesn't seem
> > >> >> > to
> > >> >> > > > > have
> > >> >> > > > > > > good
> > >> >> > > > > > > > > >> > > > >> description of the changes to the Kafka
> > >> Protocol.
> > >> >> > > > > Sounds
> > >> >> > > > > > > like
> > >> >> > > > > > > > > >> you are
> > >> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest
> and
> > >> >> > > > > > RenewTokenRequest
> > >> >> > > > > > > > (and
> > >> >> > > > > > > > > >> > > > >> matching responses), without detailing
> > the
> > >> >> > contents
> > >> >> > > > of
> > >> >> > > > > > the
> > >> >> > > > > > > > > >> requests
> > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you show the
> > >> class
> > >> >> > > > interface,
> > >> >> > > > > > but
> > >> >> > > > > > > > not
> > >> >> > > > > > > > > >> the
> > >> >> > > > > > > > > >> > > > >> underlying protocol. This could be seen
> > as
> > >> an
> > >> >> > > > > > > implementation
> > >> >> > > > > > > > > >> detail,
> > >> >> > > > > > > > > >> > > > >> but since the binary protocol is what
> we
> > >> provide
> > >> >> > to
> > >> >> > > > > > > non-Java
> > >> >> > > > > > > > > >> clients,
> > >> >> > > > > > > > > >> > > > >> we need to show the changes there.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> > >> >> > > > > > > > delegationTokenRequestHandler?
> > >> >> > > > > > > > > >> Is it
> > >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or
> > Client?
> > >> Its
> > >> >> > > > > unclear...
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even though
> > >> delegation
> > >> >> > > > > tokens
> > >> >> > > > > > > are
> > >> >> > > > > > > > a
> > >> >> > > > > > > > > >> Hadoop
> > >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly about
> > not
> > >> adding
> > >> >> > > > > > > dependency
> > >> >> > > > > > > > on
> > >> >> > > > > > > > > >> Hadoop
> > >> >> > > > > > > > > >> > > > >> when implementing delegation tokens for
> > >> Kafka.
> > >> >> > The
> > >> >> > > > KIP
> > >> >> > > > > > > > doesn't
> > >> >> > > > > > > > > >> imply
> > >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> > clarify...
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * Can we get delegation token at any
> time
> > >> after
> > >> >> > > > > > > > authenticating?
> > >> >> > > > > > > > > >> only
> > >> >> > > > > > > > > >> > > > >> immediately after?
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * My understanding is that tokens will
> > >> propagate
> > >> >> > > via
> > >> >> > > > ZK
> > >> >> > > > > > but
> > >> >> > > > > > > > > >> without
> > >> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
> > >> protocol,
> > >> >> > > > correct?
> > >> >> > > > > > > > Clients
> > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth
> > failure
> > >> >> > (IIRC),
> > >> >> > > > but
> > >> >> > > > > > > since
> > >> >> > > > > > > > > the
> > >> >> > > > > > > > > >> > > > >> tokens propagate between brokers
> asynch,
> > >> we will
> > >> >> > > need
> > >> >> > > > > to
> > >> >> > > > > > > > retry
> > >> >> > > > > > > > > a
> > >> >> > > > > > > > > >> bit
> > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to
> > timing
> > >> >> > issues.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
> > >> touching ZK
> > >> >> > > > directly
> > >> >> > > > > > :)
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having
> > >> just the
> > >> >> > > > > > controller
> > >> >> > > > > > > > > >> issue the
> > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a
> > >> shared
> > >> >> > > secret.
> > >> >> > > > > Not
> > >> >> > > > > > > sure
> > >> >> > > > > > > > > if
> > >> >> > > > > > > > > >> we
> > >> >> > > > > > > > > >> > > > >> want to continue the discussion here or
> > on
> > >> the
> > >> >> > > wiki.
> > >> >> > > > I
> > >> >> > > > > > > think
> > >> >> > > > > > > > > >> that we
> > >> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> > >> distribution"
> > >> >> > > from
> > >> >> > > > > > > "shared
> > >> >> > > > > > > > > >> secret
> > >> >> > > > > > > > > >> > > > >> distribution" and use the controller as
> > >> the only
> > >> >> > > > token
> > >> >> > > > > > > > > generator
> > >> >> > > > > > > > > >> to
> > >> >> > > > > > > > > >> > > > >> solve the second issue, while still
> using
> > >> ZK
> > >> >> > async
> > >> >> > > to
> > >> >> > > > > > > > > distribute
> > >> >> > > > > > > > > >> > > > >> tokens.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite
> > >> lifetime
> > >> >> > of
> > >> >> > > > > > tokens
> > >> >> > > > > > > > (and
> > >> >> > > > > > > > > >> hoped
> > >> >> > > > > > > > > >> > > > >> to hear from others in the community) -
> > but
> > >> >> > having
> > >> >> > > > the
> > >> >> > > > > > > option
> > >> >> > > > > > > > > and
> > >> >> > > > > > > > > >> > > > >> documenting it as less secure, allows
> > >> users to
> > >> >> > > > > configure
> > >> >> > > > > > > > their
> > >> >> > > > > > > > > >> system
> > >> >> > > > > > > > > >> > > > >> as the wish.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
> > >> kerberos auth
> > >> >> > > for
> > >> >> > > > > > > renwal,
> > >> >> > > > > > > > I
> > >> >> > > > > > > > > >> think
> > >> >> > > > > > > > > >> > > > >> it mixes the transport layer the the
> > >> request
> > >> >> > > content
> > >> >> > > > > in a
> > >> >> > > > > > > > > pretty
> > >> >> > > > > > > > > >> ugly
> > >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> > non-owner
> > >> is
> > >> >> > > better.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * More detailed explanation on what we
> > >> plan to do
> > >> >> > > for
> > >> >> > > > > the
> > >> >> > > > > > > > java
> > >> >> > > > > > > > > >> clients
> > >> >> > > > > > > > > >> > > > >> specifically - new configuration? new
> > APIs?
> > >> >> > > > > > > > > >> > > > >> The response for my question on how
> > >> multiple
> > >> >> > > > identities
> > >> >> > > > > > > will
> > >> >> > > > > > > > be
> > >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me -
> AFAIK,
> > >> we
> > >> >> > don't
> > >> >> > > > > > > > authenticate
> > >> >> > > > > > > > > >> each
> > >> >> > > > > > > > > >> > > > >> request, we authenticate connections.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are
> > only
> > >> used
> > >> >> > in
> > >> >> > > > the
> > >> >> > > > > > > Hadoop
> > >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> > >> >> > alternatives
> > >> >> > > in
> > >> >> > > > > > other
> > >> >> > > > > > > > > >> ecosystems
> > >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and
> whether
> > >> there
> > >> >> > are
> > >> >> > > > some
> > >> >> > > > > > > > > >> advantages
> > >> >> > > > > > > > > >> > > > >> there.
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> Gwen
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM,
> Harsha <
> > >> >> > > > > kafka@harsha.io
> > >> >> > > > > > >
> > >> >> > > > > > > > > wrote:
> > >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> > >> >> > > > > > > > > >> > > > >> > Can you look at Parth's
> last
> > >> reply.
> > >> >> > > Does
> > >> >> > > > > it
> > >> >> > > > > > > > answer
> > >> >> > > > > > > > > >> your
> > >> >> > > > > > > > > >> > > > >> > concerns.
> > >> >> > > > > > > > > >> > > > >> >
> > >> >> > > > > > > > > >> > > > >> > Thanks,
> > >> >> > > > > > > > > >> > > > >> > Harsha
> > >> >> > > > > > > > > >> > > > >> >
> > >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM,
> parth
> > >> >> > > brahmbhatt
> > >> >> > > > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki
> > >> already
> > >> >> > has
> > >> >> > > > > > details
> > >> >> > > > > > > on
> > >> >> > > > > > > > > >> token
> > >> >> > > > > > > > > >> > > > >> >> expiration
> > >> >> > > > > > > > > >> > > > >> >> under token acquisition process
> > >> >> > > > > > > > > >> > > > >> >> <
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> > >> >> > > > > > > > > >> > > > >> >.
> > >> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will
> > >> expire
> > >> >> > > based
> > >> >> > > > > on a
> > >> >> > > > > > > > > server
> > >> >> > > > > > > > > >> side
> > >> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours)
> > unless
> > >> >> > renewed.
> > >> >> > > > > > Renewal
> > >> >> > > > > > > > is
> > >> >> > > > > > > > > >> only
> > >> >> > > > > > > > > >> > > > allowed
> > >> >> > > > > > > > > >> > > > >> >> until the max life time of token.
> > >> >> > Alternatively
> > >> >> > > we
> > >> >> > > > > > could
> > >> >> > > > > > > > > also
> > >> >> > > > > > > > > >> make
> > >> >> > > > > > > > > >> > > > that
> > >> >> > > > > > > > > >> > > > >> >> an
> > >> >> > > > > > > > > >> > > > >> >> optional param and the server side
> > >> default can
> > >> >> > > > serve
> > >> >> > > > > > as
> > >> >> > > > > > > > the
> > >> >> > > > > > > > > >> upper
> > >> >> > > > > > > > > >> > > > bound.
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >> To your second point it will be done
> > >> exactly
> > >> >> > the
> > >> >> > > > > same
> > >> >> > > > > > > way
> > >> >> > > > > > > > we
> > >> >> > > > > > > > > >> would
> > >> >> > > > > > > > > >> > > > >> >> support
> > >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client
> > >> will have
> > >> >> > > to
> > >> >> > > > > put
> > >> >> > > > > > > the
> > >> >> > > > > > > > > >> tokens it
> > >> >> > > > > > > > > >> > > > >> wants
> > >> >> > > > > > > > > >> > > > >> >> to use in the subject instance and
> > call
> > >> >> > > > > > produce/consume
> > >> >> > > > > > > > > inside
> > >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have
> to
> > >> keep
> > >> >> > > track
> > >> >> > > > of
> > >> >> > > > > > its
> > >> >> > > > > > > > own
> > >> >> > > > > > > > > >> > > > subject. I
> > >> >> > > > > > > > > >> > > > >> >> will have to look at the code to see
> > if
> > >> we
> > >> >> > > support
> > >> >> > > > > > this
> > >> >> > > > > > > > > >> feature
> > >> >> > > > > > > > > >> > > right
> > >> >> > > > > > > > > >> > > > >> now
> > >> >> > > > > > > > > >> > > > >> >> but my understanding is delegation
> > token
> > >> >> > > shouldn't
> > >> >> > > > > > need
> > >> >> > > > > > > > any
> > >> >> > > > > > > > > >> special
> > >> >> > > > > > > > > >> > > > >> >> treatment as its just another type
> of
> > >> >> > Credential
> > >> >> > > > in
> > >> >> > > > > > the
> > >> >> > > > > > > > > >> subject.
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >> I would also like to know what is
> your
> > >> opinion
> > >> >> > > > about
> > >> >> > > > > > > > > infinite
> > >> >> > > > > > > > > >> > > renewal
> > >> >> > > > > > > > > >> > > > >> (my
> > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> > this),
> > >> tokens
> > >> >> > > > > > renewing
> > >> >> > > > > > > > them
> > >> >> > > > > > > > > >> > > self(my
> > >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> this)
> > >> and
> > >> >> > most
> > >> >> > > > > > > > importantly
> > >> >> > > > > > > > > >> your
> > >> >> > > > > > > > > >> > > > choice
> > >> >> > > > > > > > > >> > > > >> >> between the alternatives listed on
> > this
> > >> thread
> > >> >> > > > > > > > > >> > > > >> >> <
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >>
> >
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> > >> >> > > > > > > > > >> > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> > >> alternative-2 minus
> > >> >> > > > > > > controller
> > >> >> > > > > > > > > >> > > > distributing
> > >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >> Thanks
> > >> >> > > > > > > > > >> > > > >> >> Parth
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen
> > >> Shapira <
> > >> >> > > > > > > > > >> gwen@confluent.io>
> > >> >> > > > > > > > > >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >>
> > >> >> > > > > > > > > >> > > > >> >> > Harsha,
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy.
> I
> > >> didn't
> > >> >> > see
> > >> >> > > > > your
> > >> >> > > > > > > > design
> > >> >> > > > > > > > > >> yet,
> > >> >> > > > > > > > > >> > > > but in
> > >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> > >> producers, which
> > >> >> > > > will
> > >> >> > > > > > > serve
> > >> >> > > > > > > > > >> multiple
> > >> >> > > > > > > > > >> > > > users
> > >> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since
> these
> > >> users
> > >> >> > > will
> > >> >> > > > > have
> > >> >> > > > > > > > > >> different
> > >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> > >> authenticate
> > >> >> > > > > separately,
> > >> >> > > > > > > and
> > >> >> > > > > > > > > >> can't
> > >> >> > > > > > > > > >> > > > share a
> > >> >> > > > > > > > > >> > > > >> >> > token.
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > Gwen
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM,
> > >> Harsha <
> > >> >> > > > > > > kafka@harsha.io
> > >> >> > > > > > > > >
> > >> >> > > > > > > > > >> wrote:
> > >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> > >> >> > > > > > > > > >> > > > >> >> > > On your second point.
> > >> Can you
> > >> >> > > > > describe
> > >> >> > > > > > a
> > >> >> > > > > > > > > >> usecase
> > >> >> > > > > > > > > >> > > where
> > >> >> > > > > > > > > >> > > > >> >> > > mutliple clients
> ended
> > up
> > >> >> > > sharing a
> > >> >> > > > > > > > producer
> > >> >> > > > > > > > > >> and
> > >> >> > > > > > > > > >> > > even
> > >> >> > > > > > > > > >> > > > if
> > >> >> > > > > > > > > >> > > > >> they
> > >> >> > > > > > > > > >> > > > >> >> > > do why can't they not
> > use
> > >> >> > single
> > >> >> > > > > token
> > >> >> > > > > > > that
> > >> >> > > > > > > > > >> producer
> > >> >> > > > > > > > > >> > > > >> >> > > captures. Why would
> we
> > >> need
> > >> >> > > > multiple
> > >> >> > > > > > > > clients
> > >> >> > > > > > > > > >> with
> > >> >> > > > > > > > > >> > > > >> different
> > >> >> > > > > > > > > >> > > > >> >> > > tokens sharing a
> single
> > >> >> > instance
> > >> >> > > of
> > >> >> > > > > > > > producer.
> > >> >> > > > > > > > > >> Also
> > >> >> > > > > > > > > >> > > in
> > >> >> > > > > > > > > >> > > > >> this
> > >> >> > > > > > > > > >> > > > >> >> > > case other clients
> have
> > >> access
> > >> >> > > all
> > >> >> > > > > the
> > >> >> > > > > > > > tokens
> > >> >> > > > > > > > > >> no?
> > >> >> > > > > > > > > >> > > > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> > >> >> > > > > > > > > >> > > > >> >> > > Harsha
> > >> >> > > > > > > > > >> > > > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49
> AM,
> > >> Gwen
> > >> >> > > Shapira
> > >> >> > > > > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> > >> >> > > > > > > > > >> > > > >> >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't
> see
> > >> in the
> > >> >> > > wiki:
> > >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
> > >> delegation
> > >> >> > > > > tokens?
> > >> >> > > > > > > > > >> Renewal? How
> > >> >> > > > > > > > > >> > > > do we
> > >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> > >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation
> > >> tokens
> > >> >> > for
> > >> >> > > > > > "do-as"
> > >> >> > > > > > > > > (say,
> > >> >> > > > > > > > > >> > > submit
> > >> >> > > > > > > > > >> > > > >> Storm
> > >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a
> > >> producer
> > >> >> > > for
> > >> >> > > > > > every
> > >> >> > > > > > > > job
> > >> >> > > > > > > > > >> (we
> > >> >> > > > > > > > > >> > > can't
> > >> >> > > > > > > > > >> > > > >> share
> > >> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs
> > running
> > >> on
> > >> >> > same
> > >> >> > > > > node),
> > >> >> > > > > > > > since
> > >> >> > > > > > > > > >> we
> > >> >> > > > > > > > > >> > > only
> > >> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting.
> Is
> > >> there a
> > >> >> > > plan
> > >> >> > > > > to
> > >> >> > > > > > > > change
> > >> >> > > > > > > > > >> this
> > >> >> > > > > > > > > >> > > for
> > >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to
> > >> allow
> > >> >> > > multiple
> > >> >> > > > > > users
> > >> >> > > > > > > > > with
> > >> >> > > > > > > > > >> > > > different
> > >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> > >> >> > > > > > > > > >> > > > >> >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> > >> >> > > > > > > > > >> > > > >> >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM,
> > >> parth
> > >> >> > > > > brahmbhatt
> > >> >> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com>
> > >> wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more
> time,
> > >> can
> > >> >> > other
> > >> >> > > > > > > committers
> > >> >> > > > > > > > > >> review?
> > >> >> > > > > > > > > >> > > > >> >> > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> > >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> > >> >> > > > > > > > > >> > > > >> >> > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07
> > AM,
> > >> >> > Harsha <
> > >> >> > > > > > > > > >> kafka@harsha.io>
> > >> >> > > > > > > > > >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> Overall current
> > >> design looks
> > >> >> > > > good
> > >> >> > > > > to
> > >> >> > > > > > > > me. I
> > >> >> > > > > > > > > >> am +1
> > >> >> > > > > > > > > >> > > on
> > >> >> > > > > > > > > >> > > > >> the
> > >> >> > > > > > > > > >> > > > >> >> > KIP.
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review
> this
> > >> as
> > >> >> > well.
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at
> 09:57
> > >> AM,
> > >> >> > parth
> > >> >> > > > > > > > brahmbhatt
> > >> >> > > > > > > > > >> wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review
> Jitendra.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
> > >> infinite
> > >> >> > > > lifetime
> > >> >> > > > > > > but I
> > >> >> > > > > > > > > >> see the
> > >> >> > > > > > > > > >> > > > >> Streaming
> > >> >> > > > > > > > > >> > > > >> >> > use
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming
> use
> > >> case I
> > >> >> > > was
> > >> >> > > > > > hoping
> > >> >> > > > > > > > > >> there will
> > >> >> > > > > > > > > >> > > > be
> > >> >> > > > > > > > > >> > > > >> some
> > >> >> > > > > > > > > >> > > > >> >> > notion
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get
> > new
> > >> >> > > > delegation
> > >> >> > > > > > > tokens
> > >> >> > > > > > > > > at
> > >> >> > > > > > > > > >> fixed
> > >> >> > > > > > > > > >> > > > >> interval
> > >> >> > > > > > > > > >> > > > >> >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If
> > >> that is
> > >> >> > not
> > >> >> > > > the
> > >> >> > > > > > case
> > >> >> > > > > > > > for
> > >> >> > > > > > > > > >> we can
> > >> >> > > > > > > > > >> > > > >> discuss
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing
> > >> them self
> > >> >> > > and
> > >> >> > > > > the
> > >> >> > > > > > > > > >> security
> > >> >> > > > > > > > > >> > > > >> implications
> > >> >> > > > > > > > > >> > > > >> >> > of the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to
> > >> fetch
> > >> >> > tokens
> > >> >> > > > > from
> > >> >> > > > > > > > > >> zookeeper,
> > >> >> > > > > > > > > >> > > > >> overall I
> > >> >> > > > > > > > > >> > > > >> >> > think
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't
> rely
> > >> on our
> > >> >> > > > > metadata
> > >> >> > > > > > > > store
> > >> >> > > > > > > > > >> and I
> > >> >> > > > > > > > > >> > > > >> think we
> > >> >> > > > > > > > > >> > > > >> >> > are
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction
> > with
> > >> all
> > >> >> > the
> > >> >> > > > > KIP-4
> > >> >> > > > > > > > > >> improvements.
> > >> >> > > > > > > > > >> > > > I
> > >> >> > > > > > > > > >> > > > >> chose
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case
> > the
> > >> client
> > >> >> > > > will
> > >> >> > > > > > > still
> > >> >> > > > > > > > > >> just talk
> > >> >> > > > > > > > > >> > > > to
> > >> >> > > > > > > > > >> > > > >> >> > broker , its
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use
> > >> zookeeper
> > >> >> > > > which
> > >> >> > > > > we
> > >> >> > > > > > > > > >> already do
> > >> >> > > > > > > > > >> > > > for a
> > >> >> > > > > > > > > >> > > > >> lot
> > >> >> > > > > > > > > >> > > > >> >> > of
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> > >> development + and
> > >> >> > > the
> > >> >> > > > > > > ability
> > >> >> > > > > > > > > so
> > >> >> > > > > > > > > >> > > tokens
> > >> >> > > > > > > > > >> > > > >> will
> > >> >> > > > > > > > > >> > > > >> >> > survive
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> > restart/cluster
> > >> >> > > failure.
> > >> >> > > > > if a
> > >> >> > > > > > > > > >> majority
> > >> >> > > > > > > > > >> > > > agrees
> > >> >> > > > > > > > > >> > > > >> the
> > >> >> > > > > > > > > >> > > > >> >> > added
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> > controller
> > >> >> > > forwarding
> > >> >> > > > > > keys
> > >> >> > > > > > > to
> > >> >> > > > > > > > > all
> > >> >> > > > > > > > > >> > > > broker is
> > >> >> > > > > > > > > >> > > > >> >> > justified
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter
> security
> > >> , I am
> > >> >> > > fine
> > >> >> > > > > > with
> > >> >> > > > > > > > that
> > >> >> > > > > > > > > >> option
> > >> >> > > > > > > > > >> > > > too.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
> > >> support SSL
> > >> >> > > we
> > >> >> > > > > can
> > >> >> > > > > > > not
> > >> >> > > > > > > > > >> store
> > >> >> > > > > > > > > >> > > > master
> > >> >> > > > > > > > > >> > > > >> keys
> > >> >> > > > > > > > > >> > > > >> >> > in
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys
> > will
> > >> be
> > >> >> > > exposed
> > >> >> > > > > on
> > >> >> > > > > > > > wire.
> > >> >> > > > > > > > > To
> > >> >> > > > > > > > > >> > > > support
> > >> >> > > > > > > > > >> > > > >> >> > rotation
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current
> > >> clients is
> > >> >> > > > > > > something I
> > >> >> > > > > > > > > >> need to
> > >> >> > > > > > > > > >> > > > put
> > >> >> > > > > > > > > >> > > > >> more
> > >> >> > > > > > > > > >> > > > >> >> > thought
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
> > >> assumes the
> > >> >> > > > > rotation
> > >> >> > > > > > > > will
> > >> >> > > > > > > > > >> > > > invalidate
> > >> >> > > > > > > > > >> > > > >> all
> > >> >> > > > > > > > > >> > > > >> >> > current
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to
> also
> > >> review
> > >> >> > > and
> > >> >> > > > > post
> > >> >> > > > > > > > their
> > >> >> > > > > > > > > >> > > comments
> > >> >> > > > > > > > > >> > > > >> so we
> > >> >> > > > > > > > > >> > > > >> >> > can
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at
> 8:39
> > >> AM,
> > >> >> > > Ashish
> > >> >> > > > > > Singh
> > >> >> > > > > > > <
> > >> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> > >> >> > > > > > > > > >> > > > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at
> > >> 11:26 AM,
> > >> >> > > > > Harsha <
> > >> >> > > > > > > > > >> > > > kafka@harsha.io>
> > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> > >> discussion
> > >> >> > > threads
> > >> >> > > > on
> > >> >> > > > > > > this
> > >> >> > > > > > > > > KIP.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response
> > from
> > >> >> > Jitendra
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large
> > >> number of
> > >> >> > > > > clients
> > >> >> > > > > > > that
> > >> >> > > > > > > > > are
> > >> >> > > > > > > > > >> > > > running
> > >> >> > > > > > > > > >> > > > >> all
> > >> >> > > > > > > > > >> > > > >> >> > over the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> > authenticate
> > >> with
> > >> >> > > Kafka
> > >> >> > > > > > > > brokers,
> > >> >> > > > > > > > > >> is very
> > >> >> > > > > > > > > >> > > > >> similar
> > >> >> > > > > > > > > >> > > > >> >> > to the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of
> large
> > >> number
> > >> >> > of
> > >> >> > > > > tasks
> > >> >> > > > > > > > > running
> > >> >> > > > > > > > > >> > > across
> > >> >> > > > > > > > > >> > > > >> the
> > >> >> > > > > > > > > >> > > > >> >> > cluster
> > >> >> > > > > > > > > >> > > > >> >> > >> >> that
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to
> > >> Hdfs
> > >> >> > > > Namenode.
> > >> >> > > > > > > > > >> Therefore, the
> > >> >> > > > > > > > > >> > > > >> >> > delegation token
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem
> like
> > a
> > >> good
> > >> >> > fit
> > >> >> > > > for
> > >> >> > > > > > > this
> > >> >> > > > > > > > > use
> > >> >> > > > > > > > > >> case
> > >> >> > > > > > > > > >> > > > as we
> > >> >> > > > > > > > > >> > > > >> >> > have seen
> > >> >> > > > > > > > > >> > > > >> >> > >> >> it
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale
> > in
> > >> HDFS
> > >> >> > and
> > >> >> > > > > YARN.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design
> is
> > >> very
> > >> >> > much
> > >> >> > > > > > inline
> > >> >> > > > > > > > with
> > >> >> > > > > > > > > >> Hadoop
> > >> >> > > > > > > > > >> > > > >> >> > approach. A few
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys
> want
> > >> to allow
> > >> >> > > > > > infinite
> > >> >> > > > > > > > > >> renewable
> > >> >> > > > > > > > > >> > > > >> lifetime
> > >> >> > > > > > > > > >> > > > >> >> > for a
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts
> a
> > >> token
> > >> >> > to a
> > >> >> > > > max
> > >> >> > > > > > > life
> > >> >> > > > > > > > > time
> > >> >> > > > > > > > > >> > > > (default
> > >> >> > > > > > > > > >> > > > >> 7
> > >> >> > > > > > > > > >> > > > >> >> > days). A
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability
> is
> > >> >> > believed
> > >> >> > > to
> > >> >> > > > > > > > increase
> > >> >> > > > > > > > > >> with
> > >> >> > > > > > > > > >> > > > time.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having
> > infinite
> > >> >> > > lifetime
> > >> >> > > > > > might
> > >> >> > > > > > > > not
> > >> >> > > > > > > > > >> be the
> > >> >> > > > > > > > > >> > > > best
> > >> >> > > > > > > > > >> > > > >> idea.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the
> > >> tokens
> > >> >> > are
> > >> >> > > > > stored
> > >> >> > > > > > > in
> > >> >> > > > > > > > > >> zookeeper
> > >> >> > > > > > > > > >> > > > as
> > >> >> > > > > > > > > >> > > > >> well,
> > >> >> > > > > > > > > >> > > > >> >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> can
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This
> > is
> > >> clever
> > >> >> > > as
> > >> >> > > > it
> > >> >> > > > > > can
> > >> >> > > > > > > > > allow
> > >> >> > > > > > > > > >> > > > >> replacing the
> > >> >> > > > > > > > > >> > > > >> >> > tokens
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of
> max
> > >> life
> > >> >> > > time,
> > >> >> > > > > and
> > >> >> > > > > > > > > clients
> > >> >> > > > > > > > > >> can
> > >> >> > > > > > > > > >> > > > >> download
> > >> >> > > > > > > > > >> > > > >> >> > new
> > >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> > >> shouldn't be
> > >> >> > a
> > >> >> > > > big
> > >> >> > > > > > load
> > >> >> > > > > > > > on
> > >> >> > > > > > > > > >> > > zookeeper
> > >> >> > > > > > > > > >> > > > >> as a
> > >> >> > > > > > > > > >> > > > >> >> > client
> > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new
> token
> > >> once in
> > >> >> > > > > several
> > >> >> > > > > > > > days.
> > >> >> > > > > > > > > >> In this
> > >> >> > > > > > > > > >> > > > >> approach
> > >> >> > > > > > > > > >> > > > >> >> > you
> > >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime
> > on
> > >> the
> > >> >> > > token
> > >> >> > > > > even
> > >> >> > > > > > > for
> > >> >> > > > > > > > > >> long
> > >> >> > > > > > > > > >> > > > running
> > >> >> > > > > > > > > >> > > > >> >> > clients.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password
> > are
> > >> >> > > generated
> > >> >> > > > > > using
> > >> >> > > > > > > a
> > >> >> > > > > > > > > >> master
> > >> >> > > > > > > > > >> > > key.
> > >> >> > > > > > > > > >> > > > >> The
> > >> >> > > > > > > > > >> > > > >> >> > master
> > >> >> > > > > > > > > >> > > > >> >> > >> >> key
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> > >> periodically
> > >> >> > > changed.
> > >> >> > > > > In
> > >> >> > > > > > > > > Hadoop,
> > >> >> > > > > > > > > >> the
> > >> >> > > > > > > > > >> > > > >> default
> > >> >> > > > > > > > > >> > > > >> >> > renewal
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require
> > >> brokers
> > >> >> > > > > > maintaining
> > >> >> > > > > > > a
> > >> >> > > > > > > > > >> list of X
> > >> >> > > > > > > > > >> > > > most
> > >> >> > > > > > > > > >> > > > >> >> > recent
> > >> >> > > > > > > > > >> > > > >> >> > >> >> master
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will
> have
> > >> to be
> > >> >> > > > > persisted
> > >> >> > > > > > > > > >> somewhere, as
> > >> >> > > > > > > > > >> > > > if a
> > >> >> > > > > > > > > >> > > > >> >> > broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get
> > >> that list
> > >> >> > > > again
> > >> >> > > > > > and
> > >> >> > > > > > > > > >> storing
> > >> >> > > > > > > > > >> > > > master
> > >> >> > > > > > > > > >> > > > >> keys
> > >> >> > > > > > > > > >> > > > >> >> > on ZK
> > >> >> > > > > > > > > >> > > > >> >> > >> >> is
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> > However,
> > >> if a
> > >> >> > > > broker
> > >> >> > > > > > goes
> > >> >> > > > > > > > > down
> > >> >> > > > > > > > > >> then
> > >> >> > > > > > > > > >> > > we
> > >> >> > > > > > > > > >> > > > >> have
> > >> >> > > > > > > > > >> > > > >> >> > much
> > >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
> > >> client can
> > >> >> > > > always
> > >> >> > > > > > > > > >> > > re-authenticate
> > >> >> > > > > > > > > >> > > > is
> > >> >> > > > > > > > > >> > > > >> such
> > >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a
> > >> look at
> > >> >> > > other
> > >> >> > > > > > > > > >> alternatives
> > >> >> > > > > > > > > >> > > this
> > >> >> > > > > > > > > >> > > > >> list has
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough
> > >> proposal,
> > >> >> > > > great
> > >> >> > > > > > > work!"
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016,
> at
> > >> 10:28
> > >> >> > PM,
> > >> >> > > > Gwen
> > >> >> > > > > > > > Shapira
> > >> >> > > > > > > > > >> wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
> > >> Thanks!
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016
> at
> > >> 9:25
> > >> >> > PM,
> > >> >> > > > > > Harsha <
> > >> >> > > > > > > > > >> > > > kafka@harsha.io
> > >> >> > > > > > > > > >> > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need
> any
> > >> release
> > >> >> > > > > vehicle
> > >> >> > > > > > > but
> > >> >> > > > > > > > > >> still the
> > >> >> > > > > > > > > >> > > > >> work can
> > >> >> > > > > > > > > >> > > > >> >> > move
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> > >> interested in
> > >> >> > the
> > >> >> > > > KIP
> > >> >> > > > > > > > please
> > >> >> > > > > > > > > >> do the
> > >> >> > > > > > > > > >> > > > >> review and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7,
> 2016,
> > >> at
> > >> >> > 04:59
> > >> >> > > > PM,
> > >> >> > > > > > > Ismael
> > >> >> > > > > > > > > >> Juma
> > >> >> > > > > > > > > >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it
> > >> would be
> > >> >> > good
> > >> >> > > > to
> > >> >> > > > > > have
> > >> >> > > > > > > > > more
> > >> >> > > > > > > > > >> time
> > >> >> > > > > > > > > >> > > to
> > >> >> > > > > > > > > >> > > > >> review
> > >> >> > > > > > > > > >> > > > >> >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8,
> 2016
> > >> at
> > >> >> > 12:55
> > >> >> > > > AM,
> > >> >> > > > > > Gwen
> > >> >> > > > > > > > > >> Shapira <
> > >> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
> > >> depends on
> > >> >> > > > KIP-43,
> > >> >> > > > > > > which
> > >> >> > > > > > > > > is
> > >> >> > > > > > > > > >> > > > already a
> > >> >> > > > > > > > > >> > > > >> bit
> > >> >> > > > > > > > > >> > > > >> >> > of a
> > >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next
> release
> > -
> > >> any
> > >> >> > > chance
> > >> >> > > > > we
> > >> >> > > > > > > can
> > >> >> > > > > > > > > >> delay
> > >> >> > > > > > > > > >> > > > >> delegation
> > >> >> > > > > > > > > >> > > > >> >> > tokens
> > >> >> > > > > > > > > >> > > > >> >> > >> >> to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> > community
> > >> >> > working
> > >> >> > > > on a
> > >> >> > > > > > > > release
> > >> >> > > > > > > > > >> every
> > >> >> > > > > > > > > >> > > 3
> > >> >> > > > > > > > > >> > > > >> month,
> > >> >> > > > > > > > > >> > > > >> >> > this
> > >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26,
> > >> 2016 at
> > >> >> > > 5:11
> > >> >> > > > > PM,
> > >> >> > > > > > > > Ashish
> > >> >> > > > > > > > > >> Singh
> > >> >> > > > > > > > > >> > > <
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again
> > for
> > >> the
> > >> >> > > > awesome
> > >> >> > > > > > > write
> > >> >> > > > > > > > > up.
> > >> >> > > > > > > > > >> > > > Following
> > >> >> > > > > > > > > >> > > > >> our
> > >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think
> > it
> > >> will
> > >> >> > be
> > >> >> > > > > easier
> > >> >> > > > > > > to
> > >> >> > > > > > > > > >> compare
> > >> >> > > > > > > > > >> > > > >> various
> > >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> > together.
> > >> I am
> > >> >> > > > stating
> > >> >> > > > > > > > below a
> > >> >> > > > > > > > > >> few
> > >> >> > > > > > > > > >> > > > >> >> > alternatives along
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> > proposal.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
> > >> proposal)
> > >> >> > Store
> > >> >> > > > > > > Delegation
> > >> >> > > > > > > > > >> Token,
> > >> >> > > > > > > > > >> > > DT,
> > >> >> > > > > > > > > >> > > > >> on ZK.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > >> >> > > authenticates
> > >> >> > > > > > with a
> > >> >> > > > > > > > > >> broker.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> > >> client is
> > >> >> > > > > > > > authenticated,
> > >> >> > > > > > > > > >> it
> > >> >> > > > > > > > > >> > > will
> > >> >> > > > > > > > > >> > > > >> make a
> > >> >> > > > > > > > > >> > > > >> >> > broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > >> delegation
> > >> >> > > > token.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > broker
> > >> >> > > generates
> > >> >> > > > a
> > >> >> > > > > > > shared
> > >> >> > > > > > > > > >> secret
> > >> >> > > > > > > > > >> > > > based
> > >> >> > > > > > > > > >> > > > >> on
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> Password/Secret
> > >> >> > shared
> > >> >> > > > > > between
> > >> >> > > > > > > > all
> > >> >> > > > > > > > > >> > > brokers,
> > >> >> > > > > > > > > >> > > > >> >> > randomly
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker
> > >> stores
> > >> >> > this
> > >> >> > > > > token
> > >> >> > > > > > in
> > >> >> > > > > > > > its
> > >> >> > > > > > > > > >> in
> > >> >> > > > > > > > > >> > > > memory
> > >> >> > > > > > > > > >> > > > >> cache.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > >> DelegationToken
> > >> >> > > > > without
> > >> >> > > > > > > the
> > >> >> > > > > > > > > >> hmac in
> > >> >> > > > > > > > > >> > > the
> > >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All
> > >> brokers will
> > >> >> > > > have a
> > >> >> > > > > > > cache
> > >> >> > > > > > > > > >> backed
> > >> >> > > > > > > > > >> > > by
> > >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> > >> >> > > > > > > > > >> > > > >> >> > >> >> they
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get
> notified
> > >> >> > whenever
> > >> >> > > a
> > >> >> > > > > new
> > >> >> > > > > > > > token
> > >> >> > > > > > > > > is
> > >> >> > > > > > > > > >> > > > >> generated and
> > >> >> > > > > > > > > >> > > > >> >> > they
> > >> >> > > > > > > > > >> > > > >> >> > >> >> will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local
> cache
> > >> whenever
> > >> >> > > > token
> > >> >> > > > > > > state
> > >> >> > > > > > > > > >> changes.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker
> > >> returns
> > >> >> > the
> > >> >> > > > > token
> > >> >> > > > > > to
> > >> >> > > > > > > > > >> Client.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> issues
> > >> and
> > >> >> > fixes
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1.
> Probable
> > >> race
> > >> >> > > > > condition,
> > >> >> > > > > > > > client
> > >> >> > > > > > > > > >> tries
> > >> >> > > > > > > > > >> > > to
> > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > >> >> > > > > > > > > >> > > > >> >> > >> >> with
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is
> yet
> > >> to be
> > >> >> > > > updated
> > >> >> > > > > > with
> > >> >> > > > > > > > the
> > >> >> > > > > > > > > >> newly
> > >> >> > > > > > > > > >> > > > >> generated
> > >> >> > > > > > > > > >> > > > >> >> > DT.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> This
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with
> > >> making
> > >> >> > > > > dtRequest
> > >> >> > > > > > > > block
> > >> >> > > > > > > > > >> until
> > >> >> > > > > > > > > >> > > all
> > >> >> > > > > > > > > >> > > > >> >> > brokers have
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk
> > >> barrier or
> > >> >> > > > > similar
> > >> >> > > > > > > > > >> mechanism
> > >> >> > > > > > > > > >> > > can
> > >> >> > > > > > > > > >> > > > be
> > >> >> > > > > > > > > >> > > > >> used.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms
> > >> will
> > >> >> > > increase
> > >> >> > > > > > > > > complexity.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a
> > >> static
> > >> >> > > secret
> > >> >> > > > > key
> > >> >> > > > > > > > from
> > >> >> > > > > > > > > >> config
> > >> >> > > > > > > > > >> > > > >> file. Will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and
> > >> uses a
> > >> >> > > static
> > >> >> > > > > > > secret
> > >> >> > > > > > > > > >> key. It
> > >> >> > > > > > > > > >> > > is
> > >> >> > > > > > > > > >> > > > >> advised
> > >> >> > > > > > > > > >> > > > >> >> > to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > periodically.
> > >> This
> > >> >> > can
> > >> >> > > > be
> > >> >> > > > > > > > avoided
> > >> >> > > > > > > > > >> with
> > >> >> > > > > > > > > >> > > > >> controller
> > >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to
> > >> brokers
> > >> >> > > > > > > periodically.
> > >> >> > > > > > > > > >> However,
> > >> >> > > > > > > > > >> > > > >> this will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> require
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain
> > >> certain
> > >> >> > > counts
> > >> >> > > > of
> > >> >> > > > > > > > > >> secretKeys.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative
> 1)
> > >> Have
> > >> >> > > > > controller
> > >> >> > > > > > > > > >> generate
> > >> >> > > > > > > > > >> > > > >> delegation
> > >> >> > > > > > > > > >> > > > >> >> > token.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > >> >> > > authenticates
> > >> >> > > > > > with a
> > >> >> > > > > > > > > >> broker.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> > >> client is
> > >> >> > > > > > > > authenticated,
> > >> >> > > > > > > > > >> it
> > >> >> > > > > > > > > >> > > will
> > >> >> > > > > > > > > >> > > > >> make a
> > >> >> > > > > > > > > >> > > > >> >> > broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > >> delegation
> > >> >> > > > token.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker
> > >> forwards
> > >> >> > the
> > >> >> > > > > > request
> > >> >> > > > > > > > to
> > >> >> > > > > > > > > >> > > > controller.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> > Controller
> > >> >> > > generates
> > >> >> > > > a
> > >> >> > > > > DT
> > >> >> > > > > > > and
> > >> >> > > > > > > > > >> > > broadcasts
> > >> >> > > > > > > > > >> > > > >> to all
> > >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker
> > >> stores
> > >> >> > this
> > >> >> > > > > token
> > >> >> > > > > > in
> > >> >> > > > > > > > its
> > >> >> > > > > > > > > >> memory
> > >> >> > > > > > > > > >> > > > >> cache.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> > Controller
> > >> >> > responds
> > >> >> > > > to
> > >> >> > > > > > > > broker’s
> > >> >> > > > > > > > > >> DT
> > >> >> > > > > > > > > >> > > req.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker
> > >> returns
> > >> >> > the
> > >> >> > > > > token
> > >> >> > > > > > to
> > >> >> > > > > > > > > >> Client.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> issues
> > >> and
> > >> >> > fixes
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will
> > >> have to
> > >> >> > add
> > >> >> > > > new
> > >> >> > > > > > > APIs
> > >> >> > > > > > > > to
> > >> >> > > > > > > > > >> > > support
> > >> >> > > > > > > > > >> > > > >> >> > controller
> > >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on
> > >> top of
> > >> >> > the
> > >> >> > > > > > minimal
> > >> >> > > > > > > > APIs
> > >> >> > > > > > > > > >> that
> > >> >> > > > > > > > > >> > > are
> > >> >> > > > > > > > > >> > > > >> >> > currently
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will
> > >> also have
> > >> >> > > to
> > >> >> > > > > add
> > >> >> > > > > > > APIs
> > >> >> > > > > > > > > to
> > >> >> > > > > > > > > >> > > support
> > >> >> > > > > > > > > >> > > > >> the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new
> > >> broker
> > >> >> > > comes
> > >> >> > > > up
> > >> >> > > > > > it
> > >> >> > > > > > > > will
> > >> >> > > > > > > > > >> have
> > >> >> > > > > > > > > >> > > to
> > >> >> > > > > > > > > >> > > > >> get all
> > >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> > >> controller.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
> > >> catastrophic
> > >> >> > > > > failures
> > >> >> > > > > > > > where
> > >> >> > > > > > > > > >> all
> > >> >> > > > > > > > > >> > > > brokers
> > >> >> > > > > > > > > >> > > > >> go
> > >> >> > > > > > > > > >> > > > >> >> > down,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost
> even
> > >> if
> > >> >> > > servers
> > >> >> > > > > are
> > >> >> > > > > > > > > >> restarted as
> > >> >> > > > > > > > > >> > > > >> tokens
> > >> >> > > > > > > > > >> > > > >> >> > are not
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> > >> happens,
> > >> >> > then
> > >> >> > > > > there
> > >> >> > > > > > > are
> > >> >> > > > > > > > > more
> > >> >> > > > > > > > > >> > > > important
> > >> >> > > > > > > > > >> > > > >> >> > things to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it
> is
> > >> better
> > >> >> > to
> > >> >> > > > > > > > > >> re-authenticate.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative
> 2)
> > >> Do not
> > >> >> > > > > > distribute
> > >> >> > > > > > > > DT
> > >> >> > > > > > > > > to
> > >> >> > > > > > > > > >> > > other
> > >> >> > > > > > > > > >> > > > >> brokers
> > >> >> > > > > > > > > >> > > > >> >> > at
> > >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> > >> >> > > authenticates
> > >> >> > > > > > with a
> > >> >> > > > > > > > > >> broker.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> > >> client is
> > >> >> > > > > > > > authenticated,
> > >> >> > > > > > > > > >> it
> > >> >> > > > > > > > > >> > > will
> > >> >> > > > > > > > > >> > > > >> make a
> > >> >> > > > > > > > > >> > > > >> >> > broker
> > >> >> > > > > > > > > >> > > > >> >> > >> >> side
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> > >> delegation
> > >> >> > > > token.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> > broker
> > >> >> > > generates
> > >> >> > > > DT
> > >> >> > > > > > of
> > >> >> > > > > > > > > form,
> > >> >> > > > > > > > > >> > > [hmac +
> > >> >> > > > > > > > > >> > > > >> (owner,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> maxLifeTime,
> > >> id,
> > >> >> > hmac,
> > >> >> > > > > > > > > >> expirationTime)]
> > >> >> > > > > > > > > >> > > and
> > >> >> > > > > > > > > >> > > > >> passes
> > >> >> > > > > > > > > >> > > > >> >> > back
> > >> >> > > > > > > > > >> > > > >> >> > >> >> this
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is
> > >> generated
> > >> >> > via
> > >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> > >> >> > > > > > > > > >> > > > >> renewer,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> expirationTime)
> > >> >> > using
> > >> >> > > > > > > > SecretKey}.
> > >> >> > > > > > > > > >> Note
> > >> >> > > > > > > > > >> > > that
> > >> >> > > > > > > > > >> > > > >> all
> > >> >> > > > > > > > > >> > > > >> >> > brokers
> > >> >> > > > > > > > > >> > > > >> >> > >> >> have
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client
> > >> then goes
> > >> >> > to
> > >> >> > > > any
> > >> >> > > > > > > > broker
> > >> >> > > > > > > > > >> and to
> > >> >> > > > > > > > > >> > > > >> >> > authenticate
> > >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
> > >> recalculates
> > >> >> > > hmac
> > >> >> > > > > > using
> > >> >> > > > > > > > > >> (owner,
> > >> >> > > > > > > > > >> > > > >> renewer,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> expirationTime) info
> > >> >> > > > from
> > >> >> > > > > DT
> > >> >> > > > > > > and
> > >> >> > > > > > > > > its
> > >> >> > > > > > > > > >> > > > >> SecretKey. If
> > >> >> > > > > > > > > >> > > > >> >> > it
> > >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client
> > is
> > >> >> > > > > authenticated.
> > >> >> > > > > > > > Yes,
> > >> >> > > > > > > > > >> it will
> > >> >> > > > > > > > > >> > > > do
> > >> >> > > > > > > > > >> > > > >> other
> > >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp
> > >> expiry and
> > >> >> > > > such.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that
> > secret
> > >> key
> > >> >> > will
> > >> >> > > > be
> > >> >> > > > > > > > > generated
> > >> >> > > > > > > > > >> by
> > >> >> > > > > > > > > >> > > > >> controller
> > >> >> > > > > > > > > >> > > > >> >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable
> issues
> > >> and
> > >> >> > fixes
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to
> > >> delete a
> > >> >> > DT?
> > >> >> > > > > Yes,
> > >> >> > > > > > > that
> > >> >> > > > > > > > > is
> > >> >> > > > > > > > > >> a
> > >> >> > > > > > > > > >> > > > downside
> > >> >> > > > > > > > > >> > > > >> >> > here.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled
> > >> with
> > >> >> > > brokers
> > >> >> > > > > > > > > maintaining
> > >> >> > > > > > > > > >> a
> > >> >> > > > > > > > > >> > > > >> blacklist of
> > >> >> > > > > > > > > >> > > > >> >> > DTs,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be
> > >> removed after
> > >> >> > > > > expiry.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
> > >> catastrophic
> > >> >> > > > > failures
> > >> >> > > > > > > > where
> > >> >> > > > > > > > > >> all
> > >> >> > > > > > > > > >> > > > brokers
> > >> >> > > > > > > > > >> > > > >> go
> > >> >> > > > > > > > > >> > > > >> >> > down,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> the
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost
> even
> > >> if
> > >> >> > > servers
> > >> >> > > > > are
> > >> >> > > > > > > > > >> restarted as
> > >> >> > > > > > > > > >> > > > >> tokens
> > >> >> > > > > > > > > >> > > > >> >> > are not
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> > >> happens,
> > >> >> > then
> > >> >> > > > > there
> > >> >> > > > > > > are
> > >> >> > > > > > > > > more
> > >> >> > > > > > > > > >> > > > important
> > >> >> > > > > > > > > >> > > > >> >> > things to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it
> is
> > >> better
> > >> >> > to
> > >> >> > > > > > > > > >> re-authenticate.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb
> 26,
> > >> 2016 at
> > >> >> > > > 1:58
> > >> >> > > > > > PM,
> > >> >> > > > > > > > > Parth
> > >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > pbrahmbhatt@hortonworks.com>
> > >> >> > > > > > > > wrote:
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed
> > >> KIP-48 so
> > >> >> > > we
> > >> >> > > > > can
> > >> >> > > > > > > > offer
> > >> >> > > > > > > > > >> hadoop
> > >> >> > > > > > > > > >> > > > like
> > >> >> > > > > > > > > >> > > > >> >> > delegation
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You
> can
> > >> review
> > >> >> > > the
> > >> >> > > > > > design
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
> > >> depends on
> > >> >> > > KIP-43
> > >> >> > > > > and
> > >> >> > > > > > > we
> > >> >> > > > > > > > > >> have also
> > >> >> > > > > > > > > >> > > > >> >> > discussed an
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed
> > design
> > >> here<
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >>
> >
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> > >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> > >> >> > > > > > > > > >> > > > >> >> > >> >>
> > >> >> > > > > > > > > >> > > > >> >> >
> > >> >> > > > > > > > > >> > > > >>
> > >> >> > > > > > > > > >> > > >
> > >> >> > > > > > > > > >> > >
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > > >>
> > >> >> > > > > > > > > >
> > >> >> > > > > > > > >
> > >> >> > > > > > > >
> > >> >> > > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > --
> > >> >> > > > > > Regards,
> > >> >> > > > > >
> > >> >> > > > > > Rajini
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > --
> > >> >> > > > Regards,
> > >> >> > > >
> > >> >> > > > Rajini
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> Liquan Pei
> > >> >> Software Engineer, Confluent Inc
> > >>
> > >
> > >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by parth brahmbhatt <br...@gmail.com>.
1. Who / how are tokens renewed? By original requester only? or using Kerberos
auth only?
My recommendation is to do this only using Kerberos auth and only threw the
renewer specified during the acquisition request.
2. Are tokens stored on each broker or in ZK?
My recommendation is still to store in ZK or not store them at all. The
whole controller based distribution is too much overhead with not much to
achieve.
3. How are tokens invalidated / expired?
Either by expiration time out or through an explicit request to invalidate.
4. Which encryption algorithm is used?
SCRAM
5. What is the impersonation proposal (it wasn't in the KIP but was discussed
in this thread)?
There is no imperonation proposal. I tried and explained how its a
different problem and why its not really necessary to discuss that as part
of this KIP. This KIP will not support any impersonation, it will just be
another way to authenticate.
6. Do we need new ACLs, if so - for what actions?
We do not need new ACLs.
7. How would the delegation token be configured in the client?
Should be through config. I wasn't planning on supporting JAAS for tokens.
I don't believe hadoop does this either.
Thanks
Parth
On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <ju...@confluent.io> wrote:
> Harsha,
>
> Another question.
>
> 9. How would the delegation token be configured in the client? The standard
> way is to do this through JAAS. However, we will need to think through if
> this is convenient in a shared environment. For example, when a new task is
> added to a Storm worker node, do we need to dynamically add a new section
> in the JAAS file? It may be more convenient if we can pass in the token
> through the config directly w/o going through JAAS.
>
> Are you or Parth still actively working on this KIP?
>
> Thanks,
>
> Jun
>
>
>
> On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Just to add on that list.
> >
> > 2. It would be good to document the format of the data stored in ZK.
> > 7. Earlier, there was a discussion on whether the tokens should be
> > propagated through ZK like config/acl/quota, or through the controller.
> > Currently, the controller is only designed for propagating topic
> metadata,
> > but not other data.
> > 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since it's
> > deprecated?
> >
> > Also, the images in the wiki seem broken.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >
> >> From what I can see, remaining questions are:
> >>
> >> 1. Who / how are tokens renewed? By original requester only? or using
> >> Kerberos auth only?
> >> 2. Are tokens stored on each broker or in ZK?
> >> 3. How are tokens invalidated / expired?
> >> 4. Which encryption algorithm is used?
> >> 5. What is the impersonation proposal (it wasn't in the KIP but was
> >> discussed in this thread)?
> >> 6. Do we need new ACLs, if so - for what actions?
> >>
> >> Gwen
> >>
> >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> >> > Jun & Ismael,
> >> > Unfortunately I couldn't attend the KIP
> meeting
> >> > when delegation tokens discussed. Appreciate
> if
> >> > you can update the thread if you have any
> >> > further questions.
> >> > Thanks,
> >> > Harsha
> >> >
> >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> >> >> It seems that the links to images in the KIP are broken.
> >> >>
> >> >> Liquan
> >> >>
> >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> >> >> brahmbhatt.parth@gmail.com> wrote:
> >> >>
> >> >> > 110. What does getDelegationTokenAs mean?
> >> >> > In the current proposal we only allow a user to get delegation
> token
> >> for
> >> >> > the identity that it authenticated as using another mechanism, i.e.
> >> A user
> >> >> > that authenticate using a keytab for principal user1@EXAMPLE.COM
> >> will get
> >> >> > delegation tokens for that user only. In future I think we will
> have
> >> to
> >> >> > extend support such that we allow some set of users (
> >> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to acquire
> >> >> > delegation tokens on behalf of other users whose identity they have
> >> >> > verified independently. Kafka brokers will have ACLs to control
> >> which
> >> >> > users are allowed to impersonate other users and get tokens on
> >> behalf of
> >> >> > them. Overall Impersonation is a whole different problem in my
> >> opinion and
> >> >> > I think we can tackle it in separate KIP.
> >> >> >
> >> >> > 111. What's the typical rate of getting and renewing delegation
> >> tokens?
> >> >> > Typically this should be very very low, 1 request per minute is a
> >> >> > relatively high estimate. However it depends on the token
> >> expiration. I am
> >> >> > less worried about the extra load it puts on controller vs the
> added
> >> >> > complexity and the value it offers.
> >> >> >
> >> >> > Thanks
> >> >> > Parth
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk>
> >> wrote:
> >> >> >
> >> >> > > Thanks Rajini. It would probably require a separate KIP as it
> will
> >> >> > > introduce user visible changes. We could also update KIP-48 to
> >> have this
> >> >> > > information, but it seems cleaner to do it separately. We can
> >> discuss
> >> >> > that
> >> >> > > in the KIP call today.
> >> >> > >
> >> >> > > Ismael
> >> >> > >
> >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> >> >> > > rajinisivaram@googlemail.com> wrote:
> >> >> > >
> >> >> > > > Ismael,
> >> >> > > >
> >> >> > > > I have created a JIRA (
> >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> >> >> > > > for adding SCRAM as a SASL mechanism. Would that need another
> >> KIP? If
> >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA that
> gets
> >> >> > > reviewed
> >> >> > > > when the PR is ready?
> >> >> > > >
> >> >> > > > Thank you,
> >> >> > > >
> >> >> > > > Rajini
> >> >> > > >
> >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> ismael@juma.me.uk>
> >> >> > wrote:
> >> >> > > >
> >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> >> >> > > > >
> >> >> > > > > Gwen had independently mentioned this as a SASL mechanism
> that
> >> might
> >> >> > be
> >> >> > > > > useful for Kafka and I have been meaning to check it in more
> >> detail.
> >> >> > > Good
> >> >> > > > > to know that you are willing to contribute an implementation.
> >> Maybe
> >> >> > we
> >> >> > > > > should file a separate JIRA for this?
> >> >> > > > >
> >> >> > > > > Ismael
> >> >> > > > >
> >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> >> >> > > > > rajinisivaram@googlemail.com> wrote:
> >> >> > > > >
> >> >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism)
> >> is a
> >> >> > > better
> >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> built-in
> >> SCRAM
> >> >> > > > > > SaslServer or SaslClient, but I will be happy to add
> support
> >> in
> >> >> > Kafka
> >> >> > > > > since
> >> >> > > > > > it would be a useful mechanism to support anyway.
> >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol
> >> for
> >> >> > > > > > SCRAM-SHA-256.
> >> >> > > > > >
> >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <jun@confluent.io
> >
> >> wrote:
> >> >> > > > > >
> >> >> > > > > > > Parth,
> >> >> > > > > > >
> >> >> > > > > > > Thanks for the explanation. A couple of more questions.
> >> >> > > > > > >
> >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> >> >> > > > > > >
> >> >> > > > > > > 111. What's the typical rate of getting and renewing
> >> delegation
> >> >> > > > tokens?
> >> >> > > > > > > That may have an impact on whether they should be
> directed
> >> to the
> >> >> > > > > > > controller.
> >> >> > > > > > >
> >> >> > > > > > > Jun
> >> >> > > > > > >
> >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> >> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >> >> > > > > > >
> >> >> > > > > > > > Hi Jun,
> >> >> > > > > > > >
> >> >> > > > > > > > Thanks for reviewing.
> >> >> > > > > > > >
> >> >> > > > > > > > * We could add a Cluster action to add acls on who can
> >> request
> >> >> > > > > > delegation
> >> >> > > > > > > > tokens. I don't see the use case for that yet but down
> >> the line
> >> >> > > > when
> >> >> > > > > we
> >> >> > > > > > > > start supporting getDelegationTokenAs it will be
> >> necessary.
> >> >> > > > > > > > * Yes we recommend tokens to be only used/distributed
> >> over
> >> >> > secure
> >> >> > > > > > > channels.
> >> >> > > > > > > > * Depending on what design we end up choosing
> >> Invalidation will
> >> >> > > be
> >> >> > > > > > > > responsibility of every broker or controller.
> >> >> > > > > > > > * I am not sure if I documented somewhere that
> >> invalidation
> >> >> > will
> >> >> > > > > > directly
> >> >> > > > > > > > go through zookeeper but that is not the intent.
> >> Invalidation
> >> >> > > will
> >> >> > > > > > either
> >> >> > > > > > > > be request based or due to expiration. No direct
> >> zookeeper
> >> >> > > > > interaction
> >> >> > > > > > > from
> >> >> > > > > > > > any client.
> >> >> > > > > > > > * "Broker also stores the DelegationToken without the
> >> hmac in
> >> >> > the
> >> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
> >> purpose of
> >> >> > > > > zookeeper
> >> >> > > > > > in
> >> >> > > > > > > > this design is as distribution channel for tokens
> >> between all
> >> >> > > > brokers
> >> >> > > > > > > and a
> >> >> > > > > > > > layer that ensures only tokens that were generated by
> >> making a
> >> >> > > > > request
> >> >> > > > > > > to a
> >> >> > > > > > > > broker will be accepted (more on this in second
> >> paragraph). The
> >> >> > > > token
> >> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> >> expiration,
> >> >> > > hmac)
> >> >> > > > ,
> >> >> > > > > > one
> >> >> > > > > > > of
> >> >> > > > > > > > which is the finally generated hmac but hmac it self is
> >> >> > derivable
> >> >> > > > if
> >> >> > > > > > you
> >> >> > > > > > > > have all the other elements of the token + secret key
> to
> >> >> > generate
> >> >> > > > > hmac.
> >> >> > > > > > > > Given zookeeper does not provide SSL support we do not
> >> want the
> >> >> > > > > entire
> >> >> > > > > > > > token to be wire transferred to zookeeper as that will
> >> be an
> >> >> > > > insecure
> >> >> > > > > > > wire
> >> >> > > > > > > > transfer. Instead we only store all the other elements
> >> of a
> >> >> > > > > delegation
> >> >> > > > > > > > tokens. Brokers can read these elements and because
> they
> >> also
> >> >> > > have
> >> >> > > > > > access
> >> >> > > > > > > > to secret key they will be able to generate hmac on
> >> their end.
> >> >> > > > > > > >
> >> >> > > > > > > > One of the alternative proposed is to avoid zookeeper
> >> >> > > altogether. A
> >> >> > > > > > > Client
> >> >> > > > > > > > will call broker with required information (owner,
> >> renwer,
> >> >> > > > > expiration)
> >> >> > > > > > > and
> >> >> > > > > > > > get back (signed hmac, uuid). Broker won't store this
> in
> >> >> > > zookeeper.
> >> >> > > > > > From
> >> >> > > > > > > > this point a client can contact any broker with all the
> >> >> > > delegation
> >> >> > > > > > token
> >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker
> >> will
> >> >> > > > > regenerate
> >> >> > > > > > > the
> >> >> > > > > > > > hmac and as long as it matches with hmac presented by
> >> client ,
> >> >> > > > broker
> >> >> > > > > > > will
> >> >> > > > > > > > allow the request to authenticate. Only problem with
> >> this
> >> >> > > approach
> >> >> > > > > is
> >> >> > > > > > if
> >> >> > > > > > > > the secret key is compromised any client can now
> generate
> >> >> > random
> >> >> > > > > tokens
> >> >> > > > > > > and
> >> >> > > > > > > > they will still be able to authenticate as any user
> they
> >> like.
> >> >> > > with
> >> >> > > > > > > > zookeeper we guarantee that only tokens acquired via a
> >> broker
> >> >> > > > (using
> >> >> > > > > > some
> >> >> > > > > > > > auth scheme other than delegation token) will be
> >> accepted. We
> >> >> > > need
> >> >> > > > to
> >> >> > > > > > > > discuss which proposal makes more sense and we can go
> >> over it
> >> >> > in
> >> >> > > > > > > tomorrow's
> >> >> > > > > > > > meeting.
> >> >> > > > > > > >
> >> >> > > > > > > > Also, can you forward the invite to me?
> >> >> > > > > > > >
> >> >> > > > > > > > Thanks
> >> >> > > > > > > > Parth
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> >> jun@confluent.io>
> >> >> > > > wrote:
> >> >> > > > > > > >
> >> >> > > > > > > > > Thanks for the KIP. A few comments.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 100. This potentially can be useful for Kafka Connect
> >> and
> >> >> > Kafka
> >> >> > > > > rest
> >> >> > > > > > > > proxy
> >> >> > > > > > > > > where a worker agent will need to run a task on
> behalf
> >> of a
> >> >> > > > client.
> >> >> > > > > > We
> >> >> > > > > > > > will
> >> >> > > > > > > > > likely need to change how those services use Kafka
> >> clients
> >> >> > > > > > > > > (producer/consumer). Instead of a shared client per
> >> worker,
> >> >> > we
> >> >> > > > will
> >> >> > > > > > > need
> >> >> > > > > > > > a
> >> >> > > > > > > > > client per user task since the authentication happens
> >> at the
> >> >> > > > > > connection
> >> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
> >> workers.
> >> >> > So,
> >> >> > > we
> >> >> > > > > > > > probably
> >> >> > > > > > > > > need to allow multiple renewers. For Kafka rest
> proxy,
> >> the
> >> >> > > > renewer
> >> >> > > > > > can
> >> >> > > > > > > > > probably just be the creator of the token.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 101. Do we need new acl on who can request delegation
> >> tokens?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 102. Do we recommend people to send delegation tokens
> >> in an
> >> >> > > > > encrypted
> >> >> > > > > > > > > channel?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 103. Who is responsible for expiring tokens, every
> >> broker?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 104. For invalidating tokens, would it be better to
> do
> >> it in
> >> >> > a
> >> >> > > > > > request
> >> >> > > > > > > > > instead of going to ZK directly?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 105. The terminology of client in the wiki sometimes
> >> refers
> >> >> > to
> >> >> > > > the
> >> >> > > > > > end
> >> >> > > > > > > > > client and some other times refers to the client
> using
> >> the
> >> >> > > > > delegation
> >> >> > > > > > > > > tokens. It would be useful to distinguish between the
> >> two.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 106. Could you explain the sentence "Broker also
> >> stores the
> >> >> > > > > > > > DelegationToken
> >> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
> >> thought the
> >> >> > > > > > > delegation
> >> >> > > > > > > > > token is the hmac.
> >> >> > > > > > > > >
> >> >> > > > > > > > > Thanks,
> >> >> > > > > > > > >
> >> >> > > > > > > > > Jun
> >> >> > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> >> jun@confluent.io>
> >> >> > > > wrote:
> >> >> > > > > > > > >
> >> >> > > > > > > > > > Hi, Harsha,
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss
> >> this in
> >> >> > > the
> >> >> > > > > > > meeting
> >> >> > > > > > > > > > tomorrow.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Thanks,
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Jun
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> >> kafka@harsha.io>
> >> >> > > > wrote:
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >> Hi All,
> >> >> > > > > > > > > >> Can we have a KIP meeting around this.
> >> The KIP
> >> >> > is
> >> >> > > > up
> >> >> > > > > > for
> >> >> > > > > > > > > >> sometime and if there are any questions
> >> lets
> >> >> > > > quickly
> >> >> > > > > > hash
> >> >> > > > > > > > out
> >> >> > > > > > > > > >> details.
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> Thanks,
> >> >> > > > > > > > > >> Harsha
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> brahmbhatt
> >> wrote:
> >> >> > > > > > > > > >> > That is what the hadoop echo system uses so no
> >> good
> >> >> > reason
> >> >> > > > > > really.
> >> >> > > > > > > > We
> >> >> > > > > > > > > >> > could
> >> >> > > > > > > > > >> > change it to whatever is the newest recommended
> >> standard
> >> >> > > is.
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > Thanks
> >> >> > > > > > > > > >> > Parth
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
> >> >> > > > > ismael@juma.me.uk
> >> >> > > > > > >
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > > Hi Parth,
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing
> >> this and
> >> >> > > may
> >> >> > > > > have
> >> >> > > > > > > > > >> additional
> >> >> > > > > > > > > >> > > questions later. The immediate question that
> >> came to
> >> >> > > mind
> >> >> > > > is
> >> >> > > > > > our
> >> >> > > > > > > > > >> choice of
> >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
> >> OBSOLETE in
> >> >> > the
> >> >> > > > IANA
> >> >> > > > > > > > > Registry
> >> >> > > > > > > > > >> of
> >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831)
> has
> >> been
> >> >> > > moved
> >> >> > > > > to
> >> >> > > > > > > > > Historic
> >> >> > > > > > > > > >> > > status:
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > >
> >> >> > > > > >
> >> >> > >
> >> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > What is the reasoning behind that choice?
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Thanks,
> >> >> > > > > > > > > >> > > Ismael
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> Shapira <
> >> >> > > > > > > gwen@confluent.io
> >> >> > > > > > > > >
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > > Also comments inline :)
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * I want to emphasize that even though
> >> delegation
> >> >> > > > tokens
> >> >> > > > > > > are a
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > > innovation, I feel very strongly about not
> >> adding
> >> >> > > > > > dependency
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > > when implementing delegation tokens for
> >> Kafka. The
> >> >> > > KIP
> >> >> > > > > > > doesn't
> >> >> > > > > > > > > >> imply
> >> >> > > > > > > > > >> > > > > such dependency, but if you can clarify...
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will
> >> read
> >> >> > the
> >> >> > > > KIP
> >> >> > > > > > and
> >> >> > > > > > > > > panic
> >> >> > > > > > > > > >> > > > three weeks before the next release...
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * Can we get delegation token at any time
> >> after
> >> >> > > > > > > > authenticating?
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > > immediately after?
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *As long as you are authenticated you can
> >> get
> >> >> > > > delegation
> >> >> > > > > > > > tokens.
> >> >> > > > > > > > > >> We
> >> >> > > > > > > > > >> > > need
> >> >> > > > > > > > > >> > > > to
> >> >> > > > > > > > > >> > > > > discuss if a client authenticated using
> >> delegation
> >> >> > > > > token,
> >> >> > > > > > > can
> >> >> > > > > > > > > also
> >> >> > > > > > > > > >> > > > acquire
> >> >> > > > > > > > > >> > > > > delegation token again or not. Also there
> >> is the
> >> >> > > > > question
> >> >> > > > > > of
> >> >> > > > > > > > do
> >> >> > > > > > > > > we
> >> >> > > > > > > > > >> > > allow
> >> >> > > > > > > > > >> > > > > anyone to acquire delegation token or we
> >> want
> >> >> > > specific
> >> >> > > > > > ACLs
> >> >> > > > > > > (I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > its
> >> >> > > > > > > > > >> > > > an
> >> >> > > > > > > > > >> > > > > overkill.)*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > I think we are debating two options: Either
> >> require
> >> >> > > > > Kerberos
> >> >> > > > > > > > auth
> >> >> > > > > > > > > >> for
> >> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
> >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> basically
> >> >> > require
> >> >> > > a
> >> >> > > > > "job
> >> >> > > > > > > > > master"
> >> >> > > > > > > > > >> > > > to take responsibility for the renewal, it
> >> will have
> >> >> > > its
> >> >> > > > > own
> >> >> > > > > > > > > >> identity
> >> >> > > > > > > > > >> > > > anyway and I think this is the correct
> design
> >> >> > pattern
> >> >> > > > > > anyway.
> >> >> > > > > > > > For
> >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> >> renewals?),
> >> >> > but
> >> >> > > > it
> >> >> > > > > is
> >> >> > > > > > > > hard
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > > debate simplicity without looking at the
> code
> >> >> > changes
> >> >> > > > > > > required.
> >> >> > > > > > > > If
> >> >> > > > > > > > > >> you
> >> >> > > > > > > > > >> > > > have a draft of how the "require Kerberos"
> >> will look
> >> >> > > in
> >> >> > > > > > Kafka
> >> >> > > > > > > > > code,
> >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * My understanding is that tokens will
> >> propagate
> >> >> > via
> >> >> > > > ZK
> >> >> > > > > > but
> >> >> > > > > > > > > >> without
> >> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> >> protocol,
> >> >> > > > correct?
> >> >> > > > > > > > Clients
> >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure
> >> (IIRC),
> >> >> > > but
> >> >> > > > > > since
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> > > > > tokens propagate between brokers asynch,
> we
> >> will
> >> >> > > need
> >> >> > > > to
> >> >> > > > > > > > retry a
> >> >> > > > > > > > > >> bit
> >> >> > > > > > > > > >> > > > > to avoid clients failing auth due to
> timing
> >> >> > issues.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right
> now.
> >> The
> >> >> > > > current
> >> >> > > > > > > > > documented
> >> >> > > > > > > > > >> > > > approach
> >> >> > > > > > > > > >> > > > > is zookeeper based and it does not require
> >> any
> >> >> > > changes
> >> >> > > > > to
> >> >> > > > > > > > > >> > > UpdateMetadata
> >> >> > > > > > > > > >> > > > > protocol. An alternative approach can
> remove
> >> >> > > zookeeper
> >> >> > > > > > > > > dependency
> >> >> > > > > > > > > >> as
> >> >> > > > > > > > > >> > > well
> >> >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion
> >> call.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to
> ping
> >> Jun to
> >> >> > > > > > arrange a
> >> >> > > > > > > > > call?
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having
> >> just the
> >> >> > > > > > controller
> >> >> > > > > > > > > issue
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a
> shared
> >> >> > secret.
> >> >> > > > Not
> >> >> > > > > > > sure
> >> >> > > > > > > > if
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > > want to continue the discussion here or on
> >> the
> >> >> > > wiki. I
> >> >> > > > > > think
> >> >> > > > > > > > > that
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> >> distribution"
> >> >> > > from
> >> >> > > > > > > "shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > > distribution" and use the controller as
> the
> >> only
> >> >> > > token
> >> >> > > > > > > > generator
> >> >> > > > > > > > > >> to
> >> >> > > > > > > > > >> > > > > solve the second issue, while still using
> >> ZK async
> >> >> > > to
> >> >> > > > > > > > distribute
> >> >> > > > > > > > > >> > > > > tokens.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am
> >> fine with
> >> >> > > > that
> >> >> > > > > > > > approach
> >> >> > > > > > > > > >> as
> >> >> > > > > > > > > >> > > long
> >> >> > > > > > > > > >> > > > as
> >> >> > > > > > > > > >> > > > > we agree that the extra complexity of
> >> >> > > adding/updating
> >> >> > > > > APIS
> >> >> > > > > > > > adds
> >> >> > > > > > > > > >> enough
> >> >> > > > > > > > > >> > > > > value. The advantage with the controller
> >> approach
> >> >> > is
> >> >> > > > > > secret
> >> >> > > > > > > > > >> rotation
> >> >> > > > > > > > > >> > > can
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > > automated,frequent and would not require
> >> >> > > deployment. *
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Can you detail the extra complexity (or
> point
> >> me to
> >> >> > > the
> >> >> > > > > > email
> >> >> > > > > > > I
> >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> >> >> > > > > > > > > >> > > > As far as I can tell, clients can already
> >> find the
> >> >> > > > > > controller
> >> >> > > > > > > > from
> >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
> >> controller
> >> >> > > > load.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> kerberos
> >> auth
> >> >> > for
> >> >> > > > > > > renwal, I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > > > it mixes the transport layer the the
> request
> >> >> > content
> >> >> > > > in
> >> >> > > > > a
> >> >> > > > > > > > pretty
> >> >> > > > > > > > > >> ugly
> >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner
> >> is
> >> >> > > better.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While
> >> this will
> >> >> > > make
> >> >> > > > > the
> >> >> > > > > > > > kafka
> >> >> > > > > > > > > >> code
> >> >> > > > > > > > > >> > > > > pretty straight forward , forcing renewer
> >> to
> >> >> > > > non-owner
> >> >> > > > > > > pushes
> >> >> > > > > > > > > >> the code
> >> >> > > > > > > > > >> > > > > ugliness to client and makes it even
> harder
> >> to
> >> >> > > > > > integrate. *
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> >> "renewal by
> >> >> > > > other"
> >> >> > > > > > > > approach
> >> >> > > > > > > > > >> is
> >> >> > > > > > > > > >> > > > that ugly for the clients we expect to use
> >> >> > delegation
> >> >> > > > > tokens
> >> >> > > > > > > > since
> >> >> > > > > > > > > >> > > > they will have an app-master of some sort
> who
> >> >> > > requested
> >> >> > > > > the
> >> >> > > > > > > > token
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > > begin with.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > The response for my question on how
> multiple
> >> >> > > > identities
> >> >> > > > > > will
> >> >> > > > > > > > be
> >> >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK,
> >> we don't
> >> >> > > > > > > > authenticate
> >> >> > > > > > > > > >> each
> >> >> > > > > > > > > >> > > > > request, we authenticate connections.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *We authenticate connections, and only
> when
> >> they
> >> >> > are
> >> >> > > > > being
> >> >> > > > > > > > > >> established.
> >> >> > > > > > > > > >> > > > Let
> >> >> > > > > > > > > >> > > > > me try to phrase this as a question, in
> >> absence of
> >> >> > > > > > > delegation
> >> >> > > > > > > > > >> tokens if
> >> >> > > > > > > > > >> > > > we
> >> >> > > > > > > > > >> > > > > had to support the use case using user
> >> TGT's how
> >> >> > > would
> >> >> > > > > we
> >> >> > > > > > do
> >> >> > > > > > > > it?
> >> >> > > > > > > > > >> My
> >> >> > > > > > > > > >> > > point
> >> >> > > > > > > > > >> > > > > was it would be no different with
> delegation
> >> >> > tokens.
> >> >> > > > The
> >> >> > > > > > use
> >> >> > > > > > > > > case
> >> >> > > > > > > > > >> you
> >> >> > > > > > > > > >> > > are
> >> >> > > > > > > > > >> > > > > describing seems more like impersonation.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Yeah, I thought that one of the things that
> >> >> > delegation
> >> >> > > > > > tokens
> >> >> > > > > > > > > >> handled.
> >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Gwen
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > Thanks
> >> >> > > > > > > > > >> > > > > Parth
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
> >> Shapira <
> >> >> > > > > > > > > gwen@confluent.io
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Few more comments:
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
> >> doesn't seem
> >> >> > to
> >> >> > > > > have
> >> >> > > > > > > good
> >> >> > > > > > > > > >> > > > >> description of the changes to the Kafka
> >> Protocol.
> >> >> > > > > Sounds
> >> >> > > > > > > like
> >> >> > > > > > > > > >> you are
> >> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
> >> >> > > > > > RenewTokenRequest
> >> >> > > > > > > > (and
> >> >> > > > > > > > > >> > > > >> matching responses), without detailing
> the
> >> >> > contents
> >> >> > > > of
> >> >> > > > > > the
> >> >> > > > > > > > > >> requests
> >> >> > > > > > > > > >> > > > >> and responses? Or rather, you show the
> >> class
> >> >> > > > interface,
> >> >> > > > > > but
> >> >> > > > > > > > not
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > >> underlying protocol. This could be seen
> as
> >> an
> >> >> > > > > > > implementation
> >> >> > > > > > > > > >> detail,
> >> >> > > > > > > > > >> > > > >> but since the binary protocol is what we
> >> provide
> >> >> > to
> >> >> > > > > > > non-Java
> >> >> > > > > > > > > >> clients,
> >> >> > > > > > > > > >> > > > >> we need to show the changes there.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> >> >> > > > > > > > delegationTokenRequestHandler?
> >> >> > > > > > > > > >> Is it
> >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or
> Client?
> >> Its
> >> >> > > > > unclear...
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I want to emphasize that even though
> >> delegation
> >> >> > > > > tokens
> >> >> > > > > > > are
> >> >> > > > > > > > a
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly about
> not
> >> adding
> >> >> > > > > > > dependency
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> when implementing delegation tokens for
> >> Kafka.
> >> >> > The
> >> >> > > > KIP
> >> >> > > > > > > > doesn't
> >> >> > > > > > > > > >> imply
> >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> clarify...
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Can we get delegation token at any time
> >> after
> >> >> > > > > > > > authenticating?
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > >> immediately after?
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * My understanding is that tokens will
> >> propagate
> >> >> > > via
> >> >> > > > ZK
> >> >> > > > > > but
> >> >> > > > > > > > > >> without
> >> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
> >> protocol,
> >> >> > > > correct?
> >> >> > > > > > > > Clients
> >> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth
> failure
> >> >> > (IIRC),
> >> >> > > > but
> >> >> > > > > > > since
> >> >> > > > > > > > > the
> >> >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch,
> >> we will
> >> >> > > need
> >> >> > > > > to
> >> >> > > > > > > > retry
> >> >> > > > > > > > > a
> >> >> > > > > > > > > >> bit
> >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to
> timing
> >> >> > issues.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
> >> touching ZK
> >> >> > > > directly
> >> >> > > > > > :)
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having
> >> just the
> >> >> > > > > > controller
> >> >> > > > > > > > > >> issue the
> >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a
> >> shared
> >> >> > > secret.
> >> >> > > > > Not
> >> >> > > > > > > sure
> >> >> > > > > > > > > if
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > >> want to continue the discussion here or
> on
> >> the
> >> >> > > wiki.
> >> >> > > > I
> >> >> > > > > > > think
> >> >> > > > > > > > > >> that we
> >> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> >> distribution"
> >> >> > > from
> >> >> > > > > > > "shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > >> distribution" and use the controller as
> >> the only
> >> >> > > > token
> >> >> > > > > > > > > generator
> >> >> > > > > > > > > >> to
> >> >> > > > > > > > > >> > > > >> solve the second issue, while still using
> >> ZK
> >> >> > async
> >> >> > > to
> >> >> > > > > > > > > distribute
> >> >> > > > > > > > > >> > > > >> tokens.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite
> >> lifetime
> >> >> > of
> >> >> > > > > > tokens
> >> >> > > > > > > > (and
> >> >> > > > > > > > > >> hoped
> >> >> > > > > > > > > >> > > > >> to hear from others in the community) -
> but
> >> >> > having
> >> >> > > > the
> >> >> > > > > > > option
> >> >> > > > > > > > > and
> >> >> > > > > > > > > >> > > > >> documenting it as less secure, allows
> >> users to
> >> >> > > > > configure
> >> >> > > > > > > > their
> >> >> > > > > > > > > >> system
> >> >> > > > > > > > > >> > > > >> as the wish.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
> >> kerberos auth
> >> >> > > for
> >> >> > > > > > > renwal,
> >> >> > > > > > > > I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > > >> it mixes the transport layer the the
> >> request
> >> >> > > content
> >> >> > > > > in a
> >> >> > > > > > > > > pretty
> >> >> > > > > > > > > >> ugly
> >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> non-owner
> >> is
> >> >> > > better.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * More detailed explanation on what we
> >> plan to do
> >> >> > > for
> >> >> > > > > the
> >> >> > > > > > > > java
> >> >> > > > > > > > > >> clients
> >> >> > > > > > > > > >> > > > >> specifically - new configuration? new
> APIs?
> >> >> > > > > > > > > >> > > > >> The response for my question on how
> >> multiple
> >> >> > > > identities
> >> >> > > > > > > will
> >> >> > > > > > > > be
> >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK,
> >> we
> >> >> > don't
> >> >> > > > > > > > authenticate
> >> >> > > > > > > > > >> each
> >> >> > > > > > > > > >> > > > >> request, we authenticate connections.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are
> only
> >> used
> >> >> > in
> >> >> > > > the
> >> >> > > > > > > Hadoop
> >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> >> >> > alternatives
> >> >> > > in
> >> >> > > > > > other
> >> >> > > > > > > > > >> ecosystems
> >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether
> >> there
> >> >> > are
> >> >> > > > some
> >> >> > > > > > > > > >> advantages
> >> >> > > > > > > > > >> > > > >> there.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Gwen
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
> >> >> > > > > kafka@harsha.io
> >> >> > > > > > >
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> >> >> > > > > > > > > >> > > > >> > Can you look at Parth's last
> >> reply.
> >> >> > > Does
> >> >> > > > > it
> >> >> > > > > > > > answer
> >> >> > > > > > > > > >> your
> >> >> > > > > > > > > >> > > > >> > concerns.
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> > Thanks,
> >> >> > > > > > > > > >> > > > >> > Harsha
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
> >> >> > > brahmbhatt
> >> >> > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki
> >> already
> >> >> > has
> >> >> > > > > > details
> >> >> > > > > > > on
> >> >> > > > > > > > > >> token
> >> >> > > > > > > > > >> > > > >> >> expiration
> >> >> > > > > > > > > >> > > > >> >> under token acquisition process
> >> >> > > > > > > > > >> > > > >> >> <
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> >> >> > > > > > > > > >> > > > >> >.
> >> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will
> >> expire
> >> >> > > based
> >> >> > > > > on a
> >> >> > > > > > > > > server
> >> >> > > > > > > > > >> side
> >> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours)
> unless
> >> >> > renewed.
> >> >> > > > > > Renewal
> >> >> > > > > > > > is
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > allowed
> >> >> > > > > > > > > >> > > > >> >> until the max life time of token.
> >> >> > Alternatively
> >> >> > > we
> >> >> > > > > > could
> >> >> > > > > > > > > also
> >> >> > > > > > > > > >> make
> >> >> > > > > > > > > >> > > > that
> >> >> > > > > > > > > >> > > > >> >> an
> >> >> > > > > > > > > >> > > > >> >> optional param and the server side
> >> default can
> >> >> > > > serve
> >> >> > > > > > as
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> upper
> >> >> > > > > > > > > >> > > > bound.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> To your second point it will be done
> >> exactly
> >> >> > the
> >> >> > > > > same
> >> >> > > > > > > way
> >> >> > > > > > > > we
> >> >> > > > > > > > > >> would
> >> >> > > > > > > > > >> > > > >> >> support
> >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client
> >> will have
> >> >> > > to
> >> >> > > > > put
> >> >> > > > > > > the
> >> >> > > > > > > > > >> tokens it
> >> >> > > > > > > > > >> > > > >> wants
> >> >> > > > > > > > > >> > > > >> >> to use in the subject instance and
> call
> >> >> > > > > > produce/consume
> >> >> > > > > > > > > inside
> >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to
> >> keep
> >> >> > > track
> >> >> > > > of
> >> >> > > > > > its
> >> >> > > > > > > > own
> >> >> > > > > > > > > >> > > > subject. I
> >> >> > > > > > > > > >> > > > >> >> will have to look at the code to see
> if
> >> we
> >> >> > > support
> >> >> > > > > > this
> >> >> > > > > > > > > >> feature
> >> >> > > > > > > > > >> > > right
> >> >> > > > > > > > > >> > > > >> now
> >> >> > > > > > > > > >> > > > >> >> but my understanding is delegation
> token
> >> >> > > shouldn't
> >> >> > > > > > need
> >> >> > > > > > > > any
> >> >> > > > > > > > > >> special
> >> >> > > > > > > > > >> > > > >> >> treatment as its just another type of
> >> >> > Credential
> >> >> > > > in
> >> >> > > > > > the
> >> >> > > > > > > > > >> subject.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> I would also like to know what is your
> >> opinion
> >> >> > > > about
> >> >> > > > > > > > > infinite
> >> >> > > > > > > > > >> > > renewal
> >> >> > > > > > > > > >> > > > >> (my
> >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> this),
> >> tokens
> >> >> > > > > > renewing
> >> >> > > > > > > > them
> >> >> > > > > > > > > >> > > self(my
> >> >> > > > > > > > > >> > > > >> >> recommendation is to not support this)
> >> and
> >> >> > most
> >> >> > > > > > > > importantly
> >> >> > > > > > > > > >> your
> >> >> > > > > > > > > >> > > > choice
> >> >> > > > > > > > > >> > > > >> >> between the alternatives listed on
> this
> >> thread
> >> >> > > > > > > > > >> > > > >> >> <
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> >> alternative-2 minus
> >> >> > > > > > > controller
> >> >> > > > > > > > > >> > > > distributing
> >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> Thanks
> >> >> > > > > > > > > >> > > > >> >> Parth
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen
> >> Shapira <
> >> >> > > > > > > > > >> gwen@confluent.io>
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> > Harsha,
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I
> >> didn't
> >> >> > see
> >> >> > > > > your
> >> >> > > > > > > > design
> >> >> > > > > > > > > >> yet,
> >> >> > > > > > > > > >> > > > but in
> >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> >> producers, which
> >> >> > > > will
> >> >> > > > > > > serve
> >> >> > > > > > > > > >> multiple
> >> >> > > > > > > > > >> > > > users
> >> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these
> >> users
> >> >> > > will
> >> >> > > > > have
> >> >> > > > > > > > > >> different
> >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> >> authenticate
> >> >> > > > > separately,
> >> >> > > > > > > and
> >> >> > > > > > > > > >> can't
> >> >> > > > > > > > > >> > > > share a
> >> >> > > > > > > > > >> > > > >> >> > token.
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > Gwen
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM,
> >> Harsha <
> >> >> > > > > > > kafka@harsha.io
> >> >> > > > > > > > >
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> >> >> > > > > > > > > >> > > > >> >> > > On your second point.
> >> Can you
> >> >> > > > > describe
> >> >> > > > > > a
> >> >> > > > > > > > > >> usecase
> >> >> > > > > > > > > >> > > where
> >> >> > > > > > > > > >> > > > >> >> > > mutliple clients ended
> up
> >> >> > > sharing a
> >> >> > > > > > > > producer
> >> >> > > > > > > > > >> and
> >> >> > > > > > > > > >> > > even
> >> >> > > > > > > > > >> > > > if
> >> >> > > > > > > > > >> > > > >> they
> >> >> > > > > > > > > >> > > > >> >> > > do why can't they not
> use
> >> >> > single
> >> >> > > > > token
> >> >> > > > > > > that
> >> >> > > > > > > > > >> producer
> >> >> > > > > > > > > >> > > > >> >> > > captures. Why would we
> >> need
> >> >> > > > multiple
> >> >> > > > > > > > clients
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > >> different
> >> >> > > > > > > > > >> > > > >> >> > > tokens sharing a single
> >> >> > instance
> >> >> > > of
> >> >> > > > > > > > producer.
> >> >> > > > > > > > > >> Also
> >> >> > > > > > > > > >> > > in
> >> >> > > > > > > > > >> > > > >> this
> >> >> > > > > > > > > >> > > > >> >> > > case other clients have
> >> access
> >> >> > > all
> >> >> > > > > the
> >> >> > > > > > > > tokens
> >> >> > > > > > > > > >> no?
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> >> >> > > > > > > > > >> > > > >> >> > > Harsha
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM,
> >> Gwen
> >> >> > > Shapira
> >> >> > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see
> >> in the
> >> >> > > wiki:
> >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
> >> delegation
> >> >> > > > > tokens?
> >> >> > > > > > > > > >> Renewal? How
> >> >> > > > > > > > > >> > > > do we
> >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation
> >> tokens
> >> >> > for
> >> >> > > > > > "do-as"
> >> >> > > > > > > > > (say,
> >> >> > > > > > > > > >> > > submit
> >> >> > > > > > > > > >> > > > >> Storm
> >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a
> >> producer
> >> >> > > for
> >> >> > > > > > every
> >> >> > > > > > > > job
> >> >> > > > > > > > > >> (we
> >> >> > > > > > > > > >> > > can't
> >> >> > > > > > > > > >> > > > >> share
> >> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs
> running
> >> on
> >> >> > same
> >> >> > > > > node),
> >> >> > > > > > > > since
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > only
> >> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is
> >> there a
> >> >> > > plan
> >> >> > > > > to
> >> >> > > > > > > > change
> >> >> > > > > > > > > >> this
> >> >> > > > > > > > > >> > > for
> >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to
> >> allow
> >> >> > > multiple
> >> >> > > > > > users
> >> >> > > > > > > > > with
> >> >> > > > > > > > > >> > > > different
> >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM,
> >> parth
> >> >> > > > > brahmbhatt
> >> >> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com>
> >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time,
> >> can
> >> >> > other
> >> >> > > > > > > committers
> >> >> > > > > > > > > >> review?
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07
> AM,
> >> >> > Harsha <
> >> >> > > > > > > > > >> kafka@harsha.io>
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> >> >> > > > > > > > > >> > > > >> >> > >> >> Overall current
> >> design looks
> >> >> > > > good
> >> >> > > > > to
> >> >> > > > > > > > me. I
> >> >> > > > > > > > > >> am +1
> >> >> > > > > > > > > >> > > on
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this
> >> as
> >> >> > well.
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57
> >> AM,
> >> >> > parth
> >> >> > > > > > > > brahmbhatt
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
> >> infinite
> >> >> > > > lifetime
> >> >> > > > > > > but I
> >> >> > > > > > > > > >> see the
> >> >> > > > > > > > > >> > > > >> Streaming
> >> >> > > > > > > > > >> > > > >> >> > use
> >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use
> >> case I
> >> >> > > was
> >> >> > > > > > hoping
> >> >> > > > > > > > > >> there will
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > >> some
> >> >> > > > > > > > > >> > > > >> >> > notion
> >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get
> new
> >> >> > > > delegation
> >> >> > > > > > > tokens
> >> >> > > > > > > > > at
> >> >> > > > > > > > > >> fixed
> >> >> > > > > > > > > >> > > > >> interval
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If
> >> that is
> >> >> > not
> >> >> > > > the
> >> >> > > > > > case
> >> >> > > > > > > > for
> >> >> > > > > > > > > >> we can
> >> >> > > > > > > > > >> > > > >> discuss
> >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing
> >> them self
> >> >> > > and
> >> >> > > > > the
> >> >> > > > > > > > > >> security
> >> >> > > > > > > > > >> > > > >> implications
> >> >> > > > > > > > > >> > > > >> >> > of the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to
> >> fetch
> >> >> > tokens
> >> >> > > > > from
> >> >> > > > > > > > > >> zookeeper,
> >> >> > > > > > > > > >> > > > >> overall I
> >> >> > > > > > > > > >> > > > >> >> > think
> >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely
> >> on our
> >> >> > > > > metadata
> >> >> > > > > > > > store
> >> >> > > > > > > > > >> and I
> >> >> > > > > > > > > >> > > > >> think we
> >> >> > > > > > > > > >> > > > >> >> > are
> >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction
> with
> >> all
> >> >> > the
> >> >> > > > > KIP-4
> >> >> > > > > > > > > >> improvements.
> >> >> > > > > > > > > >> > > > I
> >> >> > > > > > > > > >> > > > >> chose
> >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case
> the
> >> client
> >> >> > > > will
> >> >> > > > > > > still
> >> >> > > > > > > > > >> just talk
> >> >> > > > > > > > > >> > > > to
> >> >> > > > > > > > > >> > > > >> >> > broker , its
> >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use
> >> zookeeper
> >> >> > > > which
> >> >> > > > > we
> >> >> > > > > > > > > >> already do
> >> >> > > > > > > > > >> > > > for a
> >> >> > > > > > > > > >> > > > >> lot
> >> >> > > > > > > > > >> > > > >> >> > of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> >> development + and
> >> >> > > the
> >> >> > > > > > > ability
> >> >> > > > > > > > > so
> >> >> > > > > > > > > >> > > tokens
> >> >> > > > > > > > > >> > > > >> will
> >> >> > > > > > > > > >> > > > >> >> > survive
> >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> restart/cluster
> >> >> > > failure.
> >> >> > > > > if a
> >> >> > > > > > > > > >> majority
> >> >> > > > > > > > > >> > > > agrees
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > added
> >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> controller
> >> >> > > forwarding
> >> >> > > > > > keys
> >> >> > > > > > > to
> >> >> > > > > > > > > all
> >> >> > > > > > > > > >> > > > broker is
> >> >> > > > > > > > > >> > > > >> >> > justified
> >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security
> >> , I am
> >> >> > > fine
> >> >> > > > > > with
> >> >> > > > > > > > that
> >> >> > > > > > > > > >> option
> >> >> > > > > > > > > >> > > > too.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
> >> support SSL
> >> >> > > we
> >> >> > > > > can
> >> >> > > > > > > not
> >> >> > > > > > > > > >> store
> >> >> > > > > > > > > >> > > > master
> >> >> > > > > > > > > >> > > > >> keys
> >> >> > > > > > > > > >> > > > >> >> > in
> >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys
> will
> >> be
> >> >> > > exposed
> >> >> > > > > on
> >> >> > > > > > > > wire.
> >> >> > > > > > > > > To
> >> >> > > > > > > > > >> > > > support
> >> >> > > > > > > > > >> > > > >> >> > rotation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current
> >> clients is
> >> >> > > > > > > something I
> >> >> > > > > > > > > >> need to
> >> >> > > > > > > > > >> > > > put
> >> >> > > > > > > > > >> > > > >> more
> >> >> > > > > > > > > >> > > > >> >> > thought
> >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
> >> assumes the
> >> >> > > > > rotation
> >> >> > > > > > > > will
> >> >> > > > > > > > > >> > > > invalidate
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > current
> >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also
> >> review
> >> >> > > and
> >> >> > > > > post
> >> >> > > > > > > > their
> >> >> > > > > > > > > >> > > comments
> >> >> > > > > > > > > >> > > > >> so we
> >> >> > > > > > > > > >> > > > >> >> > can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39
> >> AM,
> >> >> > > Ashish
> >> >> > > > > > Singh
> >> >> > > > > > > <
> >> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at
> >> 11:26 AM,
> >> >> > > > > Harsha <
> >> >> > > > > > > > > >> > > > kafka@harsha.io>
> >> >> > > > > > > > > >> > > > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> >> discussion
> >> >> > > threads
> >> >> > > > on
> >> >> > > > > > > this
> >> >> > > > > > > > > KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response
> from
> >> >> > Jitendra
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large
> >> number of
> >> >> > > > > clients
> >> >> > > > > > > that
> >> >> > > > > > > > > are
> >> >> > > > > > > > > >> > > > running
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > over the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> authenticate
> >> with
> >> >> > > Kafka
> >> >> > > > > > > > brokers,
> >> >> > > > > > > > > >> is very
> >> >> > > > > > > > > >> > > > >> similar
> >> >> > > > > > > > > >> > > > >> >> > to the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large
> >> number
> >> >> > of
> >> >> > > > > tasks
> >> >> > > > > > > > > running
> >> >> > > > > > > > > >> > > across
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > cluster
> >> >> > > > > > > > > >> > > > >> >> > >> >> that
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to
> >> Hdfs
> >> >> > > > Namenode.
> >> >> > > > > > > > > >> Therefore, the
> >> >> > > > > > > > > >> > > > >> >> > delegation token
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like
> a
> >> good
> >> >> > fit
> >> >> > > > for
> >> >> > > > > > > this
> >> >> > > > > > > > > use
> >> >> > > > > > > > > >> case
> >> >> > > > > > > > > >> > > > as we
> >> >> > > > > > > > > >> > > > >> >> > have seen
> >> >> > > > > > > > > >> > > > >> >> > >> >> it
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale
> in
> >> HDFS
> >> >> > and
> >> >> > > > > YARN.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is
> >> very
> >> >> > much
> >> >> > > > > > inline
> >> >> > > > > > > > with
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> >> > approach. A few
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want
> >> to allow
> >> >> > > > > > infinite
> >> >> > > > > > > > > >> renewable
> >> >> > > > > > > > > >> > > > >> lifetime
> >> >> > > > > > > > > >> > > > >> >> > for a
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a
> >> token
> >> >> > to a
> >> >> > > > max
> >> >> > > > > > > life
> >> >> > > > > > > > > time
> >> >> > > > > > > > > >> > > > (default
> >> >> > > > > > > > > >> > > > >> 7
> >> >> > > > > > > > > >> > > > >> >> > days). A
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
> >> >> > believed
> >> >> > > to
> >> >> > > > > > > > increase
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > time.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having
> infinite
> >> >> > > lifetime
> >> >> > > > > > might
> >> >> > > > > > > > not
> >> >> > > > > > > > > >> be the
> >> >> > > > > > > > > >> > > > best
> >> >> > > > > > > > > >> > > > >> idea.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the
> >> tokens
> >> >> > are
> >> >> > > > > stored
> >> >> > > > > > > in
> >> >> > > > > > > > > >> zookeeper
> >> >> > > > > > > > > >> > > > as
> >> >> > > > > > > > > >> > > > >> well,
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This
> is
> >> clever
> >> >> > > as
> >> >> > > > it
> >> >> > > > > > can
> >> >> > > > > > > > > allow
> >> >> > > > > > > > > >> > > > >> replacing the
> >> >> > > > > > > > > >> > > > >> >> > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max
> >> life
> >> >> > > time,
> >> >> > > > > and
> >> >> > > > > > > > > clients
> >> >> > > > > > > > > >> can
> >> >> > > > > > > > > >> > > > >> download
> >> >> > > > > > > > > >> > > > >> >> > new
> >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> >> shouldn't be
> >> >> > a
> >> >> > > > big
> >> >> > > > > > load
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> > > zookeeper
> >> >> > > > > > > > > >> > > > >> as a
> >> >> > > > > > > > > >> > > > >> >> > client
> >> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token
> >> once in
> >> >> > > > > several
> >> >> > > > > > > > days.
> >> >> > > > > > > > > >> In this
> >> >> > > > > > > > > >> > > > >> approach
> >> >> > > > > > > > > >> > > > >> >> > you
> >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime
> on
> >> the
> >> >> > > token
> >> >> > > > > even
> >> >> > > > > > > for
> >> >> > > > > > > > > >> long
> >> >> > > > > > > > > >> > > > running
> >> >> > > > > > > > > >> > > > >> >> > clients.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password
> are
> >> >> > > generated
> >> >> > > > > > using
> >> >> > > > > > > a
> >> >> > > > > > > > > >> master
> >> >> > > > > > > > > >> > > key.
> >> >> > > > > > > > > >> > > > >> The
> >> >> > > > > > > > > >> > > > >> >> > master
> >> >> > > > > > > > > >> > > > >> >> > >> >> key
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> >> periodically
> >> >> > > changed.
> >> >> > > > > In
> >> >> > > > > > > > > Hadoop,
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > >> default
> >> >> > > > > > > > > >> > > > >> >> > renewal
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require
> >> brokers
> >> >> > > > > > maintaining
> >> >> > > > > > > a
> >> >> > > > > > > > > >> list of X
> >> >> > > > > > > > > >> > > > most
> >> >> > > > > > > > > >> > > > >> >> > recent
> >> >> > > > > > > > > >> > > > >> >> > >> >> master
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have
> >> to be
> >> >> > > > > persisted
> >> >> > > > > > > > > >> somewhere, as
> >> >> > > > > > > > > >> > > > if a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get
> >> that list
> >> >> > > > again
> >> >> > > > > > and
> >> >> > > > > > > > > >> storing
> >> >> > > > > > > > > >> > > > master
> >> >> > > > > > > > > >> > > > >> keys
> >> >> > > > > > > > > >> > > > >> >> > on ZK
> >> >> > > > > > > > > >> > > > >> >> > >> >> is
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> However,
> >> if a
> >> >> > > > broker
> >> >> > > > > > goes
> >> >> > > > > > > > > down
> >> >> > > > > > > > > >> then
> >> >> > > > > > > > > >> > > we
> >> >> > > > > > > > > >> > > > >> have
> >> >> > > > > > > > > >> > > > >> >> > much
> >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
> >> client can
> >> >> > > > always
> >> >> > > > > > > > > >> > > re-authenticate
> >> >> > > > > > > > > >> > > > is
> >> >> > > > > > > > > >> > > > >> such
> >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a
> >> look at
> >> >> > > other
> >> >> > > > > > > > > >> alternatives
> >> >> > > > > > > > > >> > > this
> >> >> > > > > > > > > >> > > > >> list has
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough
> >> proposal,
> >> >> > > > great
> >> >> > > > > > > work!"
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at
> >> 10:28
> >> >> > PM,
> >> >> > > > Gwen
> >> >> > > > > > > > Shapira
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
> >> Thanks!
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at
> >> 9:25
> >> >> > PM,
> >> >> > > > > > Harsha <
> >> >> > > > > > > > > >> > > > kafka@harsha.io
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any
> >> release
> >> >> > > > > vehicle
> >> >> > > > > > > but
> >> >> > > > > > > > > >> still the
> >> >> > > > > > > > > >> > > > >> work can
> >> >> > > > > > > > > >> > > > >> >> > move
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> >> interested in
> >> >> > the
> >> >> > > > KIP
> >> >> > > > > > > > please
> >> >> > > > > > > > > >> do the
> >> >> > > > > > > > > >> > > > >> review and
> >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016,
> >> at
> >> >> > 04:59
> >> >> > > > PM,
> >> >> > > > > > > Ismael
> >> >> > > > > > > > > >> Juma
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it
> >> would be
> >> >> > good
> >> >> > > > to
> >> >> > > > > > have
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> time
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> review
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016
> >> at
> >> >> > 12:55
> >> >> > > > AM,
> >> >> > > > > > Gwen
> >> >> > > > > > > > > >> Shapira <
> >> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
> >> depends on
> >> >> > > > KIP-43,
> >> >> > > > > > > which
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> > > > already a
> >> >> > > > > > > > > >> > > > >> bit
> >> >> > > > > > > > > >> > > > >> >> > of a
> >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release
> -
> >> any
> >> >> > > chance
> >> >> > > > > we
> >> >> > > > > > > can
> >> >> > > > > > > > > >> delay
> >> >> > > > > > > > > >> > > > >> delegation
> >> >> > > > > > > > > >> > > > >> >> > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> community
> >> >> > working
> >> >> > > > on a
> >> >> > > > > > > > release
> >> >> > > > > > > > > >> every
> >> >> > > > > > > > > >> > > 3
> >> >> > > > > > > > > >> > > > >> month,
> >> >> > > > > > > > > >> > > > >> >> > this
> >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26,
> >> 2016 at
> >> >> > > 5:11
> >> >> > > > > PM,
> >> >> > > > > > > > Ashish
> >> >> > > > > > > > > >> Singh
> >> >> > > > > > > > > >> > > <
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again
> for
> >> the
> >> >> > > > awesome
> >> >> > > > > > > write
> >> >> > > > > > > > > up.
> >> >> > > > > > > > > >> > > > Following
> >> >> > > > > > > > > >> > > > >> our
> >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think
> it
> >> will
> >> >> > be
> >> >> > > > > easier
> >> >> > > > > > > to
> >> >> > > > > > > > > >> compare
> >> >> > > > > > > > > >> > > > >> various
> >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> together.
> >> I am
> >> >> > > > stating
> >> >> > > > > > > > below a
> >> >> > > > > > > > > >> few
> >> >> > > > > > > > > >> > > > >> >> > alternatives along
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> proposal.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
> >> proposal)
> >> >> > Store
> >> >> > > > > > > Delegation
> >> >> > > > > > > > > >> Token,
> >> >> > > > > > > > > >> > > DT,
> >> >> > > > > > > > > >> > > > >> on ZK.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> broker
> >> >> > > generates
> >> >> > > > a
> >> >> > > > > > > shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > based
> >> >> > > > > > > > > >> > > > >> on
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> Password/Secret
> >> >> > shared
> >> >> > > > > > between
> >> >> > > > > > > > all
> >> >> > > > > > > > > >> > > brokers,
> >> >> > > > > > > > > >> > > > >> >> > randomly
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker
> >> stores
> >> >> > this
> >> >> > > > > token
> >> >> > > > > > in
> >> >> > > > > > > > its
> >> >> > > > > > > > > >> in
> >> >> > > > > > > > > >> > > > memory
> >> >> > > > > > > > > >> > > > >> cache.
> >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> >> DelegationToken
> >> >> > > > > without
> >> >> > > > > > > the
> >> >> > > > > > > > > >> hmac in
> >> >> > > > > > > > > >> > > the
> >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All
> >> brokers will
> >> >> > > > have a
> >> >> > > > > > > cache
> >> >> > > > > > > > > >> backed
> >> >> > > > > > > > > >> > > by
> >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> >> >> > > > > > > > > >> > > > >> >> > >> >> they
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified
> >> >> > whenever
> >> >> > > a
> >> >> > > > > new
> >> >> > > > > > > > token
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> > > > >> generated and
> >> >> > > > > > > > > >> > > > >> >> > they
> >> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache
> >> whenever
> >> >> > > > token
> >> >> > > > > > > state
> >> >> > > > > > > > > >> changes.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker
> >> returns
> >> >> > the
> >> >> > > > > token
> >> >> > > > > > to
> >> >> > > > > > > > > >> Client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable
> >> race
> >> >> > > > > condition,
> >> >> > > > > > > > client
> >> >> > > > > > > > > >> tries
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> >> > authenticate
> >> >> > > > > > > > > >> > > > >> >> > >> >> with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet
> >> to be
> >> >> > > > updated
> >> >> > > > > > with
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> newly
> >> >> > > > > > > > > >> > > > >> generated
> >> >> > > > > > > > > >> > > > >> >> > DT.
> >> >> > > > > > > > > >> > > > >> >> > >> >> This
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with
> >> making
> >> >> > > > > dtRequest
> >> >> > > > > > > > block
> >> >> > > > > > > > > >> until
> >> >> > > > > > > > > >> > > all
> >> >> > > > > > > > > >> > > > >> >> > brokers have
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk
> >> barrier or
> >> >> > > > > similar
> >> >> > > > > > > > > >> mechanism
> >> >> > > > > > > > > >> > > can
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > >> used.
> >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms
> >> will
> >> >> > > increase
> >> >> > > > > > > > > complexity.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a
> >> static
> >> >> > > secret
> >> >> > > > > key
> >> >> > > > > > > > from
> >> >> > > > > > > > > >> config
> >> >> > > > > > > > > >> > > > >> file. Will
> >> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and
> >> uses a
> >> >> > > static
> >> >> > > > > > > secret
> >> >> > > > > > > > > >> key. It
> >> >> > > > > > > > > >> > > is
> >> >> > > > > > > > > >> > > > >> advised
> >> >> > > > > > > > > >> > > > >> >> > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> periodically.
> >> This
> >> >> > can
> >> >> > > > be
> >> >> > > > > > > > avoided
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > >> controller
> >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to
> >> brokers
> >> >> > > > > > > periodically.
> >> >> > > > > > > > > >> However,
> >> >> > > > > > > > > >> > > > >> this will
> >> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain
> >> certain
> >> >> > > counts
> >> >> > > > of
> >> >> > > > > > > > > >> secretKeys.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1)
> >> Have
> >> >> > > > > controller
> >> >> > > > > > > > > >> generate
> >> >> > > > > > > > > >> > > > >> delegation
> >> >> > > > > > > > > >> > > > >> >> > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker
> >> forwards
> >> >> > the
> >> >> > > > > > request
> >> >> > > > > > > > to
> >> >> > > > > > > > > >> > > > controller.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4.
> Controller
> >> >> > > generates
> >> >> > > > a
> >> >> > > > > DT
> >> >> > > > > > > and
> >> >> > > > > > > > > >> > > broadcasts
> >> >> > > > > > > > > >> > > > >> to all
> >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker
> >> stores
> >> >> > this
> >> >> > > > > token
> >> >> > > > > > in
> >> >> > > > > > > > its
> >> >> > > > > > > > > >> memory
> >> >> > > > > > > > > >> > > > >> cache.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6.
> Controller
> >> >> > responds
> >> >> > > > to
> >> >> > > > > > > > broker’s
> >> >> > > > > > > > > >> DT
> >> >> > > > > > > > > >> > > req.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker
> >> returns
> >> >> > the
> >> >> > > > > token
> >> >> > > > > > to
> >> >> > > > > > > > > >> Client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will
> >> have to
> >> >> > add
> >> >> > > > new
> >> >> > > > > > > APIs
> >> >> > > > > > > > to
> >> >> > > > > > > > > >> > > support
> >> >> > > > > > > > > >> > > > >> >> > controller
> >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on
> >> top of
> >> >> > the
> >> >> > > > > > minimal
> >> >> > > > > > > > APIs
> >> >> > > > > > > > > >> that
> >> >> > > > > > > > > >> > > are
> >> >> > > > > > > > > >> > > > >> >> > currently
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will
> >> also have
> >> >> > > to
> >> >> > > > > add
> >> >> > > > > > > APIs
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > support
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new
> >> broker
> >> >> > > comes
> >> >> > > > up
> >> >> > > > > > it
> >> >> > > > > > > > will
> >> >> > > > > > > > > >> have
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> get all
> >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> >> controller.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
> >> catastrophic
> >> >> > > > > failures
> >> >> > > > > > > > where
> >> >> > > > > > > > > >> all
> >> >> > > > > > > > > >> > > > brokers
> >> >> > > > > > > > > >> > > > >> go
> >> >> > > > > > > > > >> > > > >> >> > down,
> >> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even
> >> if
> >> >> > > servers
> >> >> > > > > are
> >> >> > > > > > > > > >> restarted as
> >> >> > > > > > > > > >> > > > >> tokens
> >> >> > > > > > > > > >> > > > >> >> > are not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> >> happens,
> >> >> > then
> >> >> > > > > there
> >> >> > > > > > > are
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> > > > important
> >> >> > > > > > > > > >> > > > >> >> > things to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
> >> better
> >> >> > to
> >> >> > > > > > > > > >> re-authenticate.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2)
> >> Do not
> >> >> > > > > > distribute
> >> >> > > > > > > > DT
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > other
> >> >> > > > > > > > > >> > > > >> brokers
> >> >> > > > > > > > > >> > > > >> >> > at
> >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The
> broker
> >> >> > > generates
> >> >> > > > DT
> >> >> > > > > > of
> >> >> > > > > > > > > form,
> >> >> > > > > > > > > >> > > [hmac +
> >> >> > > > > > > > > >> > > > >> (owner,
> >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime,
> >> id,
> >> >> > hmac,
> >> >> > > > > > > > > >> expirationTime)]
> >> >> > > > > > > > > >> > > and
> >> >> > > > > > > > > >> > > > >> passes
> >> >> > > > > > > > > >> > > > >> >> > back
> >> >> > > > > > > > > >> > > > >> >> > >> >> this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is
> >> generated
> >> >> > via
> >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> >> >> > > > > > > > > >> > > > >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> expirationTime)
> >> >> > using
> >> >> > > > > > > > SecretKey}.
> >> >> > > > > > > > > >> Note
> >> >> > > > > > > > > >> > > that
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > brokers
> >> >> > > > > > > > > >> > > > >> >> > >> >> have
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client
> >> then goes
> >> >> > to
> >> >> > > > any
> >> >> > > > > > > > broker
> >> >> > > > > > > > > >> and to
> >> >> > > > > > > > > >> > > > >> >> > authenticate
> >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
> >> recalculates
> >> >> > > hmac
> >> >> > > > > > using
> >> >> > > > > > > > > >> (owner,
> >> >> > > > > > > > > >> > > > >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> expirationTime) info
> >> >> > > > from
> >> >> > > > > DT
> >> >> > > > > > > and
> >> >> > > > > > > > > its
> >> >> > > > > > > > > >> > > > >> SecretKey. If
> >> >> > > > > > > > > >> > > > >> >> > it
> >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client
> is
> >> >> > > > > authenticated.
> >> >> > > > > > > > Yes,
> >> >> > > > > > > > > >> it will
> >> >> > > > > > > > > >> > > > do
> >> >> > > > > > > > > >> > > > >> other
> >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp
> >> expiry and
> >> >> > > > such.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that
> secret
> >> key
> >> >> > will
> >> >> > > > be
> >> >> > > > > > > > > generated
> >> >> > > > > > > > > >> by
> >> >> > > > > > > > > >> > > > >> controller
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to
> >> delete a
> >> >> > DT?
> >> >> > > > > Yes,
> >> >> > > > > > > that
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> a
> >> >> > > > > > > > > >> > > > downside
> >> >> > > > > > > > > >> > > > >> >> > here.
> >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled
> >> with
> >> >> > > brokers
> >> >> > > > > > > > > maintaining
> >> >> > > > > > > > > >> a
> >> >> > > > > > > > > >> > > > >> blacklist of
> >> >> > > > > > > > > >> > > > >> >> > DTs,
> >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be
> >> removed after
> >> >> > > > > expiry.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
> >> catastrophic
> >> >> > > > > failures
> >> >> > > > > > > > where
> >> >> > > > > > > > > >> all
> >> >> > > > > > > > > >> > > > brokers
> >> >> > > > > > > > > >> > > > >> go
> >> >> > > > > > > > > >> > > > >> >> > down,
> >> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even
> >> if
> >> >> > > servers
> >> >> > > > > are
> >> >> > > > > > > > > >> restarted as
> >> >> > > > > > > > > >> > > > >> tokens
> >> >> > > > > > > > > >> > > > >> >> > are not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> >> happens,
> >> >> > then
> >> >> > > > > there
> >> >> > > > > > > are
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> > > > important
> >> >> > > > > > > > > >> > > > >> >> > things to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
> >> better
> >> >> > to
> >> >> > > > > > > > > >> re-authenticate.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26,
> >> 2016 at
> >> >> > > > 1:58
> >> >> > > > > > PM,
> >> >> > > > > > > > > Parth
> >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > pbrahmbhatt@hortonworks.com>
> >> >> > > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed
> >> KIP-48 so
> >> >> > > we
> >> >> > > > > can
> >> >> > > > > > > > offer
> >> >> > > > > > > > > >> hadoop
> >> >> > > > > > > > > >> > > > like
> >> >> > > > > > > > > >> > > > >> >> > delegation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can
> >> review
> >> >> > > the
> >> >> > > > > > design
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
> >> depends on
> >> >> > > KIP-43
> >> >> > > > > and
> >> >> > > > > > > we
> >> >> > > > > > > > > >> have also
> >> >> > > > > > > > > >> > > > >> >> > discussed an
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed
> design
> >> here<
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > --
> >> >> > > > > > Regards,
> >> >> > > > > >
> >> >> > > > > > Rajini
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > --
> >> >> > > > Regards,
> >> >> > > >
> >> >> > > > Rajini
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Liquan Pei
> >> >> Software Engineer, Confluent Inc
> >>
> >
> >
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Harsha,
Another question.
9. How would the delegation token be configured in the client? The standard
way is to do this through JAAS. However, we will need to think through if
this is convenient in a shared environment. For example, when a new task is
added to a Storm worker node, do we need to dynamically add a new section
in the JAAS file? It may be more convenient if we can pass in the token
through the config directly w/o going through JAAS.
Are you or Parth still actively working on this KIP?
Thanks,
Jun
On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <ju...@confluent.io> wrote:
> Just to add on that list.
>
> 2. It would be good to document the format of the data stored in ZK.
> 7. Earlier, there was a discussion on whether the tokens should be
> propagated through ZK like config/acl/quota, or through the controller.
> Currently, the controller is only designed for propagating topic metadata,
> but not other data.
> 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since it's
> deprecated?
>
> Also, the images in the wiki seem broken.
>
> Thanks,
>
> Jun
>
> On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
>> From what I can see, remaining questions are:
>>
>> 1. Who / how are tokens renewed? By original requester only? or using
>> Kerberos auth only?
>> 2. Are tokens stored on each broker or in ZK?
>> 3. How are tokens invalidated / expired?
>> 4. Which encryption algorithm is used?
>> 5. What is the impersonation proposal (it wasn't in the KIP but was
>> discussed in this thread)?
>> 6. Do we need new ACLs, if so - for what actions?
>>
>> Gwen
>>
>> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
>> > Jun & Ismael,
>> > Unfortunately I couldn't attend the KIP meeting
>> > when delegation tokens discussed. Appreciate if
>> > you can update the thread if you have any
>> > further questions.
>> > Thanks,
>> > Harsha
>> >
>> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
>> >> It seems that the links to images in the KIP are broken.
>> >>
>> >> Liquan
>> >>
>> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
>> >> brahmbhatt.parth@gmail.com> wrote:
>> >>
>> >> > 110. What does getDelegationTokenAs mean?
>> >> > In the current proposal we only allow a user to get delegation token
>> for
>> >> > the identity that it authenticated as using another mechanism, i.e.
>> A user
>> >> > that authenticate using a keytab for principal user1@EXAMPLE.COM
>> will get
>> >> > delegation tokens for that user only. In future I think we will have
>> to
>> >> > extend support such that we allow some set of users (
>> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to acquire
>> >> > delegation tokens on behalf of other users whose identity they have
>> >> > verified independently. Kafka brokers will have ACLs to control
>> which
>> >> > users are allowed to impersonate other users and get tokens on
>> behalf of
>> >> > them. Overall Impersonation is a whole different problem in my
>> opinion and
>> >> > I think we can tackle it in separate KIP.
>> >> >
>> >> > 111. What's the typical rate of getting and renewing delegation
>> tokens?
>> >> > Typically this should be very very low, 1 request per minute is a
>> >> > relatively high estimate. However it depends on the token
>> expiration. I am
>> >> > less worried about the extra load it puts on controller vs the added
>> >> > complexity and the value it offers.
>> >> >
>> >> > Thanks
>> >> > Parth
>> >> >
>> >> >
>> >> >
>> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk>
>> wrote:
>> >> >
>> >> > > Thanks Rajini. It would probably require a separate KIP as it will
>> >> > > introduce user visible changes. We could also update KIP-48 to
>> have this
>> >> > > information, but it seems cleaner to do it separately. We can
>> discuss
>> >> > that
>> >> > > in the KIP call today.
>> >> > >
>> >> > > Ismael
>> >> > >
>> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
>> >> > > rajinisivaram@googlemail.com> wrote:
>> >> > >
>> >> > > > Ismael,
>> >> > > >
>> >> > > > I have created a JIRA (
>> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
>> >> > > > for adding SCRAM as a SASL mechanism. Would that need another
>> KIP? If
>> >> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets
>> >> > > reviewed
>> >> > > > when the PR is ready?
>> >> > > >
>> >> > > > Thank you,
>> >> > > >
>> >> > > > Rajini
>> >> > > >
>> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <is...@juma.me.uk>
>> >> > wrote:
>> >> > > >
>> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
>> >> > > > >
>> >> > > > > Gwen had independently mentioned this as a SASL mechanism that
>> might
>> >> > be
>> >> > > > > useful for Kafka and I have been meaning to check it in more
>> detail.
>> >> > > Good
>> >> > > > > to know that you are willing to contribute an implementation.
>> Maybe
>> >> > we
>> >> > > > > should file a separate JIRA for this?
>> >> > > > >
>> >> > > > > Ismael
>> >> > > > >
>> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
>> >> > > > > rajinisivaram@googlemail.com> wrote:
>> >> > > > >
>> >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism)
>> is a
>> >> > > better
>> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in
>> SCRAM
>> >> > > > > > SaslServer or SaslClient, but I will be happy to add support
>> in
>> >> > Kafka
>> >> > > > > since
>> >> > > > > > it would be a useful mechanism to support anyway.
>> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol
>> for
>> >> > > > > > SCRAM-SHA-256.
>> >> > > > > >
>> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <ju...@confluent.io>
>> wrote:
>> >> > > > > >
>> >> > > > > > > Parth,
>> >> > > > > > >
>> >> > > > > > > Thanks for the explanation. A couple of more questions.
>> >> > > > > > >
>> >> > > > > > > 110. What does getDelegationTokenAs mean?
>> >> > > > > > >
>> >> > > > > > > 111. What's the typical rate of getting and renewing
>> delegation
>> >> > > > tokens?
>> >> > > > > > > That may have an impact on whether they should be directed
>> to the
>> >> > > > > > > controller.
>> >> > > > > > >
>> >> > > > > > > Jun
>> >> > > > > > >
>> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
>> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> >> > > > > > >
>> >> > > > > > > > Hi Jun,
>> >> > > > > > > >
>> >> > > > > > > > Thanks for reviewing.
>> >> > > > > > > >
>> >> > > > > > > > * We could add a Cluster action to add acls on who can
>> request
>> >> > > > > > delegation
>> >> > > > > > > > tokens. I don't see the use case for that yet but down
>> the line
>> >> > > > when
>> >> > > > > we
>> >> > > > > > > > start supporting getDelegationTokenAs it will be
>> necessary.
>> >> > > > > > > > * Yes we recommend tokens to be only used/distributed
>> over
>> >> > secure
>> >> > > > > > > channels.
>> >> > > > > > > > * Depending on what design we end up choosing
>> Invalidation will
>> >> > > be
>> >> > > > > > > > responsibility of every broker or controller.
>> >> > > > > > > > * I am not sure if I documented somewhere that
>> invalidation
>> >> > will
>> >> > > > > > directly
>> >> > > > > > > > go through zookeeper but that is not the intent.
>> Invalidation
>> >> > > will
>> >> > > > > > either
>> >> > > > > > > > be request based or due to expiration. No direct
>> zookeeper
>> >> > > > > interaction
>> >> > > > > > > from
>> >> > > > > > > > any client.
>> >> > > > > > > > * "Broker also stores the DelegationToken without the
>> hmac in
>> >> > the
>> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
>> purpose of
>> >> > > > > zookeeper
>> >> > > > > > in
>> >> > > > > > > > this design is as distribution channel for tokens
>> between all
>> >> > > > brokers
>> >> > > > > > > and a
>> >> > > > > > > > layer that ensures only tokens that were generated by
>> making a
>> >> > > > > request
>> >> > > > > > > to a
>> >> > > > > > > > broker will be accepted (more on this in second
>> paragraph). The
>> >> > > > token
>> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
>> expiration,
>> >> > > hmac)
>> >> > > > ,
>> >> > > > > > one
>> >> > > > > > > of
>> >> > > > > > > > which is the finally generated hmac but hmac it self is
>> >> > derivable
>> >> > > > if
>> >> > > > > > you
>> >> > > > > > > > have all the other elements of the token + secret key to
>> >> > generate
>> >> > > > > hmac.
>> >> > > > > > > > Given zookeeper does not provide SSL support we do not
>> want the
>> >> > > > > entire
>> >> > > > > > > > token to be wire transferred to zookeeper as that will
>> be an
>> >> > > > insecure
>> >> > > > > > > wire
>> >> > > > > > > > transfer. Instead we only store all the other elements
>> of a
>> >> > > > > delegation
>> >> > > > > > > > tokens. Brokers can read these elements and because they
>> also
>> >> > > have
>> >> > > > > > access
>> >> > > > > > > > to secret key they will be able to generate hmac on
>> their end.
>> >> > > > > > > >
>> >> > > > > > > > One of the alternative proposed is to avoid zookeeper
>> >> > > altogether. A
>> >> > > > > > > Client
>> >> > > > > > > > will call broker with required information (owner,
>> renwer,
>> >> > > > > expiration)
>> >> > > > > > > and
>> >> > > > > > > > get back (signed hmac, uuid). Broker won't store this in
>> >> > > zookeeper.
>> >> > > > > > From
>> >> > > > > > > > this point a client can contact any broker with all the
>> >> > > delegation
>> >> > > > > > token
>> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker
>> will
>> >> > > > > regenerate
>> >> > > > > > > the
>> >> > > > > > > > hmac and as long as it matches with hmac presented by
>> client ,
>> >> > > > broker
>> >> > > > > > > will
>> >> > > > > > > > allow the request to authenticate. Only problem with
>> this
>> >> > > approach
>> >> > > > > is
>> >> > > > > > if
>> >> > > > > > > > the secret key is compromised any client can now generate
>> >> > random
>> >> > > > > tokens
>> >> > > > > > > and
>> >> > > > > > > > they will still be able to authenticate as any user they
>> like.
>> >> > > with
>> >> > > > > > > > zookeeper we guarantee that only tokens acquired via a
>> broker
>> >> > > > (using
>> >> > > > > > some
>> >> > > > > > > > auth scheme other than delegation token) will be
>> accepted. We
>> >> > > need
>> >> > > > to
>> >> > > > > > > > discuss which proposal makes more sense and we can go
>> over it
>> >> > in
>> >> > > > > > > tomorrow's
>> >> > > > > > > > meeting.
>> >> > > > > > > >
>> >> > > > > > > > Also, can you forward the invite to me?
>> >> > > > > > > >
>> >> > > > > > > > Thanks
>> >> > > > > > > > Parth
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
>> jun@confluent.io>
>> >> > > > wrote:
>> >> > > > > > > >
>> >> > > > > > > > > Thanks for the KIP. A few comments.
>> >> > > > > > > > >
>> >> > > > > > > > > 100. This potentially can be useful for Kafka Connect
>> and
>> >> > Kafka
>> >> > > > > rest
>> >> > > > > > > > proxy
>> >> > > > > > > > > where a worker agent will need to run a task on behalf
>> of a
>> >> > > > client.
>> >> > > > > > We
>> >> > > > > > > > will
>> >> > > > > > > > > likely need to change how those services use Kafka
>> clients
>> >> > > > > > > > > (producer/consumer). Instead of a shared client per
>> worker,
>> >> > we
>> >> > > > will
>> >> > > > > > > need
>> >> > > > > > > > a
>> >> > > > > > > > > client per user task since the authentication happens
>> at the
>> >> > > > > > connection
>> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
>> workers.
>> >> > So,
>> >> > > we
>> >> > > > > > > > probably
>> >> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy,
>> the
>> >> > > > renewer
>> >> > > > > > can
>> >> > > > > > > > > probably just be the creator of the token.
>> >> > > > > > > > >
>> >> > > > > > > > > 101. Do we need new acl on who can request delegation
>> tokens?
>> >> > > > > > > > >
>> >> > > > > > > > > 102. Do we recommend people to send delegation tokens
>> in an
>> >> > > > > encrypted
>> >> > > > > > > > > channel?
>> >> > > > > > > > >
>> >> > > > > > > > > 103. Who is responsible for expiring tokens, every
>> broker?
>> >> > > > > > > > >
>> >> > > > > > > > > 104. For invalidating tokens, would it be better to do
>> it in
>> >> > a
>> >> > > > > > request
>> >> > > > > > > > > instead of going to ZK directly?
>> >> > > > > > > > >
>> >> > > > > > > > > 105. The terminology of client in the wiki sometimes
>> refers
>> >> > to
>> >> > > > the
>> >> > > > > > end
>> >> > > > > > > > > client and some other times refers to the client using
>> the
>> >> > > > > delegation
>> >> > > > > > > > > tokens. It would be useful to distinguish between the
>> two.
>> >> > > > > > > > >
>> >> > > > > > > > > 106. Could you explain the sentence "Broker also
>> stores the
>> >> > > > > > > > DelegationToken
>> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
>> thought the
>> >> > > > > > > delegation
>> >> > > > > > > > > token is the hmac.
>> >> > > > > > > > >
>> >> > > > > > > > > Thanks,
>> >> > > > > > > > >
>> >> > > > > > > > > Jun
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
>> jun@confluent.io>
>> >> > > > wrote:
>> >> > > > > > > > >
>> >> > > > > > > > > > Hi, Harsha,
>> >> > > > > > > > > >
>> >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss
>> this in
>> >> > > the
>> >> > > > > > > meeting
>> >> > > > > > > > > > tomorrow.
>> >> > > > > > > > > >
>> >> > > > > > > > > > Thanks,
>> >> > > > > > > > > >
>> >> > > > > > > > > > Jun
>> >> > > > > > > > > >
>> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
>> kafka@harsha.io>
>> >> > > > wrote:
>> >> > > > > > > > > >
>> >> > > > > > > > > >> Hi All,
>> >> > > > > > > > > >> Can we have a KIP meeting around this.
>> The KIP
>> >> > is
>> >> > > > up
>> >> > > > > > for
>> >> > > > > > > > > >> sometime and if there are any questions
>> lets
>> >> > > > quickly
>> >> > > > > > hash
>> >> > > > > > > > out
>> >> > > > > > > > > >> details.
>> >> > > > > > > > > >>
>> >> > > > > > > > > >> Thanks,
>> >> > > > > > > > > >> Harsha
>> >> > > > > > > > > >>
>> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt
>> wrote:
>> >> > > > > > > > > >> > That is what the hadoop echo system uses so no
>> good
>> >> > reason
>> >> > > > > > really.
>> >> > > > > > > > We
>> >> > > > > > > > > >> > could
>> >> > > > > > > > > >> > change it to whatever is the newest recommended
>> standard
>> >> > > is.
>> >> > > > > > > > > >> >
>> >> > > > > > > > > >> > Thanks
>> >> > > > > > > > > >> > Parth
>> >> > > > > > > > > >> >
>> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
>> >> > > > > ismael@juma.me.uk
>> >> > > > > > >
>> >> > > > > > > > > wrote:
>> >> > > > > > > > > >> >
>> >> > > > > > > > > >> > > Hi Parth,
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing
>> this and
>> >> > > may
>> >> > > > > have
>> >> > > > > > > > > >> additional
>> >> > > > > > > > > >> > > questions later. The immediate question that
>> came to
>> >> > > mind
>> >> > > > is
>> >> > > > > > our
>> >> > > > > > > > > >> choice of
>> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
>> OBSOLETE in
>> >> > the
>> >> > > > IANA
>> >> > > > > > > > > Registry
>> >> > > > > > > > > >> of
>> >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has
>> been
>> >> > > moved
>> >> > > > > to
>> >> > > > > > > > > Historic
>> >> > > > > > > > > >> > > status:
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
>> >> > > > > > > > > >> > >
>> >> > > > > > > > >
>> >> > > > > >
>> >> > >
>> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > What is the reasoning behind that choice?
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > Thanks,
>> >> > > > > > > > > >> > > Ismael
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira <
>> >> > > > > > > gwen@confluent.io
>> >> > > > > > > > >
>> >> > > > > > > > > >> wrote:
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >> > > > Also comments inline :)
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > > * I want to emphasize that even though
>> delegation
>> >> > > > tokens
>> >> > > > > > > are a
>> >> > > > > > > > > >> Hadoop
>> >> > > > > > > > > >> > > > > innovation, I feel very strongly about not
>> adding
>> >> > > > > > dependency
>> >> > > > > > > > on
>> >> > > > > > > > > >> Hadoop
>> >> > > > > > > > > >> > > > > when implementing delegation tokens for
>> Kafka. The
>> >> > > KIP
>> >> > > > > > > doesn't
>> >> > > > > > > > > >> imply
>> >> > > > > > > > > >> > > > > such dependency, but if you can clarify...
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *No hadoop dependency.*
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will
>> read
>> >> > the
>> >> > > > KIP
>> >> > > > > > and
>> >> > > > > > > > > panic
>> >> > > > > > > > > >> > > > three weeks before the next release...
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > > * Can we get delegation token at any time
>> after
>> >> > > > > > > > authenticating?
>> >> > > > > > > > > >> only
>> >> > > > > > > > > >> > > > > immediately after?
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *As long as you are authenticated you can
>> get
>> >> > > > delegation
>> >> > > > > > > > tokens.
>> >> > > > > > > > > >> We
>> >> > > > > > > > > >> > > need
>> >> > > > > > > > > >> > > > to
>> >> > > > > > > > > >> > > > > discuss if a client authenticated using
>> delegation
>> >> > > > > token,
>> >> > > > > > > can
>> >> > > > > > > > > also
>> >> > > > > > > > > >> > > > acquire
>> >> > > > > > > > > >> > > > > delegation token again or not. Also there
>> is the
>> >> > > > > question
>> >> > > > > > of
>> >> > > > > > > > do
>> >> > > > > > > > > we
>> >> > > > > > > > > >> > > allow
>> >> > > > > > > > > >> > > > > anyone to acquire delegation token or we
>> want
>> >> > > specific
>> >> > > > > > ACLs
>> >> > > > > > > (I
>> >> > > > > > > > > >> think
>> >> > > > > > > > > >> > > its
>> >> > > > > > > > > >> > > > an
>> >> > > > > > > > > >> > > > > overkill.)*
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > I think we are debating two options: Either
>> require
>> >> > > > > Kerberos
>> >> > > > > > > > auth
>> >> > > > > > > > > >> for
>> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
>> >> > > > > > > > > >> > > > I *think* the latter is simpler (it basically
>> >> > require
>> >> > > a
>> >> > > > > "job
>> >> > > > > > > > > master"
>> >> > > > > > > > > >> > > > to take responsibility for the renewal, it
>> will have
>> >> > > its
>> >> > > > > own
>> >> > > > > > > > > >> identity
>> >> > > > > > > > > >> > > > anyway and I think this is the correct design
>> >> > pattern
>> >> > > > > > anyway.
>> >> > > > > > > > For
>> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
>> renewals?),
>> >> > but
>> >> > > > it
>> >> > > > > is
>> >> > > > > > > > hard
>> >> > > > > > > > > to
>> >> > > > > > > > > >> > > > debate simplicity without looking at the code
>> >> > changes
>> >> > > > > > > required.
>> >> > > > > > > > If
>> >> > > > > > > > > >> you
>> >> > > > > > > > > >> > > > have a draft of how the "require Kerberos"
>> will look
>> >> > > in
>> >> > > > > > Kafka
>> >> > > > > > > > > code,
>> >> > > > > > > > > >> > > > I'll be happy to take a look.
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > > * My understanding is that tokens will
>> propagate
>> >> > via
>> >> > > > ZK
>> >> > > > > > but
>> >> > > > > > > > > >> without
>> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
>> protocol,
>> >> > > > correct?
>> >> > > > > > > > Clients
>> >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure
>> (IIRC),
>> >> > > but
>> >> > > > > > since
>> >> > > > > > > > the
>> >> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we
>> will
>> >> > > need
>> >> > > > to
>> >> > > > > > > > retry a
>> >> > > > > > > > > >> bit
>> >> > > > > > > > > >> > > > > to avoid clients failing auth due to timing
>> >> > issues.
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right now.
>> The
>> >> > > > current
>> >> > > > > > > > > documented
>> >> > > > > > > > > >> > > > approach
>> >> > > > > > > > > >> > > > > is zookeeper based and it does not require
>> any
>> >> > > changes
>> >> > > > > to
>> >> > > > > > > > > >> > > UpdateMetadata
>> >> > > > > > > > > >> > > > > protocol. An alternative approach can remove
>> >> > > zookeeper
>> >> > > > > > > > > dependency
>> >> > > > > > > > > >> as
>> >> > > > > > > > > >> > > well
>> >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion
>> call.*
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping
>> Jun to
>> >> > > > > > arrange a
>> >> > > > > > > > > call?
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having
>> just the
>> >> > > > > > controller
>> >> > > > > > > > > issue
>> >> > > > > > > > > >> the
>> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared
>> >> > secret.
>> >> > > > Not
>> >> > > > > > > sure
>> >> > > > > > > > if
>> >> > > > > > > > > >> we
>> >> > > > > > > > > >> > > > > want to continue the discussion here or on
>> the
>> >> > > wiki. I
>> >> > > > > > think
>> >> > > > > > > > > that
>> >> > > > > > > > > >> we
>> >> > > > > > > > > >> > > > > can decouple the problem of "token
>> distribution"
>> >> > > from
>> >> > > > > > > "shared
>> >> > > > > > > > > >> secret
>> >> > > > > > > > > >> > > > > distribution" and use the controller as the
>> only
>> >> > > token
>> >> > > > > > > > generator
>> >> > > > > > > > > >> to
>> >> > > > > > > > > >> > > > > solve the second issue, while still using
>> ZK async
>> >> > > to
>> >> > > > > > > > distribute
>> >> > > > > > > > > >> > > > > tokens.
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am
>> fine with
>> >> > > > that
>> >> > > > > > > > approach
>> >> > > > > > > > > >> as
>> >> > > > > > > > > >> > > long
>> >> > > > > > > > > >> > > > as
>> >> > > > > > > > > >> > > > > we agree that the extra complexity of
>> >> > > adding/updating
>> >> > > > > APIS
>> >> > > > > > > > adds
>> >> > > > > > > > > >> enough
>> >> > > > > > > > > >> > > > > value. The advantage with the controller
>> approach
>> >> > is
>> >> > > > > > secret
>> >> > > > > > > > > >> rotation
>> >> > > > > > > > > >> > > can
>> >> > > > > > > > > >> > > > be
>> >> > > > > > > > > >> > > > > automated,frequent and would not require
>> >> > > deployment. *
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Can you detail the extra complexity (or point
>> me to
>> >> > > the
>> >> > > > > > email
>> >> > > > > > > I
>> >> > > > > > > > > >> > > > missed?) - which Apis are required?
>> >> > > > > > > > > >> > > > As far as I can tell, clients can already
>> find the
>> >> > > > > > controller
>> >> > > > > > > > from
>> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
>> controller
>> >> > > > load.
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos
>> auth
>> >> > for
>> >> > > > > > > renwal, I
>> >> > > > > > > > > >> think
>> >> > > > > > > > > >> > > > > it mixes the transport layer the the request
>> >> > content
>> >> > > > in
>> >> > > > > a
>> >> > > > > > > > pretty
>> >> > > > > > > > > >> ugly
>> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner
>> is
>> >> > > better.
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While
>> this will
>> >> > > make
>> >> > > > > the
>> >> > > > > > > > kafka
>> >> > > > > > > > > >> code
>> >> > > > > > > > > >> > > > > pretty straight forward , forcing renewer
>> to
>> >> > > > non-owner
>> >> > > > > > > pushes
>> >> > > > > > > > > >> the code
>> >> > > > > > > > > >> > > > > ugliness to client and makes it even harder
>> to
>> >> > > > > > integrate. *
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > As mentioned before, I don't think the
>> "renewal by
>> >> > > > other"
>> >> > > > > > > > approach
>> >> > > > > > > > > >> is
>> >> > > > > > > > > >> > > > that ugly for the clients we expect to use
>> >> > delegation
>> >> > > > > tokens
>> >> > > > > > > > since
>> >> > > > > > > > > >> > > > they will have an app-master of some sort who
>> >> > > requested
>> >> > > > > the
>> >> > > > > > > > token
>> >> > > > > > > > > to
>> >> > > > > > > > > >> > > > begin with.
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > The response for my question on how multiple
>> >> > > > identities
>> >> > > > > > will
>> >> > > > > > > > be
>> >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK,
>> we don't
>> >> > > > > > > > authenticate
>> >> > > > > > > > > >> each
>> >> > > > > > > > > >> > > > > request, we authenticate connections.
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > *We authenticate connections, and only when
>> they
>> >> > are
>> >> > > > > being
>> >> > > > > > > > > >> established.
>> >> > > > > > > > > >> > > > Let
>> >> > > > > > > > > >> > > > > me try to phrase this as a question, in
>> absence of
>> >> > > > > > > delegation
>> >> > > > > > > > > >> tokens if
>> >> > > > > > > > > >> > > > we
>> >> > > > > > > > > >> > > > > had to support the use case using user
>> TGT's how
>> >> > > would
>> >> > > > > we
>> >> > > > > > do
>> >> > > > > > > > it?
>> >> > > > > > > > > >> My
>> >> > > > > > > > > >> > > point
>> >> > > > > > > > > >> > > > > was it would be no different with delegation
>> >> > tokens.
>> >> > > > The
>> >> > > > > > use
>> >> > > > > > > > > case
>> >> > > > > > > > > >> you
>> >> > > > > > > > > >> > > are
>> >> > > > > > > > > >> > > > > describing seems more like impersonation.*
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Yeah, I thought that one of the things that
>> >> > delegation
>> >> > > > > > tokens
>> >> > > > > > > > > >> handled.
>> >> > > > > > > > > >> > > > Maybe I got it wrong :)
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Thanks for the detailed answers.
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > Gwen
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > > > > Thanks
>> >> > > > > > > > > >> > > > > Parth
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
>> Shapira <
>> >> > > > > > > > > gwen@confluent.io
>> >> > > > > > > > > >> >
>> >> > > > > > > > > >> > > > wrote:
>> >> > > > > > > > > >> > > > >
>> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> Few more comments:
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
>> doesn't seem
>> >> > to
>> >> > > > > have
>> >> > > > > > > good
>> >> > > > > > > > > >> > > > >> description of the changes to the Kafka
>> Protocol.
>> >> > > > > Sounds
>> >> > > > > > > like
>> >> > > > > > > > > >> you are
>> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
>> >> > > > > > RenewTokenRequest
>> >> > > > > > > > (and
>> >> > > > > > > > > >> > > > >> matching responses), without detailing the
>> >> > contents
>> >> > > > of
>> >> > > > > > the
>> >> > > > > > > > > >> requests
>> >> > > > > > > > > >> > > > >> and responses? Or rather, you show the
>> class
>> >> > > > interface,
>> >> > > > > > but
>> >> > > > > > > > not
>> >> > > > > > > > > >> the
>> >> > > > > > > > > >> > > > >> underlying protocol. This could be seen as
>> an
>> >> > > > > > > implementation
>> >> > > > > > > > > >> detail,
>> >> > > > > > > > > >> > > > >> but since the binary protocol is what we
>> provide
>> >> > to
>> >> > > > > > > non-Java
>> >> > > > > > > > > >> clients,
>> >> > > > > > > > > >> > > > >> we need to show the changes there.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
>> >> > > > > > > > delegationTokenRequestHandler?
>> >> > > > > > > > > >> Is it
>> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client?
>> Its
>> >> > > > > unclear...
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * I want to emphasize that even though
>> delegation
>> >> > > > > tokens
>> >> > > > > > > are
>> >> > > > > > > > a
>> >> > > > > > > > > >> Hadoop
>> >> > > > > > > > > >> > > > >> innovation, I feel very strongly about not
>> adding
>> >> > > > > > > dependency
>> >> > > > > > > > on
>> >> > > > > > > > > >> Hadoop
>> >> > > > > > > > > >> > > > >> when implementing delegation tokens for
>> Kafka.
>> >> > The
>> >> > > > KIP
>> >> > > > > > > > doesn't
>> >> > > > > > > > > >> imply
>> >> > > > > > > > > >> > > > >> such dependency, but if you can clarify...
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * Can we get delegation token at any time
>> after
>> >> > > > > > > > authenticating?
>> >> > > > > > > > > >> only
>> >> > > > > > > > > >> > > > >> immediately after?
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * My understanding is that tokens will
>> propagate
>> >> > > via
>> >> > > > ZK
>> >> > > > > > but
>> >> > > > > > > > > >> without
>> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
>> protocol,
>> >> > > > correct?
>> >> > > > > > > > Clients
>> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure
>> >> > (IIRC),
>> >> > > > but
>> >> > > > > > > since
>> >> > > > > > > > > the
>> >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch,
>> we will
>> >> > > need
>> >> > > > > to
>> >> > > > > > > > retry
>> >> > > > > > > > > a
>> >> > > > > > > > > >> bit
>> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing
>> >> > issues.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
>> touching ZK
>> >> > > > directly
>> >> > > > > > :)
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having
>> just the
>> >> > > > > > controller
>> >> > > > > > > > > >> issue the
>> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a
>> shared
>> >> > > secret.
>> >> > > > > Not
>> >> > > > > > > sure
>> >> > > > > > > > > if
>> >> > > > > > > > > >> we
>> >> > > > > > > > > >> > > > >> want to continue the discussion here or on
>> the
>> >> > > wiki.
>> >> > > > I
>> >> > > > > > > think
>> >> > > > > > > > > >> that we
>> >> > > > > > > > > >> > > > >> can decouple the problem of "token
>> distribution"
>> >> > > from
>> >> > > > > > > "shared
>> >> > > > > > > > > >> secret
>> >> > > > > > > > > >> > > > >> distribution" and use the controller as
>> the only
>> >> > > > token
>> >> > > > > > > > > generator
>> >> > > > > > > > > >> to
>> >> > > > > > > > > >> > > > >> solve the second issue, while still using
>> ZK
>> >> > async
>> >> > > to
>> >> > > > > > > > > distribute
>> >> > > > > > > > > >> > > > >> tokens.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite
>> lifetime
>> >> > of
>> >> > > > > > tokens
>> >> > > > > > > > (and
>> >> > > > > > > > > >> hoped
>> >> > > > > > > > > >> > > > >> to hear from others in the community) - but
>> >> > having
>> >> > > > the
>> >> > > > > > > option
>> >> > > > > > > > > and
>> >> > > > > > > > > >> > > > >> documenting it as less secure, allows
>> users to
>> >> > > > > configure
>> >> > > > > > > > their
>> >> > > > > > > > > >> system
>> >> > > > > > > > > >> > > > >> as the wish.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
>> kerberos auth
>> >> > > for
>> >> > > > > > > renwal,
>> >> > > > > > > > I
>> >> > > > > > > > > >> think
>> >> > > > > > > > > >> > > > >> it mixes the transport layer the the
>> request
>> >> > > content
>> >> > > > > in a
>> >> > > > > > > > > pretty
>> >> > > > > > > > > >> ugly
>> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner
>> is
>> >> > > better.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> Things I'd still like to see:
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * More detailed explanation on what we
>> plan to do
>> >> > > for
>> >> > > > > the
>> >> > > > > > > > java
>> >> > > > > > > > > >> clients
>> >> > > > > > > > > >> > > > >> specifically - new configuration? new APIs?
>> >> > > > > > > > > >> > > > >> The response for my question on how
>> multiple
>> >> > > > identities
>> >> > > > > > > will
>> >> > > > > > > > be
>> >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK,
>> we
>> >> > don't
>> >> > > > > > > > authenticate
>> >> > > > > > > > > >> each
>> >> > > > > > > > > >> > > > >> request, we authenticate connections.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only
>> used
>> >> > in
>> >> > > > the
>> >> > > > > > > Hadoop
>> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
>> >> > alternatives
>> >> > > in
>> >> > > > > > other
>> >> > > > > > > > > >> ecosystems
>> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether
>> there
>> >> > are
>> >> > > > some
>> >> > > > > > > > > >> advantages
>> >> > > > > > > > > >> > > > >> there.
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> Gwen
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
>> >> > > > > kafka@harsha.io
>> >> > > > > > >
>> >> > > > > > > > > wrote:
>> >> > > > > > > > > >> > > > >> > Hi Gwen,
>> >> > > > > > > > > >> > > > >> > Can you look at Parth's last
>> reply.
>> >> > > Does
>> >> > > > > it
>> >> > > > > > > > answer
>> >> > > > > > > > > >> your
>> >> > > > > > > > > >> > > > >> > concerns.
>> >> > > > > > > > > >> > > > >> >
>> >> > > > > > > > > >> > > > >> > Thanks,
>> >> > > > > > > > > >> > > > >> > Harsha
>> >> > > > > > > > > >> > > > >> >
>> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
>> >> > > brahmbhatt
>> >> > > > > > wrote:
>> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki
>> already
>> >> > has
>> >> > > > > > details
>> >> > > > > > > on
>> >> > > > > > > > > >> token
>> >> > > > > > > > > >> > > > >> >> expiration
>> >> > > > > > > > > >> > > > >> >> under token acquisition process
>> >> > > > > > > > > >> > > > >> >> <
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >>
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
>> >> > > > > > > > > >> > > > >> >.
>> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will
>> expire
>> >> > > based
>> >> > > > > on a
>> >> > > > > > > > > server
>> >> > > > > > > > > >> side
>> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless
>> >> > renewed.
>> >> > > > > > Renewal
>> >> > > > > > > > is
>> >> > > > > > > > > >> only
>> >> > > > > > > > > >> > > > allowed
>> >> > > > > > > > > >> > > > >> >> until the max life time of token.
>> >> > Alternatively
>> >> > > we
>> >> > > > > > could
>> >> > > > > > > > > also
>> >> > > > > > > > > >> make
>> >> > > > > > > > > >> > > > that
>> >> > > > > > > > > >> > > > >> >> an
>> >> > > > > > > > > >> > > > >> >> optional param and the server side
>> default can
>> >> > > > serve
>> >> > > > > > as
>> >> > > > > > > > the
>> >> > > > > > > > > >> upper
>> >> > > > > > > > > >> > > > bound.
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >> To your second point it will be done
>> exactly
>> >> > the
>> >> > > > > same
>> >> > > > > > > way
>> >> > > > > > > > we
>> >> > > > > > > > > >> would
>> >> > > > > > > > > >> > > > >> >> support
>> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client
>> will have
>> >> > > to
>> >> > > > > put
>> >> > > > > > > the
>> >> > > > > > > > > >> tokens it
>> >> > > > > > > > > >> > > > >> wants
>> >> > > > > > > > > >> > > > >> >> to use in the subject instance and call
>> >> > > > > > produce/consume
>> >> > > > > > > > > inside
>> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to
>> keep
>> >> > > track
>> >> > > > of
>> >> > > > > > its
>> >> > > > > > > > own
>> >> > > > > > > > > >> > > > subject. I
>> >> > > > > > > > > >> > > > >> >> will have to look at the code to see if
>> we
>> >> > > support
>> >> > > > > > this
>> >> > > > > > > > > >> feature
>> >> > > > > > > > > >> > > right
>> >> > > > > > > > > >> > > > >> now
>> >> > > > > > > > > >> > > > >> >> but my understanding is delegation token
>> >> > > shouldn't
>> >> > > > > > need
>> >> > > > > > > > any
>> >> > > > > > > > > >> special
>> >> > > > > > > > > >> > > > >> >> treatment as its just another type of
>> >> > Credential
>> >> > > > in
>> >> > > > > > the
>> >> > > > > > > > > >> subject.
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >> I would also like to know what is your
>> opinion
>> >> > > > about
>> >> > > > > > > > > infinite
>> >> > > > > > > > > >> > > renewal
>> >> > > > > > > > > >> > > > >> (my
>> >> > > > > > > > > >> > > > >> >> recommendation is to not support this),
>> tokens
>> >> > > > > > renewing
>> >> > > > > > > > them
>> >> > > > > > > > > >> > > self(my
>> >> > > > > > > > > >> > > > >> >> recommendation is to not support this)
>> and
>> >> > most
>> >> > > > > > > > importantly
>> >> > > > > > > > > >> your
>> >> > > > > > > > > >> > > > choice
>> >> > > > > > > > > >> > > > >> >> between the alternatives listed on this
>> thread
>> >> > > > > > > > > >> > > > >> >> <
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >>
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
>> >> > > > > > > > > >> > > > >> >
>> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
>> alternative-2 minus
>> >> > > > > > > controller
>> >> > > > > > > > > >> > > > distributing
>> >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >> Thanks
>> >> > > > > > > > > >> > > > >> >> Parth
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen
>> Shapira <
>> >> > > > > > > > > >> gwen@confluent.io>
>> >> > > > > > > > > >> > > > wrote:
>> >> > > > > > > > > >> > > > >> >>
>> >> > > > > > > > > >> > > > >> >> > Harsha,
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I
>> didn't
>> >> > see
>> >> > > > > your
>> >> > > > > > > > design
>> >> > > > > > > > > >> yet,
>> >> > > > > > > > > >> > > > but in
>> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
>> producers, which
>> >> > > > will
>> >> > > > > > > serve
>> >> > > > > > > > > >> multiple
>> >> > > > > > > > > >> > > > users
>> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these
>> users
>> >> > > will
>> >> > > > > have
>> >> > > > > > > > > >> different
>> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
>> authenticate
>> >> > > > > separately,
>> >> > > > > > > and
>> >> > > > > > > > > >> can't
>> >> > > > > > > > > >> > > > share a
>> >> > > > > > > > > >> > > > >> >> > token.
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >> >> > Gwen
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM,
>> Harsha <
>> >> > > > > > > kafka@harsha.io
>> >> > > > > > > > >
>> >> > > > > > > > > >> wrote:
>> >> > > > > > > > > >> > > > >> >> > > Gwen,
>> >> > > > > > > > > >> > > > >> >> > > On your second point.
>> Can you
>> >> > > > > describe
>> >> > > > > > a
>> >> > > > > > > > > >> usecase
>> >> > > > > > > > > >> > > where
>> >> > > > > > > > > >> > > > >> >> > > mutliple clients ended up
>> >> > > sharing a
>> >> > > > > > > > producer
>> >> > > > > > > > > >> and
>> >> > > > > > > > > >> > > even
>> >> > > > > > > > > >> > > > if
>> >> > > > > > > > > >> > > > >> they
>> >> > > > > > > > > >> > > > >> >> > > do why can't they not use
>> >> > single
>> >> > > > > token
>> >> > > > > > > that
>> >> > > > > > > > > >> producer
>> >> > > > > > > > > >> > > > >> >> > > captures. Why would we
>> need
>> >> > > > multiple
>> >> > > > > > > > clients
>> >> > > > > > > > > >> with
>> >> > > > > > > > > >> > > > >> different
>> >> > > > > > > > > >> > > > >> >> > > tokens sharing a single
>> >> > instance
>> >> > > of
>> >> > > > > > > > producer.
>> >> > > > > > > > > >> Also
>> >> > > > > > > > > >> > > in
>> >> > > > > > > > > >> > > > >> this
>> >> > > > > > > > > >> > > > >> >> > > case other clients have
>> access
>> >> > > all
>> >> > > > > the
>> >> > > > > > > > tokens
>> >> > > > > > > > > >> no?
>> >> > > > > > > > > >> > > > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > > Thanks,
>> >> > > > > > > > > >> > > > >> >> > > Harsha
>> >> > > > > > > > > >> > > > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM,
>> Gwen
>> >> > > Shapira
>> >> > > > > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
>> >> > > > > > > > > >> > > > >> >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see
>> in the
>> >> > > wiki:
>> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
>> delegation
>> >> > > > > tokens?
>> >> > > > > > > > > >> Renewal? How
>> >> > > > > > > > > >> > > > do we
>> >> > > > > > > > > >> > > > >> >> > >> revoke them?
>> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation
>> tokens
>> >> > for
>> >> > > > > > "do-as"
>> >> > > > > > > > > (say,
>> >> > > > > > > > > >> > > submit
>> >> > > > > > > > > >> > > > >> Storm
>> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a
>> producer
>> >> > > for
>> >> > > > > > every
>> >> > > > > > > > job
>> >> > > > > > > > > >> (we
>> >> > > > > > > > > >> > > can't
>> >> > > > > > > > > >> > > > >> share
>> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running
>> on
>> >> > same
>> >> > > > > node),
>> >> > > > > > > > since
>> >> > > > > > > > > >> we
>> >> > > > > > > > > >> > > only
>> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is
>> there a
>> >> > > plan
>> >> > > > > to
>> >> > > > > > > > change
>> >> > > > > > > > > >> this
>> >> > > > > > > > > >> > > for
>> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to
>> allow
>> >> > > multiple
>> >> > > > > > users
>> >> > > > > > > > > with
>> >> > > > > > > > > >> > > > different
>> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
>> >> > > > > > > > > >> > > > >> >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> Gwen
>> >> > > > > > > > > >> > > > >> >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM,
>> parth
>> >> > > > > brahmbhatt
>> >> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com>
>> wrote:
>> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time,
>> can
>> >> > other
>> >> > > > > > > committers
>> >> > > > > > > > > >> review?
>> >> > > > > > > > > >> > > > >> >> > >> >
>> >> > > > > > > > > >> > > > >> >> > >> > Thanks
>> >> > > > > > > > > >> > > > >> >> > >> > Parth
>> >> > > > > > > > > >> > > > >> >> > >> >
>> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM,
>> >> > Harsha <
>> >> > > > > > > > > >> kafka@harsha.io>
>> >> > > > > > > > > >> > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
>> >> > > > > > > > > >> > > > >> >> > >> >> Overall current
>> design looks
>> >> > > > good
>> >> > > > > to
>> >> > > > > > > > me. I
>> >> > > > > > > > > >> am +1
>> >> > > > > > > > > >> > > on
>> >> > > > > > > > > >> > > > >> the
>> >> > > > > > > > > >> > > > >> >> > KIP.
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this
>> as
>> >> > well.
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57
>> AM,
>> >> > parth
>> >> > > > > > > > brahmbhatt
>> >> > > > > > > > > >> wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
>> infinite
>> >> > > > lifetime
>> >> > > > > > > but I
>> >> > > > > > > > > >> see the
>> >> > > > > > > > > >> > > > >> Streaming
>> >> > > > > > > > > >> > > > >> >> > use
>> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use
>> case I
>> >> > > was
>> >> > > > > > hoping
>> >> > > > > > > > > >> there will
>> >> > > > > > > > > >> > > > be
>> >> > > > > > > > > >> > > > >> some
>> >> > > > > > > > > >> > > > >> >> > notion
>> >> > > > > > > > > >> > > > >> >> > >> >> > of
>> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new
>> >> > > > delegation
>> >> > > > > > > tokens
>> >> > > > > > > > > at
>> >> > > > > > > > > >> fixed
>> >> > > > > > > > > >> > > > >> interval
>> >> > > > > > > > > >> > > > >> >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If
>> that is
>> >> > not
>> >> > > > the
>> >> > > > > > case
>> >> > > > > > > > for
>> >> > > > > > > > > >> we can
>> >> > > > > > > > > >> > > > >> discuss
>> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing
>> them self
>> >> > > and
>> >> > > > > the
>> >> > > > > > > > > >> security
>> >> > > > > > > > > >> > > > >> implications
>> >> > > > > > > > > >> > > > >> >> > of the
>> >> > > > > > > > > >> > > > >> >> > >> >> > same.
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to
>> fetch
>> >> > tokens
>> >> > > > > from
>> >> > > > > > > > > >> zookeeper,
>> >> > > > > > > > > >> > > > >> overall I
>> >> > > > > > > > > >> > > > >> >> > think
>> >> > > > > > > > > >> > > > >> >> > >> >> > its
>> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely
>> on our
>> >> > > > > metadata
>> >> > > > > > > > store
>> >> > > > > > > > > >> and I
>> >> > > > > > > > > >> > > > >> think we
>> >> > > > > > > > > >> > > > >> >> > are
>> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with
>> all
>> >> > the
>> >> > > > > KIP-4
>> >> > > > > > > > > >> improvements.
>> >> > > > > > > > > >> > > > I
>> >> > > > > > > > > >> > > > >> chose
>> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the
>> client
>> >> > > > will
>> >> > > > > > > still
>> >> > > > > > > > > >> just talk
>> >> > > > > > > > > >> > > > to
>> >> > > > > > > > > >> > > > >> >> > broker , its
>> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use
>> zookeeper
>> >> > > > which
>> >> > > > > we
>> >> > > > > > > > > >> already do
>> >> > > > > > > > > >> > > > for a
>> >> > > > > > > > > >> > > > >> lot
>> >> > > > > > > > > >> > > > >> >> > of
>> >> > > > > > > > > >> > > > >> >> > >> >> > other
>> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
>> development + and
>> >> > > the
>> >> > > > > > > ability
>> >> > > > > > > > > so
>> >> > > > > > > > > >> > > tokens
>> >> > > > > > > > > >> > > > >> will
>> >> > > > > > > > > >> > > > >> >> > survive
>> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster
>> >> > > failure.
>> >> > > > > if a
>> >> > > > > > > > > >> majority
>> >> > > > > > > > > >> > > > agrees
>> >> > > > > > > > > >> > > > >> the
>> >> > > > > > > > > >> > > > >> >> > added
>> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller
>> >> > > forwarding
>> >> > > > > > keys
>> >> > > > > > > to
>> >> > > > > > > > > all
>> >> > > > > > > > > >> > > > broker is
>> >> > > > > > > > > >> > > > >> >> > justified
>> >> > > > > > > > > >> > > > >> >> > >> >> > as
>> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security
>> , I am
>> >> > > fine
>> >> > > > > > with
>> >> > > > > > > > that
>> >> > > > > > > > > >> option
>> >> > > > > > > > > >> > > > too.
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
>> support SSL
>> >> > > we
>> >> > > > > can
>> >> > > > > > > not
>> >> > > > > > > > > >> store
>> >> > > > > > > > > >> > > > master
>> >> > > > > > > > > >> > > > >> keys
>> >> > > > > > > > > >> > > > >> >> > in
>> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will
>> be
>> >> > > exposed
>> >> > > > > on
>> >> > > > > > > > wire.
>> >> > > > > > > > > To
>> >> > > > > > > > > >> > > > support
>> >> > > > > > > > > >> > > > >> >> > rotation
>> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current
>> clients is
>> >> > > > > > > something I
>> >> > > > > > > > > >> need to
>> >> > > > > > > > > >> > > > put
>> >> > > > > > > > > >> > > > >> more
>> >> > > > > > > > > >> > > > >> >> > thought
>> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
>> assumes the
>> >> > > > > rotation
>> >> > > > > > > > will
>> >> > > > > > > > > >> > > > invalidate
>> >> > > > > > > > > >> > > > >> all
>> >> > > > > > > > > >> > > > >> >> > current
>> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also
>> review
>> >> > > and
>> >> > > > > post
>> >> > > > > > > > their
>> >> > > > > > > > > >> > > comments
>> >> > > > > > > > > >> > > > >> so we
>> >> > > > > > > > > >> > > > >> >> > can
>> >> > > > > > > > > >> > > > >> >> > >> >> > make
>> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
>> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39
>> AM,
>> >> > > Ashish
>> >> > > > > > Singh
>> >> > > > > > > <
>> >> > > > > > > > > >> > > > >> asingh@cloudera.com
>> >> > > > > > > > > >> > > > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at
>> 11:26 AM,
>> >> > > > > Harsha <
>> >> > > > > > > > > >> > > > kafka@harsha.io>
>> >> > > > > > > > > >> > > > >> >> > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
>> discussion
>> >> > > threads
>> >> > > > on
>> >> > > > > > > this
>> >> > > > > > > > > KIP.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from
>> >> > Jitendra
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large
>> number of
>> >> > > > > clients
>> >> > > > > > > that
>> >> > > > > > > > > are
>> >> > > > > > > > > >> > > > running
>> >> > > > > > > > > >> > > > >> all
>> >> > > > > > > > > >> > > > >> >> > over the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate
>> with
>> >> > > Kafka
>> >> > > > > > > > brokers,
>> >> > > > > > > > > >> is very
>> >> > > > > > > > > >> > > > >> similar
>> >> > > > > > > > > >> > > > >> >> > to the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large
>> number
>> >> > of
>> >> > > > > tasks
>> >> > > > > > > > > running
>> >> > > > > > > > > >> > > across
>> >> > > > > > > > > >> > > > >> the
>> >> > > > > > > > > >> > > > >> >> > cluster
>> >> > > > > > > > > >> > > > >> >> > >> >> that
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to
>> Hdfs
>> >> > > > Namenode.
>> >> > > > > > > > > >> Therefore, the
>> >> > > > > > > > > >> > > > >> >> > delegation token
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a
>> good
>> >> > fit
>> >> > > > for
>> >> > > > > > > this
>> >> > > > > > > > > use
>> >> > > > > > > > > >> case
>> >> > > > > > > > > >> > > > as we
>> >> > > > > > > > > >> > > > >> >> > have seen
>> >> > > > > > > > > >> > > > >> >> > >> >> it
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in
>> HDFS
>> >> > and
>> >> > > > > YARN.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is
>> very
>> >> > much
>> >> > > > > > inline
>> >> > > > > > > > with
>> >> > > > > > > > > >> Hadoop
>> >> > > > > > > > > >> > > > >> >> > approach. A few
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want
>> to allow
>> >> > > > > > infinite
>> >> > > > > > > > > >> renewable
>> >> > > > > > > > > >> > > > >> lifetime
>> >> > > > > > > > > >> > > > >> >> > for a
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a
>> token
>> >> > to a
>> >> > > > max
>> >> > > > > > > life
>> >> > > > > > > > > time
>> >> > > > > > > > > >> > > > (default
>> >> > > > > > > > > >> > > > >> 7
>> >> > > > > > > > > >> > > > >> >> > days). A
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
>> >> > believed
>> >> > > to
>> >> > > > > > > > increase
>> >> > > > > > > > > >> with
>> >> > > > > > > > > >> > > > time.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite
>> >> > > lifetime
>> >> > > > > > might
>> >> > > > > > > > not
>> >> > > > > > > > > >> be the
>> >> > > > > > > > > >> > > > best
>> >> > > > > > > > > >> > > > >> idea.
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the
>> tokens
>> >> > are
>> >> > > > > stored
>> >> > > > > > > in
>> >> > > > > > > > > >> zookeeper
>> >> > > > > > > > > >> > > > as
>> >> > > > > > > > > >> > > > >> well,
>> >> > > > > > > > > >> > > > >> >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> can
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is
>> clever
>> >> > > as
>> >> > > > it
>> >> > > > > > can
>> >> > > > > > > > > allow
>> >> > > > > > > > > >> > > > >> replacing the
>> >> > > > > > > > > >> > > > >> >> > tokens
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max
>> life
>> >> > > time,
>> >> > > > > and
>> >> > > > > > > > > clients
>> >> > > > > > > > > >> can
>> >> > > > > > > > > >> > > > >> download
>> >> > > > > > > > > >> > > > >> >> > new
>> >> > > > > > > > > >> > > > >> >> > >> >> tokens
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
>> shouldn't be
>> >> > a
>> >> > > > big
>> >> > > > > > load
>> >> > > > > > > > on
>> >> > > > > > > > > >> > > zookeeper
>> >> > > > > > > > > >> > > > >> as a
>> >> > > > > > > > > >> > > > >> >> > client
>> >> > > > > > > > > >> > > > >> >> > >> >> will
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token
>> once in
>> >> > > > > several
>> >> > > > > > > > days.
>> >> > > > > > > > > >> In this
>> >> > > > > > > > > >> > > > >> approach
>> >> > > > > > > > > >> > > > >> >> > you
>> >> > > > > > > > > >> > > > >> >> > >> >> don't
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on
>> the
>> >> > > token
>> >> > > > > even
>> >> > > > > > > for
>> >> > > > > > > > > >> long
>> >> > > > > > > > > >> > > > running
>> >> > > > > > > > > >> > > > >> >> > clients.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are
>> >> > > generated
>> >> > > > > > using
>> >> > > > > > > a
>> >> > > > > > > > > >> master
>> >> > > > > > > > > >> > > key.
>> >> > > > > > > > > >> > > > >> The
>> >> > > > > > > > > >> > > > >> >> > master
>> >> > > > > > > > > >> > > > >> >> > >> >> key
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
>> periodically
>> >> > > changed.
>> >> > > > > In
>> >> > > > > > > > > Hadoop,
>> >> > > > > > > > > >> the
>> >> > > > > > > > > >> > > > >> default
>> >> > > > > > > > > >> > > > >> >> > renewal
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require
>> brokers
>> >> > > > > > maintaining
>> >> > > > > > > a
>> >> > > > > > > > > >> list of X
>> >> > > > > > > > > >> > > > most
>> >> > > > > > > > > >> > > > >> >> > recent
>> >> > > > > > > > > >> > > > >> >> > >> >> master
>> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have
>> to be
>> >> > > > > persisted
>> >> > > > > > > > > >> somewhere, as
>> >> > > > > > > > > >> > > > if a
>> >> > > > > > > > > >> > > > >> >> > broker
>> >> > > > > > > > > >> > > > >> >> > >> >> goes
>> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get
>> that list
>> >> > > > again
>> >> > > > > > and
>> >> > > > > > > > > >> storing
>> >> > > > > > > > > >> > > > master
>> >> > > > > > > > > >> > > > >> keys
>> >> > > > > > > > > >> > > > >> >> > on ZK
>> >> > > > > > > > > >> > > > >> >> > >> >> is
>> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However,
>> if a
>> >> > > > broker
>> >> > > > > > goes
>> >> > > > > > > > > down
>> >> > > > > > > > > >> then
>> >> > > > > > > > > >> > > we
>> >> > > > > > > > > >> > > > >> have
>> >> > > > > > > > > >> > > > >> >> > much
>> >> > > > > > > > > >> > > > >> >> > >> >> bigger
>> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
>> client can
>> >> > > > always
>> >> > > > > > > > > >> > > re-authenticate
>> >> > > > > > > > > >> > > > is
>> >> > > > > > > > > >> > > > >> such
>> >> > > > > > > > > >> > > > >> >> > >> >> events.
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a
>> look at
>> >> > > other
>> >> > > > > > > > > >> alternatives
>> >> > > > > > > > > >> > > this
>> >> > > > > > > > > >> > > > >> list has
>> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough
>> proposal,
>> >> > > > great
>> >> > > > > > > work!"
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at
>> 10:28
>> >> > PM,
>> >> > > > Gwen
>> >> > > > > > > > Shapira
>> >> > > > > > > > > >> wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
>> Thanks!
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at
>> 9:25
>> >> > PM,
>> >> > > > > > Harsha <
>> >> > > > > > > > > >> > > > kafka@harsha.io
>> >> > > > > > > > > >> > > > >> >
>> >> > > > > > > > > >> > > > >> >> > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any
>> release
>> >> > > > > vehicle
>> >> > > > > > > but
>> >> > > > > > > > > >> still the
>> >> > > > > > > > > >> > > > >> work can
>> >> > > > > > > > > >> > > > >> >> > move
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
>> interested in
>> >> > the
>> >> > > > KIP
>> >> > > > > > > > please
>> >> > > > > > > > > >> do the
>> >> > > > > > > > > >> > > > >> review and
>> >> > > > > > > > > >> > > > >> >> > >> >> provide
>> >> > > > > > > > > >> > > > >> >> > >> >> > > the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016,
>> at
>> >> > 04:59
>> >> > > > PM,
>> >> > > > > > > Ismael
>> >> > > > > > > > > >> Juma
>> >> > > > > > > > > >> > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it
>> would be
>> >> > good
>> >> > > > to
>> >> > > > > > have
>> >> > > > > > > > > more
>> >> > > > > > > > > >> time
>> >> > > > > > > > > >> > > to
>> >> > > > > > > > > >> > > > >> review
>> >> > > > > > > > > >> > > > >> >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016
>> at
>> >> > 12:55
>> >> > > > AM,
>> >> > > > > > Gwen
>> >> > > > > > > > > >> Shapira <
>> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
>> depends on
>> >> > > > KIP-43,
>> >> > > > > > > which
>> >> > > > > > > > > is
>> >> > > > > > > > > >> > > > already a
>> >> > > > > > > > > >> > > > >> bit
>> >> > > > > > > > > >> > > > >> >> > of a
>> >> > > > > > > > > >> > > > >> >> > >> >> risk
>> >> > > > > > > > > >> > > > >> >> > >> >> > > for
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release -
>> any
>> >> > > chance
>> >> > > > > we
>> >> > > > > > > can
>> >> > > > > > > > > >> delay
>> >> > > > > > > > > >> > > > >> delegation
>> >> > > > > > > > > >> > > > >> >> > tokens
>> >> > > > > > > > > >> > > > >> >> > >> >> to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community
>> >> > working
>> >> > > > on a
>> >> > > > > > > > release
>> >> > > > > > > > > >> every
>> >> > > > > > > > > >> > > 3
>> >> > > > > > > > > >> > > > >> month,
>> >> > > > > > > > > >> > > > >> >> > this
>> >> > > > > > > > > >> > > > >> >> > >> >> is not
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26,
>> 2016 at
>> >> > > 5:11
>> >> > > > > PM,
>> >> > > > > > > > Ashish
>> >> > > > > > > > > >> Singh
>> >> > > > > > > > > >> > > <
>> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for
>> the
>> >> > > > awesome
>> >> > > > > > > write
>> >> > > > > > > > > up.
>> >> > > > > > > > > >> > > > Following
>> >> > > > > > > > > >> > > > >> our
>> >> > > > > > > > > >> > > > >> >> > >> >> discussion
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it
>> will
>> >> > be
>> >> > > > > easier
>> >> > > > > > > to
>> >> > > > > > > > > >> compare
>> >> > > > > > > > > >> > > > >> various
>> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together.
>> I am
>> >> > > > stating
>> >> > > > > > > > below a
>> >> > > > > > > > > >> few
>> >> > > > > > > > > >> > > > >> >> > alternatives along
>> >> > > > > > > > > >> > > > >> >> > >> >> > > with
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
>> proposal)
>> >> > Store
>> >> > > > > > > Delegation
>> >> > > > > > > > > >> Token,
>> >> > > > > > > > > >> > > DT,
>> >> > > > > > > > > >> > > > >> on ZK.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> >> > > authenticates
>> >> > > > > > with a
>> >> > > > > > > > > >> broker.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
>> client is
>> >> > > > > > > > authenticated,
>> >> > > > > > > > > >> it
>> >> > > > > > > > > >> > > will
>> >> > > > > > > > > >> > > > >> make a
>> >> > > > > > > > > >> > > > >> >> > broker
>> >> > > > > > > > > >> > > > >> >> > >> >> side
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
>> delegation
>> >> > > > token.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
>> >> > > generates
>> >> > > > a
>> >> > > > > > > shared
>> >> > > > > > > > > >> secret
>> >> > > > > > > > > >> > > > based
>> >> > > > > > > > > >> > > > >> on
>> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> Password/Secret
>> >> > shared
>> >> > > > > > between
>> >> > > > > > > > all
>> >> > > > > > > > > >> > > brokers,
>> >> > > > > > > > > >> > > > >> >> > randomly
>> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker
>> stores
>> >> > this
>> >> > > > > token
>> >> > > > > > in
>> >> > > > > > > > its
>> >> > > > > > > > > >> in
>> >> > > > > > > > > >> > > > memory
>> >> > > > > > > > > >> > > > >> cache.
>> >> > > > > > > > > >> > > > >> >> > >> >> Broker
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
>> DelegationToken
>> >> > > > > without
>> >> > > > > > > the
>> >> > > > > > > > > >> hmac in
>> >> > > > > > > > > >> > > the
>> >> > > > > > > > > >> > > > >> >> > zookeeper.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All
>> brokers will
>> >> > > > have a
>> >> > > > > > > cache
>> >> > > > > > > > > >> backed
>> >> > > > > > > > > >> > > by
>> >> > > > > > > > > >> > > > >> >> > zookeeper so
>> >> > > > > > > > > >> > > > >> >> > >> >> they
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified
>> >> > whenever
>> >> > > a
>> >> > > > > new
>> >> > > > > > > > token
>> >> > > > > > > > > is
>> >> > > > > > > > > >> > > > >> generated and
>> >> > > > > > > > > >> > > > >> >> > they
>> >> > > > > > > > > >> > > > >> >> > >> >> will
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache
>> whenever
>> >> > > > token
>> >> > > > > > > state
>> >> > > > > > > > > >> changes.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker
>> returns
>> >> > the
>> >> > > > > token
>> >> > > > > > to
>> >> > > > > > > > > >> Client.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
>> and
>> >> > fixes
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable
>> race
>> >> > > > > condition,
>> >> > > > > > > > client
>> >> > > > > > > > > >> tries
>> >> > > > > > > > > >> > > to
>> >> > > > > > > > > >> > > > >> >> > authenticate
>> >> > > > > > > > > >> > > > >> >> > >> >> with
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet
>> to be
>> >> > > > updated
>> >> > > > > > with
>> >> > > > > > > > the
>> >> > > > > > > > > >> newly
>> >> > > > > > > > > >> > > > >> generated
>> >> > > > > > > > > >> > > > >> >> > DT.
>> >> > > > > > > > > >> > > > >> >> > >> >> This
>> >> > > > > > > > > >> > > > >> >> > >> >> > > can
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with
>> making
>> >> > > > > dtRequest
>> >> > > > > > > > block
>> >> > > > > > > > > >> until
>> >> > > > > > > > > >> > > all
>> >> > > > > > > > > >> > > > >> >> > brokers have
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk
>> barrier or
>> >> > > > > similar
>> >> > > > > > > > > >> mechanism
>> >> > > > > > > > > >> > > can
>> >> > > > > > > > > >> > > > be
>> >> > > > > > > > > >> > > > >> used.
>> >> > > > > > > > > >> > > > >> >> > >> >> However,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms
>> will
>> >> > > increase
>> >> > > > > > > > > complexity.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a
>> static
>> >> > > secret
>> >> > > > > key
>> >> > > > > > > > from
>> >> > > > > > > > > >> config
>> >> > > > > > > > > >> > > > >> file. Will
>> >> > > > > > > > > >> > > > >> >> > >> >> require
>> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and
>> uses a
>> >> > > static
>> >> > > > > > > secret
>> >> > > > > > > > > >> key. It
>> >> > > > > > > > > >> > > is
>> >> > > > > > > > > >> > > > >> advised
>> >> > > > > > > > > >> > > > >> >> > to
>> >> > > > > > > > > >> > > > >> >> > >> >> rotate
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
>> This
>> >> > can
>> >> > > > be
>> >> > > > > > > > avoided
>> >> > > > > > > > > >> with
>> >> > > > > > > > > >> > > > >> controller
>> >> > > > > > > > > >> > > > >> >> > >> >> generating
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to
>> brokers
>> >> > > > > > > periodically.
>> >> > > > > > > > > >> However,
>> >> > > > > > > > > >> > > > >> this will
>> >> > > > > > > > > >> > > > >> >> > >> >> require
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain
>> certain
>> >> > > counts
>> >> > > > of
>> >> > > > > > > > > >> secretKeys.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1)
>> Have
>> >> > > > > controller
>> >> > > > > > > > > >> generate
>> >> > > > > > > > > >> > > > >> delegation
>> >> > > > > > > > > >> > > > >> >> > token.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> >> > > authenticates
>> >> > > > > > with a
>> >> > > > > > > > > >> broker.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
>> client is
>> >> > > > > > > > authenticated,
>> >> > > > > > > > > >> it
>> >> > > > > > > > > >> > > will
>> >> > > > > > > > > >> > > > >> make a
>> >> > > > > > > > > >> > > > >> >> > broker
>> >> > > > > > > > > >> > > > >> >> > >> >> side
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
>> delegation
>> >> > > > token.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker
>> forwards
>> >> > the
>> >> > > > > > request
>> >> > > > > > > > to
>> >> > > > > > > > > >> > > > controller.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller
>> >> > > generates
>> >> > > > a
>> >> > > > > DT
>> >> > > > > > > and
>> >> > > > > > > > > >> > > broadcasts
>> >> > > > > > > > > >> > > > >> to all
>> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker
>> stores
>> >> > this
>> >> > > > > token
>> >> > > > > > in
>> >> > > > > > > > its
>> >> > > > > > > > > >> memory
>> >> > > > > > > > > >> > > > >> cache.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller
>> >> > responds
>> >> > > > to
>> >> > > > > > > > broker’s
>> >> > > > > > > > > >> DT
>> >> > > > > > > > > >> > > req.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker
>> returns
>> >> > the
>> >> > > > > token
>> >> > > > > > to
>> >> > > > > > > > > >> Client.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
>> and
>> >> > fixes
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will
>> have to
>> >> > add
>> >> > > > new
>> >> > > > > > > APIs
>> >> > > > > > > > to
>> >> > > > > > > > > >> > > support
>> >> > > > > > > > > >> > > > >> >> > controller
>> >> > > > > > > > > >> > > > >> >> > >> >> pushing
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on
>> top of
>> >> > the
>> >> > > > > > minimal
>> >> > > > > > > > APIs
>> >> > > > > > > > > >> that
>> >> > > > > > > > > >> > > are
>> >> > > > > > > > > >> > > > >> >> > currently
>> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will
>> also have
>> >> > > to
>> >> > > > > add
>> >> > > > > > > APIs
>> >> > > > > > > > > to
>> >> > > > > > > > > >> > > support
>> >> > > > > > > > > >> > > > >> the
>> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new
>> broker
>> >> > > comes
>> >> > > > up
>> >> > > > > > it
>> >> > > > > > > > will
>> >> > > > > > > > > >> have
>> >> > > > > > > > > >> > > to
>> >> > > > > > > > > >> > > > >> get all
>> >> > > > > > > > > >> > > > >> >> > >> >> delegation
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
>> controller.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
>> catastrophic
>> >> > > > > failures
>> >> > > > > > > > where
>> >> > > > > > > > > >> all
>> >> > > > > > > > > >> > > > brokers
>> >> > > > > > > > > >> > > > >> go
>> >> > > > > > > > > >> > > > >> >> > down,
>> >> > > > > > > > > >> > > > >> >> > >> >> the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even
>> if
>> >> > > servers
>> >> > > > > are
>> >> > > > > > > > > >> restarted as
>> >> > > > > > > > > >> > > > >> tokens
>> >> > > > > > > > > >> > > > >> >> > are not
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
>> happens,
>> >> > then
>> >> > > > > there
>> >> > > > > > > are
>> >> > > > > > > > > more
>> >> > > > > > > > > >> > > > important
>> >> > > > > > > > > >> > > > >> >> > things to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
>> better
>> >> > to
>> >> > > > > > > > > >> re-authenticate.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2)
>> Do not
>> >> > > > > > distribute
>> >> > > > > > > > DT
>> >> > > > > > > > > to
>> >> > > > > > > > > >> > > other
>> >> > > > > > > > > >> > > > >> brokers
>> >> > > > > > > > > >> > > > >> >> > at
>> >> > > > > > > > > >> > > > >> >> > >> >> all.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> >> > > authenticates
>> >> > > > > > with a
>> >> > > > > > > > > >> broker.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
>> client is
>> >> > > > > > > > authenticated,
>> >> > > > > > > > > >> it
>> >> > > > > > > > > >> > > will
>> >> > > > > > > > > >> > > > >> make a
>> >> > > > > > > > > >> > > > >> >> > broker
>> >> > > > > > > > > >> > > > >> >> > >> >> side
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
>> delegation
>> >> > > > token.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
>> >> > > generates
>> >> > > > DT
>> >> > > > > > of
>> >> > > > > > > > > form,
>> >> > > > > > > > > >> > > [hmac +
>> >> > > > > > > > > >> > > > >> (owner,
>> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime,
>> id,
>> >> > hmac,
>> >> > > > > > > > > >> expirationTime)]
>> >> > > > > > > > > >> > > and
>> >> > > > > > > > > >> > > > >> passes
>> >> > > > > > > > > >> > > > >> >> > back
>> >> > > > > > > > > >> > > > >> >> > >> >> this
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is
>> generated
>> >> > via
>> >> > > > > > > > > >> {HMAC-SHA256(owner,
>> >> > > > > > > > > >> > > > >> renewer,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> expirationTime)
>> >> > using
>> >> > > > > > > > SecretKey}.
>> >> > > > > > > > > >> Note
>> >> > > > > > > > > >> > > that
>> >> > > > > > > > > >> > > > >> all
>> >> > > > > > > > > >> > > > >> >> > brokers
>> >> > > > > > > > > >> > > > >> >> > >> >> have
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client
>> then goes
>> >> > to
>> >> > > > any
>> >> > > > > > > > broker
>> >> > > > > > > > > >> and to
>> >> > > > > > > > > >> > > > >> >> > authenticate
>> >> > > > > > > > > >> > > > >> >> > >> >> sends
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
>> recalculates
>> >> > > hmac
>> >> > > > > > using
>> >> > > > > > > > > >> (owner,
>> >> > > > > > > > > >> > > > >> renewer,
>> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> expirationTime) info
>> >> > > > from
>> >> > > > > DT
>> >> > > > > > > and
>> >> > > > > > > > > its
>> >> > > > > > > > > >> > > > >> SecretKey. If
>> >> > > > > > > > > >> > > > >> >> > it
>> >> > > > > > > > > >> > > > >> >> > >> >> matches
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is
>> >> > > > > authenticated.
>> >> > > > > > > > Yes,
>> >> > > > > > > > > >> it will
>> >> > > > > > > > > >> > > > do
>> >> > > > > > > > > >> > > > >> other
>> >> > > > > > > > > >> > > > >> >> > >> >> obvious
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp
>> expiry and
>> >> > > > such.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret
>> key
>> >> > will
>> >> > > > be
>> >> > > > > > > > > generated
>> >> > > > > > > > > >> by
>> >> > > > > > > > > >> > > > >> controller
>> >> > > > > > > > > >> > > > >> >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> passed
>> >> > > > > > > > > >> > > > >> >> > >> >> > > to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
>> and
>> >> > fixes
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to
>> delete a
>> >> > DT?
>> >> > > > > Yes,
>> >> > > > > > > that
>> >> > > > > > > > > is
>> >> > > > > > > > > >> a
>> >> > > > > > > > > >> > > > downside
>> >> > > > > > > > > >> > > > >> >> > here.
>> >> > > > > > > > > >> > > > >> >> > >> >> However,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled
>> with
>> >> > > brokers
>> >> > > > > > > > > maintaining
>> >> > > > > > > > > >> a
>> >> > > > > > > > > >> > > > >> blacklist of
>> >> > > > > > > > > >> > > > >> >> > DTs,
>> >> > > > > > > > > >> > > > >> >> > >> >> DTs
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be
>> removed after
>> >> > > > > expiry.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
>> catastrophic
>> >> > > > > failures
>> >> > > > > > > > where
>> >> > > > > > > > > >> all
>> >> > > > > > > > > >> > > > brokers
>> >> > > > > > > > > >> > > > >> go
>> >> > > > > > > > > >> > > > >> >> > down,
>> >> > > > > > > > > >> > > > >> >> > >> >> the
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even
>> if
>> >> > > servers
>> >> > > > > are
>> >> > > > > > > > > >> restarted as
>> >> > > > > > > > > >> > > > >> tokens
>> >> > > > > > > > > >> > > > >> >> > are not
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
>> happens,
>> >> > then
>> >> > > > > there
>> >> > > > > > > are
>> >> > > > > > > > > more
>> >> > > > > > > > > >> > > > important
>> >> > > > > > > > > >> > > > >> >> > things to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
>> better
>> >> > to
>> >> > > > > > > > > >> re-authenticate.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26,
>> 2016 at
>> >> > > > 1:58
>> >> > > > > > PM,
>> >> > > > > > > > > Parth
>> >> > > > > > > > > >> > > > >> Brahmbhatt <
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > pbrahmbhatt@hortonworks.com>
>> >> > > > > > > > wrote:
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed
>> KIP-48 so
>> >> > > we
>> >> > > > > can
>> >> > > > > > > > offer
>> >> > > > > > > > > >> hadoop
>> >> > > > > > > > > >> > > > like
>> >> > > > > > > > > >> > > > >> >> > delegation
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can
>> review
>> >> > > the
>> >> > > > > > design
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >>
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
>> depends on
>> >> > > KIP-43
>> >> > > > > and
>> >> > > > > > > we
>> >> > > > > > > > > >> have also
>> >> > > > > > > > > >> > > > >> >> > discussed an
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design
>> here<
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >>
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
>> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > --
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
>> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
>> >> > > > > > > > > >> > > > >> >> > >> >> > >
>> >> > > > > > > > > >> > > > >> >> > >> >>
>> >> > > > > > > > > >> > > > >> >> >
>> >> > > > > > > > > >> > > > >>
>> >> > > > > > > > > >> > > >
>> >> > > > > > > > > >> > >
>> >> > > > > > > > > >>
>> >> > > > > > > > > >>
>> >> > > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > --
>> >> > > > > > Regards,
>> >> > > > > >
>> >> > > > > > Rajini
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > --
>> >> > > > Regards,
>> >> > > >
>> >> > > > Rajini
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Liquan Pei
>> >> Software Engineer, Confluent Inc
>>
>
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Jun Rao <ju...@confluent.io>.
Just to add on that list.
2. It would be good to document the format of the data stored in ZK.
7. Earlier, there was a discussion on whether the tokens should be
propagated through ZK like config/acl/quota, or through the controller.
Currently, the controller is only designed for propagating topic metadata,
but not other data.
8. Should we use SCRAM to send the token instead of DIGEST-MD5 since it's
deprecated?
Also, the images in the wiki seem broken.
Thanks,
Jun
On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <gw...@confluent.io> wrote:
> From what I can see, remaining questions are:
>
> 1. Who / how are tokens renewed? By original requester only? or using
> Kerberos auth only?
> 2. Are tokens stored on each broker or in ZK?
> 3. How are tokens invalidated / expired?
> 4. Which encryption algorithm is used?
> 5. What is the impersonation proposal (it wasn't in the KIP but was
> discussed in this thread)?
> 6. Do we need new ACLs, if so - for what actions?
>
> Gwen
>
> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> > Jun & Ismael,
> > Unfortunately I couldn't attend the KIP meeting
> > when delegation tokens discussed. Appreciate if
> > you can update the thread if you have any
> > further questions.
> > Thanks,
> > Harsha
> >
> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> >> It seems that the links to images in the KIP are broken.
> >>
> >> Liquan
> >>
> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> >> brahmbhatt.parth@gmail.com> wrote:
> >>
> >> > 110. What does getDelegationTokenAs mean?
> >> > In the current proposal we only allow a user to get delegation token
> for
> >> > the identity that it authenticated as using another mechanism, i.e. A
> user
> >> > that authenticate using a keytab for principal user1@EXAMPLE.COM
> will get
> >> > delegation tokens for that user only. In future I think we will have
> to
> >> > extend support such that we allow some set of users (
> >> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to acquire
> >> > delegation tokens on behalf of other users whose identity they have
> >> > verified independently. Kafka brokers will have ACLs to control which
> >> > users are allowed to impersonate other users and get tokens on behalf
> of
> >> > them. Overall Impersonation is a whole different problem in my
> opinion and
> >> > I think we can tackle it in separate KIP.
> >> >
> >> > 111. What's the typical rate of getting and renewing delegation
> tokens?
> >> > Typically this should be very very low, 1 request per minute is a
> >> > relatively high estimate. However it depends on the token expiration.
> I am
> >> > less worried about the extra load it puts on controller vs the added
> >> > complexity and the value it offers.
> >> >
> >> > Thanks
> >> > Parth
> >> >
> >> >
> >> >
> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk>
> wrote:
> >> >
> >> > > Thanks Rajini. It would probably require a separate KIP as it will
> >> > > introduce user visible changes. We could also update KIP-48 to have
> this
> >> > > information, but it seems cleaner to do it separately. We can
> discuss
> >> > that
> >> > > in the KIP call today.
> >> > >
> >> > > Ismael
> >> > >
> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> >> > > rajinisivaram@googlemail.com> wrote:
> >> > >
> >> > > > Ismael,
> >> > > >
> >> > > > I have created a JIRA (
> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> >> > > > for adding SCRAM as a SASL mechanism. Would that need another
> KIP? If
> >> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets
> >> > > reviewed
> >> > > > when the PR is ready?
> >> > > >
> >> > > > Thank you,
> >> > > >
> >> > > > Rajini
> >> > > >
> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <is...@juma.me.uk>
> >> > wrote:
> >> > > >
> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> >> > > > >
> >> > > > > Gwen had independently mentioned this as a SASL mechanism that
> might
> >> > be
> >> > > > > useful for Kafka and I have been meaning to check it in more
> detail.
> >> > > Good
> >> > > > > to know that you are willing to contribute an implementation.
> Maybe
> >> > we
> >> > > > > should file a separate JIRA for this?
> >> > > > >
> >> > > > > Ismael
> >> > > > >
> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> >> > > > > rajinisivaram@googlemail.com> wrote:
> >> > > > >
> >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is
> a
> >> > > better
> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in
> SCRAM
> >> > > > > > SaslServer or SaslClient, but I will be happy to add support
> in
> >> > Kafka
> >> > > > > since
> >> > > > > > it would be a useful mechanism to support anyway.
> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol
> for
> >> > > > > > SCRAM-SHA-256.
> >> > > > > >
> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <ju...@confluent.io>
> wrote:
> >> > > > > >
> >> > > > > > > Parth,
> >> > > > > > >
> >> > > > > > > Thanks for the explanation. A couple of more questions.
> >> > > > > > >
> >> > > > > > > 110. What does getDelegationTokenAs mean?
> >> > > > > > >
> >> > > > > > > 111. What's the typical rate of getting and renewing
> delegation
> >> > > > tokens?
> >> > > > > > > That may have an impact on whether they should be directed
> to the
> >> > > > > > > controller.
> >> > > > > > >
> >> > > > > > > Jun
> >> > > > > > >
> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> >> > > > > > > brahmbhatt.parth@gmail.com> wrote:
> >> > > > > > >
> >> > > > > > > > Hi Jun,
> >> > > > > > > >
> >> > > > > > > > Thanks for reviewing.
> >> > > > > > > >
> >> > > > > > > > * We could add a Cluster action to add acls on who can
> request
> >> > > > > > delegation
> >> > > > > > > > tokens. I don't see the use case for that yet but down
> the line
> >> > > > when
> >> > > > > we
> >> > > > > > > > start supporting getDelegationTokenAs it will be
> necessary.
> >> > > > > > > > * Yes we recommend tokens to be only used/distributed over
> >> > secure
> >> > > > > > > channels.
> >> > > > > > > > * Depending on what design we end up choosing
> Invalidation will
> >> > > be
> >> > > > > > > > responsibility of every broker or controller.
> >> > > > > > > > * I am not sure if I documented somewhere that
> invalidation
> >> > will
> >> > > > > > directly
> >> > > > > > > > go through zookeeper but that is not the intent.
> Invalidation
> >> > > will
> >> > > > > > either
> >> > > > > > > > be request based or due to expiration. No direct zookeeper
> >> > > > > interaction
> >> > > > > > > from
> >> > > > > > > > any client.
> >> > > > > > > > * "Broker also stores the DelegationToken without the
> hmac in
> >> > the
> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole purpose
> of
> >> > > > > zookeeper
> >> > > > > > in
> >> > > > > > > > this design is as distribution channel for tokens between
> all
> >> > > > brokers
> >> > > > > > > and a
> >> > > > > > > > layer that ensures only tokens that were generated by
> making a
> >> > > > > request
> >> > > > > > > to a
> >> > > > > > > > broker will be accepted (more on this in second
> paragraph). The
> >> > > > token
> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> expiration,
> >> > > hmac)
> >> > > > ,
> >> > > > > > one
> >> > > > > > > of
> >> > > > > > > > which is the finally generated hmac but hmac it self is
> >> > derivable
> >> > > > if
> >> > > > > > you
> >> > > > > > > > have all the other elements of the token + secret key to
> >> > generate
> >> > > > > hmac.
> >> > > > > > > > Given zookeeper does not provide SSL support we do not
> want the
> >> > > > > entire
> >> > > > > > > > token to be wire transferred to zookeeper as that will be
> an
> >> > > > insecure
> >> > > > > > > wire
> >> > > > > > > > transfer. Instead we only store all the other elements of
> a
> >> > > > > delegation
> >> > > > > > > > tokens. Brokers can read these elements and because they
> also
> >> > > have
> >> > > > > > access
> >> > > > > > > > to secret key they will be able to generate hmac on their
> end.
> >> > > > > > > >
> >> > > > > > > > One of the alternative proposed is to avoid zookeeper
> >> > > altogether. A
> >> > > > > > > Client
> >> > > > > > > > will call broker with required information (owner, renwer,
> >> > > > > expiration)
> >> > > > > > > and
> >> > > > > > > > get back (signed hmac, uuid). Broker won't store this in
> >> > > zookeeper.
> >> > > > > > From
> >> > > > > > > > this point a client can contact any broker with all the
> >> > > delegation
> >> > > > > > token
> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker
> will
> >> > > > > regenerate
> >> > > > > > > the
> >> > > > > > > > hmac and as long as it matches with hmac presented by
> client ,
> >> > > > broker
> >> > > > > > > will
> >> > > > > > > > allow the request to authenticate. Only problem with this
> >> > > approach
> >> > > > > is
> >> > > > > > if
> >> > > > > > > > the secret key is compromised any client can now generate
> >> > random
> >> > > > > tokens
> >> > > > > > > and
> >> > > > > > > > they will still be able to authenticate as any user they
> like.
> >> > > with
> >> > > > > > > > zookeeper we guarantee that only tokens acquired via a
> broker
> >> > > > (using
> >> > > > > > some
> >> > > > > > > > auth scheme other than delegation token) will be
> accepted. We
> >> > > need
> >> > > > to
> >> > > > > > > > discuss which proposal makes more sense and we can go
> over it
> >> > in
> >> > > > > > > tomorrow's
> >> > > > > > > > meeting.
> >> > > > > > > >
> >> > > > > > > > Also, can you forward the invite to me?
> >> > > > > > > >
> >> > > > > > > > Thanks
> >> > > > > > > > Parth
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> jun@confluent.io>
> >> > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Thanks for the KIP. A few comments.
> >> > > > > > > > >
> >> > > > > > > > > 100. This potentially can be useful for Kafka Connect
> and
> >> > Kafka
> >> > > > > rest
> >> > > > > > > > proxy
> >> > > > > > > > > where a worker agent will need to run a task on behalf
> of a
> >> > > > client.
> >> > > > > > We
> >> > > > > > > > will
> >> > > > > > > > > likely need to change how those services use Kafka
> clients
> >> > > > > > > > > (producer/consumer). Instead of a shared client per
> worker,
> >> > we
> >> > > > will
> >> > > > > > > need
> >> > > > > > > > a
> >> > > > > > > > > client per user task since the authentication happens
> at the
> >> > > > > > connection
> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
> workers.
> >> > So,
> >> > > we
> >> > > > > > > > probably
> >> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy,
> the
> >> > > > renewer
> >> > > > > > can
> >> > > > > > > > > probably just be the creator of the token.
> >> > > > > > > > >
> >> > > > > > > > > 101. Do we need new acl on who can request delegation
> tokens?
> >> > > > > > > > >
> >> > > > > > > > > 102. Do we recommend people to send delegation tokens
> in an
> >> > > > > encrypted
> >> > > > > > > > > channel?
> >> > > > > > > > >
> >> > > > > > > > > 103. Who is responsible for expiring tokens, every
> broker?
> >> > > > > > > > >
> >> > > > > > > > > 104. For invalidating tokens, would it be better to do
> it in
> >> > a
> >> > > > > > request
> >> > > > > > > > > instead of going to ZK directly?
> >> > > > > > > > >
> >> > > > > > > > > 105. The terminology of client in the wiki sometimes
> refers
> >> > to
> >> > > > the
> >> > > > > > end
> >> > > > > > > > > client and some other times refers to the client using
> the
> >> > > > > delegation
> >> > > > > > > > > tokens. It would be useful to distinguish between the
> two.
> >> > > > > > > > >
> >> > > > > > > > > 106. Could you explain the sentence "Broker also stores
> the
> >> > > > > > > > DelegationToken
> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
> thought the
> >> > > > > > > delegation
> >> > > > > > > > > token is the hmac.
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > >
> >> > > > > > > > > Jun
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> jun@confluent.io>
> >> > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Hi, Harsha,
> >> > > > > > > > > >
> >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss
> this in
> >> > > the
> >> > > > > > > meeting
> >> > > > > > > > > > tomorrow.
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > >
> >> > > > > > > > > > Jun
> >> > > > > > > > > >
> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> kafka@harsha.io>
> >> > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > >> Hi All,
> >> > > > > > > > > >> Can we have a KIP meeting around this.
> The KIP
> >> > is
> >> > > > up
> >> > > > > > for
> >> > > > > > > > > >> sometime and if there are any questions
> lets
> >> > > > quickly
> >> > > > > > hash
> >> > > > > > > > out
> >> > > > > > > > > >> details.
> >> > > > > > > > > >>
> >> > > > > > > > > >> Thanks,
> >> > > > > > > > > >> Harsha
> >> > > > > > > > > >>
> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt
> wrote:
> >> > > > > > > > > >> > That is what the hadoop echo system uses so no good
> >> > reason
> >> > > > > > really.
> >> > > > > > > > We
> >> > > > > > > > > >> > could
> >> > > > > > > > > >> > change it to whatever is the newest recommended
> standard
> >> > > is.
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > Thanks
> >> > > > > > > > > >> > Parth
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
> >> > > > > ismael@juma.me.uk
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > > Hi Parth,
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing
> this and
> >> > > may
> >> > > > > have
> >> > > > > > > > > >> additional
> >> > > > > > > > > >> > > questions later. The immediate question that
> came to
> >> > > mind
> >> > > > is
> >> > > > > > our
> >> > > > > > > > > >> choice of
> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE
> in
> >> > the
> >> > > > IANA
> >> > > > > > > > > Registry
> >> > > > > > > > > >> of
> >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has
> been
> >> > > moved
> >> > > > > to
> >> > > > > > > > > Historic
> >> > > > > > > > > >> > > status:
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> >> > > > > > > > > >> > >
> >> > > > > > > > >
> >> > > > > >
> >> > >
> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > What is the reasoning behind that choice?
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > Thanks,
> >> > > > > > > > > >> > > Ismael
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira <
> >> > > > > > > gwen@confluent.io
> >> > > > > > > > >
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > > Also comments inline :)
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > > * I want to emphasize that even though
> delegation
> >> > > > tokens
> >> > > > > > > are a
> >> > > > > > > > > >> Hadoop
> >> > > > > > > > > >> > > > > innovation, I feel very strongly about not
> adding
> >> > > > > > dependency
> >> > > > > > > > on
> >> > > > > > > > > >> Hadoop
> >> > > > > > > > > >> > > > > when implementing delegation tokens for
> Kafka. The
> >> > > KIP
> >> > > > > > > doesn't
> >> > > > > > > > > >> imply
> >> > > > > > > > > >> > > > > such dependency, but if you can clarify...
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will
> read
> >> > the
> >> > > > KIP
> >> > > > > > and
> >> > > > > > > > > panic
> >> > > > > > > > > >> > > > three weeks before the next release...
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > > * Can we get delegation token at any time
> after
> >> > > > > > > > authenticating?
> >> > > > > > > > > >> only
> >> > > > > > > > > >> > > > > immediately after?
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *As long as you are authenticated you can get
> >> > > > delegation
> >> > > > > > > > tokens.
> >> > > > > > > > > >> We
> >> > > > > > > > > >> > > need
> >> > > > > > > > > >> > > > to
> >> > > > > > > > > >> > > > > discuss if a client authenticated using
> delegation
> >> > > > > token,
> >> > > > > > > can
> >> > > > > > > > > also
> >> > > > > > > > > >> > > > acquire
> >> > > > > > > > > >> > > > > delegation token again or not. Also there is
> the
> >> > > > > question
> >> > > > > > of
> >> > > > > > > > do
> >> > > > > > > > > we
> >> > > > > > > > > >> > > allow
> >> > > > > > > > > >> > > > > anyone to acquire delegation token or we want
> >> > > specific
> >> > > > > > ACLs
> >> > > > > > > (I
> >> > > > > > > > > >> think
> >> > > > > > > > > >> > > its
> >> > > > > > > > > >> > > > an
> >> > > > > > > > > >> > > > > overkill.)*
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > I think we are debating two options: Either
> require
> >> > > > > Kerberos
> >> > > > > > > > auth
> >> > > > > > > > > >> for
> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
> >> > > > > > > > > >> > > > I *think* the latter is simpler (it basically
> >> > require
> >> > > a
> >> > > > > "job
> >> > > > > > > > > master"
> >> > > > > > > > > >> > > > to take responsibility for the renewal, it
> will have
> >> > > its
> >> > > > > own
> >> > > > > > > > > >> identity
> >> > > > > > > > > >> > > > anyway and I think this is the correct design
> >> > pattern
> >> > > > > > anyway.
> >> > > > > > > > For
> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> renewals?),
> >> > but
> >> > > > it
> >> > > > > is
> >> > > > > > > > hard
> >> > > > > > > > > to
> >> > > > > > > > > >> > > > debate simplicity without looking at the code
> >> > changes
> >> > > > > > > required.
> >> > > > > > > > If
> >> > > > > > > > > >> you
> >> > > > > > > > > >> > > > have a draft of how the "require Kerberos"
> will look
> >> > > in
> >> > > > > > Kafka
> >> > > > > > > > > code,
> >> > > > > > > > > >> > > > I'll be happy to take a look.
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > > * My understanding is that tokens will
> propagate
> >> > via
> >> > > > ZK
> >> > > > > > but
> >> > > > > > > > > >> without
> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> protocol,
> >> > > > correct?
> >> > > > > > > > Clients
> >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure
> (IIRC),
> >> > > but
> >> > > > > > since
> >> > > > > > > > the
> >> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we
> will
> >> > > need
> >> > > > to
> >> > > > > > > > retry a
> >> > > > > > > > > >> bit
> >> > > > > > > > > >> > > > > to avoid clients failing auth due to timing
> >> > issues.
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right now.
> The
> >> > > > current
> >> > > > > > > > > documented
> >> > > > > > > > > >> > > > approach
> >> > > > > > > > > >> > > > > is zookeeper based and it does not require
> any
> >> > > changes
> >> > > > > to
> >> > > > > > > > > >> > > UpdateMetadata
> >> > > > > > > > > >> > > > > protocol. An alternative approach can remove
> >> > > zookeeper
> >> > > > > > > > > dependency
> >> > > > > > > > > >> as
> >> > > > > > > > > >> > > well
> >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion
> call.*
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping
> Jun to
> >> > > > > > arrange a
> >> > > > > > > > > call?
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just
> the
> >> > > > > > controller
> >> > > > > > > > > issue
> >> > > > > > > > > >> the
> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared
> >> > secret.
> >> > > > Not
> >> > > > > > > sure
> >> > > > > > > > if
> >> > > > > > > > > >> we
> >> > > > > > > > > >> > > > > want to continue the discussion here or on
> the
> >> > > wiki. I
> >> > > > > > think
> >> > > > > > > > > that
> >> > > > > > > > > >> we
> >> > > > > > > > > >> > > > > can decouple the problem of "token
> distribution"
> >> > > from
> >> > > > > > > "shared
> >> > > > > > > > > >> secret
> >> > > > > > > > > >> > > > > distribution" and use the controller as the
> only
> >> > > token
> >> > > > > > > > generator
> >> > > > > > > > > >> to
> >> > > > > > > > > >> > > > > solve the second issue, while still using ZK
> async
> >> > > to
> >> > > > > > > > distribute
> >> > > > > > > > > >> > > > > tokens.
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am
> fine with
> >> > > > that
> >> > > > > > > > approach
> >> > > > > > > > > >> as
> >> > > > > > > > > >> > > long
> >> > > > > > > > > >> > > > as
> >> > > > > > > > > >> > > > > we agree that the extra complexity of
> >> > > adding/updating
> >> > > > > APIS
> >> > > > > > > > adds
> >> > > > > > > > > >> enough
> >> > > > > > > > > >> > > > > value. The advantage with the controller
> approach
> >> > is
> >> > > > > > secret
> >> > > > > > > > > >> rotation
> >> > > > > > > > > >> > > can
> >> > > > > > > > > >> > > > be
> >> > > > > > > > > >> > > > > automated,frequent and would not require
> >> > > deployment. *
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Can you detail the extra complexity (or point
> me to
> >> > > the
> >> > > > > > email
> >> > > > > > > I
> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> >> > > > > > > > > >> > > > As far as I can tell, clients can already find
> the
> >> > > > > > controller
> >> > > > > > > > from
> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
> controller
> >> > > > load.
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos
> auth
> >> > for
> >> > > > > > > renwal, I
> >> > > > > > > > > >> think
> >> > > > > > > > > >> > > > > it mixes the transport layer the the request
> >> > content
> >> > > > in
> >> > > > > a
> >> > > > > > > > pretty
> >> > > > > > > > > >> ugly
> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is
> >> > > better.
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While this
> will
> >> > > make
> >> > > > > the
> >> > > > > > > > kafka
> >> > > > > > > > > >> code
> >> > > > > > > > > >> > > > > pretty straight forward , forcing renewer to
> >> > > > non-owner
> >> > > > > > > pushes
> >> > > > > > > > > >> the code
> >> > > > > > > > > >> > > > > ugliness to client and makes it even harder
> to
> >> > > > > > integrate. *
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> "renewal by
> >> > > > other"
> >> > > > > > > > approach
> >> > > > > > > > > >> is
> >> > > > > > > > > >> > > > that ugly for the clients we expect to use
> >> > delegation
> >> > > > > tokens
> >> > > > > > > > since
> >> > > > > > > > > >> > > > they will have an app-master of some sort who
> >> > > requested
> >> > > > > the
> >> > > > > > > > token
> >> > > > > > > > > to
> >> > > > > > > > > >> > > > begin with.
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > The response for my question on how multiple
> >> > > > identities
> >> > > > > > will
> >> > > > > > > > be
> >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we
> don't
> >> > > > > > > > authenticate
> >> > > > > > > > > >> each
> >> > > > > > > > > >> > > > > request, we authenticate connections.
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > *We authenticate connections, and only when
> they
> >> > are
> >> > > > > being
> >> > > > > > > > > >> established.
> >> > > > > > > > > >> > > > Let
> >> > > > > > > > > >> > > > > me try to phrase this as a question, in
> absence of
> >> > > > > > > delegation
> >> > > > > > > > > >> tokens if
> >> > > > > > > > > >> > > > we
> >> > > > > > > > > >> > > > > had to support the use case using user TGT's
> how
> >> > > would
> >> > > > > we
> >> > > > > > do
> >> > > > > > > > it?
> >> > > > > > > > > >> My
> >> > > > > > > > > >> > > point
> >> > > > > > > > > >> > > > > was it would be no different with delegation
> >> > tokens.
> >> > > > The
> >> > > > > > use
> >> > > > > > > > > case
> >> > > > > > > > > >> you
> >> > > > > > > > > >> > > are
> >> > > > > > > > > >> > > > > describing seems more like impersonation.*
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Yeah, I thought that one of the things that
> >> > delegation
> >> > > > > > tokens
> >> > > > > > > > > >> handled.
> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > Gwen
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > > > > Thanks
> >> > > > > > > > > >> > > > > Parth
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
> Shapira <
> >> > > > > > > > > gwen@confluent.io
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > > > wrote:
> >> > > > > > > > > >> > > > >
> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> Few more comments:
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * The API / RequestResponse section doesn't
> seem
> >> > to
> >> > > > > have
> >> > > > > > > good
> >> > > > > > > > > >> > > > >> description of the changes to the Kafka
> Protocol.
> >> > > > > Sounds
> >> > > > > > > like
> >> > > > > > > > > >> you are
> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
> >> > > > > > RenewTokenRequest
> >> > > > > > > > (and
> >> > > > > > > > > >> > > > >> matching responses), without detailing the
> >> > contents
> >> > > > of
> >> > > > > > the
> >> > > > > > > > > >> requests
> >> > > > > > > > > >> > > > >> and responses? Or rather, you show the class
> >> > > > interface,
> >> > > > > > but
> >> > > > > > > > not
> >> > > > > > > > > >> the
> >> > > > > > > > > >> > > > >> underlying protocol. This could be seen as
> an
> >> > > > > > > implementation
> >> > > > > > > > > >> detail,
> >> > > > > > > > > >> > > > >> but since the binary protocol is what we
> provide
> >> > to
> >> > > > > > > non-Java
> >> > > > > > > > > >> clients,
> >> > > > > > > > > >> > > > >> we need to show the changes there.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> >> > > > > > > > delegationTokenRequestHandler?
> >> > > > > > > > > >> Is it
> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client?
> Its
> >> > > > > unclear...
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * I want to emphasize that even though
> delegation
> >> > > > > tokens
> >> > > > > > > are
> >> > > > > > > > a
> >> > > > > > > > > >> Hadoop
> >> > > > > > > > > >> > > > >> innovation, I feel very strongly about not
> adding
> >> > > > > > > dependency
> >> > > > > > > > on
> >> > > > > > > > > >> Hadoop
> >> > > > > > > > > >> > > > >> when implementing delegation tokens for
> Kafka.
> >> > The
> >> > > > KIP
> >> > > > > > > > doesn't
> >> > > > > > > > > >> imply
> >> > > > > > > > > >> > > > >> such dependency, but if you can clarify...
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * Can we get delegation token at any time
> after
> >> > > > > > > > authenticating?
> >> > > > > > > > > >> only
> >> > > > > > > > > >> > > > >> immediately after?
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * My understanding is that tokens will
> propagate
> >> > > via
> >> > > > ZK
> >> > > > > > but
> >> > > > > > > > > >> without
> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
> protocol,
> >> > > > correct?
> >> > > > > > > > Clients
> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure
> >> > (IIRC),
> >> > > > but
> >> > > > > > > since
> >> > > > > > > > > the
> >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch, we
> will
> >> > > need
> >> > > > > to
> >> > > > > > > > retry
> >> > > > > > > > > a
> >> > > > > > > > > >> bit
> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing
> >> > issues.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not touching
> ZK
> >> > > > directly
> >> > > > > > :)
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having
> just the
> >> > > > > > controller
> >> > > > > > > > > >> issue the
> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared
> >> > > secret.
> >> > > > > Not
> >> > > > > > > sure
> >> > > > > > > > > if
> >> > > > > > > > > >> we
> >> > > > > > > > > >> > > > >> want to continue the discussion here or on
> the
> >> > > wiki.
> >> > > > I
> >> > > > > > > think
> >> > > > > > > > > >> that we
> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> distribution"
> >> > > from
> >> > > > > > > "shared
> >> > > > > > > > > >> secret
> >> > > > > > > > > >> > > > >> distribution" and use the controller as the
> only
> >> > > > token
> >> > > > > > > > > generator
> >> > > > > > > > > >> to
> >> > > > > > > > > >> > > > >> solve the second issue, while still using ZK
> >> > async
> >> > > to
> >> > > > > > > > > distribute
> >> > > > > > > > > >> > > > >> tokens.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite
> lifetime
> >> > of
> >> > > > > > tokens
> >> > > > > > > > (and
> >> > > > > > > > > >> hoped
> >> > > > > > > > > >> > > > >> to hear from others in the community) - but
> >> > having
> >> > > > the
> >> > > > > > > option
> >> > > > > > > > > and
> >> > > > > > > > > >> > > > >> documenting it as less secure, allows users
> to
> >> > > > > configure
> >> > > > > > > > their
> >> > > > > > > > > >> system
> >> > > > > > > > > >> > > > >> as the wish.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * While I like the idea of forcing kerberos
> auth
> >> > > for
> >> > > > > > > renwal,
> >> > > > > > > > I
> >> > > > > > > > > >> think
> >> > > > > > > > > >> > > > >> it mixes the transport layer the the request
> >> > > content
> >> > > > > in a
> >> > > > > > > > > pretty
> >> > > > > > > > > >> ugly
> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner
> is
> >> > > better.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * More detailed explanation on what we plan
> to do
> >> > > for
> >> > > > > the
> >> > > > > > > > java
> >> > > > > > > > > >> clients
> >> > > > > > > > > >> > > > >> specifically - new configuration? new APIs?
> >> > > > > > > > > >> > > > >> The response for my question on how multiple
> >> > > > identities
> >> > > > > > > will
> >> > > > > > > > be
> >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we
> >> > don't
> >> > > > > > > > authenticate
> >> > > > > > > > > >> each
> >> > > > > > > > > >> > > > >> request, we authenticate connections.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only
> used
> >> > in
> >> > > > the
> >> > > > > > > Hadoop
> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> >> > alternatives
> >> > > in
> >> > > > > > other
> >> > > > > > > > > >> ecosystems
> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether
> there
> >> > are
> >> > > > some
> >> > > > > > > > > >> advantages
> >> > > > > > > > > >> > > > >> there.
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> Gwen
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
> >> > > > > kafka@harsha.io
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > > >> > > > >> > Hi Gwen,
> >> > > > > > > > > >> > > > >> > Can you look at Parth's last
> reply.
> >> > > Does
> >> > > > > it
> >> > > > > > > > answer
> >> > > > > > > > > >> your
> >> > > > > > > > > >> > > > >> > concerns.
> >> > > > > > > > > >> > > > >> >
> >> > > > > > > > > >> > > > >> > Thanks,
> >> > > > > > > > > >> > > > >> > Harsha
> >> > > > > > > > > >> > > > >> >
> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
> >> > > brahmbhatt
> >> > > > > > wrote:
> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki
> already
> >> > has
> >> > > > > > details
> >> > > > > > > on
> >> > > > > > > > > >> token
> >> > > > > > > > > >> > > > >> >> expiration
> >> > > > > > > > > >> > > > >> >> under token acquisition process
> >> > > > > > > > > >> > > > >> >> <
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> >> > > > > > > > > >> > > > >> >.
> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will
> expire
> >> > > based
> >> > > > > on a
> >> > > > > > > > > server
> >> > > > > > > > > >> side
> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless
> >> > renewed.
> >> > > > > > Renewal
> >> > > > > > > > is
> >> > > > > > > > > >> only
> >> > > > > > > > > >> > > > allowed
> >> > > > > > > > > >> > > > >> >> until the max life time of token.
> >> > Alternatively
> >> > > we
> >> > > > > > could
> >> > > > > > > > > also
> >> > > > > > > > > >> make
> >> > > > > > > > > >> > > > that
> >> > > > > > > > > >> > > > >> >> an
> >> > > > > > > > > >> > > > >> >> optional param and the server side
> default can
> >> > > > serve
> >> > > > > > as
> >> > > > > > > > the
> >> > > > > > > > > >> upper
> >> > > > > > > > > >> > > > bound.
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >> To your second point it will be done
> exactly
> >> > the
> >> > > > > same
> >> > > > > > > way
> >> > > > > > > > we
> >> > > > > > > > > >> would
> >> > > > > > > > > >> > > > >> >> support
> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client
> will have
> >> > > to
> >> > > > > put
> >> > > > > > > the
> >> > > > > > > > > >> tokens it
> >> > > > > > > > > >> > > > >> wants
> >> > > > > > > > > >> > > > >> >> to use in the subject instance and call
> >> > > > > > produce/consume
> >> > > > > > > > > inside
> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to
> keep
> >> > > track
> >> > > > of
> >> > > > > > its
> >> > > > > > > > own
> >> > > > > > > > > >> > > > subject. I
> >> > > > > > > > > >> > > > >> >> will have to look at the code to see if
> we
> >> > > support
> >> > > > > > this
> >> > > > > > > > > >> feature
> >> > > > > > > > > >> > > right
> >> > > > > > > > > >> > > > >> now
> >> > > > > > > > > >> > > > >> >> but my understanding is delegation token
> >> > > shouldn't
> >> > > > > > need
> >> > > > > > > > any
> >> > > > > > > > > >> special
> >> > > > > > > > > >> > > > >> >> treatment as its just another type of
> >> > Credential
> >> > > > in
> >> > > > > > the
> >> > > > > > > > > >> subject.
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >> I would also like to know what is your
> opinion
> >> > > > about
> >> > > > > > > > > infinite
> >> > > > > > > > > >> > > renewal
> >> > > > > > > > > >> > > > >> (my
> >> > > > > > > > > >> > > > >> >> recommendation is to not support this),
> tokens
> >> > > > > > renewing
> >> > > > > > > > them
> >> > > > > > > > > >> > > self(my
> >> > > > > > > > > >> > > > >> >> recommendation is to not support this)
> and
> >> > most
> >> > > > > > > > importantly
> >> > > > > > > > > >> your
> >> > > > > > > > > >> > > > choice
> >> > > > > > > > > >> > > > >> >> between the alternatives listed on this
> thread
> >> > > > > > > > > >> > > > >> >> <
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> >> > > > > > > > > >> > > > >> >
> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2
> minus
> >> > > > > > > controller
> >> > > > > > > > > >> > > > distributing
> >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >> Thanks
> >> > > > > > > > > >> > > > >> >> Parth
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen
> Shapira <
> >> > > > > > > > > >> gwen@confluent.io>
> >> > > > > > > > > >> > > > wrote:
> >> > > > > > > > > >> > > > >> >>
> >> > > > > > > > > >> > > > >> >> > Harsha,
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I
> didn't
> >> > see
> >> > > > > your
> >> > > > > > > > design
> >> > > > > > > > > >> yet,
> >> > > > > > > > > >> > > > but in
> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of producers,
> which
> >> > > > will
> >> > > > > > > serve
> >> > > > > > > > > >> multiple
> >> > > > > > > > > >> > > > users
> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these
> users
> >> > > will
> >> > > > > have
> >> > > > > > > > > >> different
> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> authenticate
> >> > > > > separately,
> >> > > > > > > and
> >> > > > > > > > > >> can't
> >> > > > > > > > > >> > > > share a
> >> > > > > > > > > >> > > > >> >> > token.
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >> >> > Gwen
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha
> <
> >> > > > > > > kafka@harsha.io
> >> > > > > > > > >
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >> > > > >> >> > > Gwen,
> >> > > > > > > > > >> > > > >> >> > > On your second point. Can
> you
> >> > > > > describe
> >> > > > > > a
> >> > > > > > > > > >> usecase
> >> > > > > > > > > >> > > where
> >> > > > > > > > > >> > > > >> >> > > mutliple clients ended up
> >> > > sharing a
> >> > > > > > > > producer
> >> > > > > > > > > >> and
> >> > > > > > > > > >> > > even
> >> > > > > > > > > >> > > > if
> >> > > > > > > > > >> > > > >> they
> >> > > > > > > > > >> > > > >> >> > > do why can't they not use
> >> > single
> >> > > > > token
> >> > > > > > > that
> >> > > > > > > > > >> producer
> >> > > > > > > > > >> > > > >> >> > > captures. Why would we
> need
> >> > > > multiple
> >> > > > > > > > clients
> >> > > > > > > > > >> with
> >> > > > > > > > > >> > > > >> different
> >> > > > > > > > > >> > > > >> >> > > tokens sharing a single
> >> > instance
> >> > > of
> >> > > > > > > > producer.
> >> > > > > > > > > >> Also
> >> > > > > > > > > >> > > in
> >> > > > > > > > > >> > > > >> this
> >> > > > > > > > > >> > > > >> >> > > case other clients have
> access
> >> > > all
> >> > > > > the
> >> > > > > > > > tokens
> >> > > > > > > > > >> no?
> >> > > > > > > > > >> > > > >> >> > >
> >> > > > > > > > > >> > > > >> >> > > Thanks,
> >> > > > > > > > > >> > > > >> >> > > Harsha
> >> > > > > > > > > >> > > > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >
> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM,
> Gwen
> >> > > Shapira
> >> > > > > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> >> > > > > > > > > >> > > > >> >> > >>
> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in
> the
> >> > > wiki:
> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
> delegation
> >> > > > > tokens?
> >> > > > > > > > > >> Renewal? How
> >> > > > > > > > > >> > > > do we
> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation
> tokens
> >> > for
> >> > > > > > "do-as"
> >> > > > > > > > > (say,
> >> > > > > > > > > >> > > submit
> >> > > > > > > > > >> > > > >> Storm
> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a
> producer
> >> > > for
> >> > > > > > every
> >> > > > > > > > job
> >> > > > > > > > > >> (we
> >> > > > > > > > > >> > > can't
> >> > > > > > > > > >> > > > >> share
> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running
> on
> >> > same
> >> > > > > node),
> >> > > > > > > > since
> >> > > > > > > > > >> we
> >> > > > > > > > > >> > > only
> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is
> there a
> >> > > plan
> >> > > > > to
> >> > > > > > > > change
> >> > > > > > > > > >> this
> >> > > > > > > > > >> > > for
> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow
> >> > > multiple
> >> > > > > > users
> >> > > > > > > > > with
> >> > > > > > > > > >> > > > different
> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> >> > > > > > > > > >> > > > >> >> > >>
> >> > > > > > > > > >> > > > >> >> > >> Gwen
> >> > > > > > > > > >> > > > >> >> > >>
> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM,
> parth
> >> > > > > brahmbhatt
> >> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com> wrote:
> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can
> >> > other
> >> > > > > > > committers
> >> > > > > > > > > >> review?
> >> > > > > > > > > >> > > > >> >> > >> >
> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> >> > > > > > > > > >> > > > >> >> > >> > Parth
> >> > > > > > > > > >> > > > >> >> > >> >
> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM,
> >> > Harsha <
> >> > > > > > > > > >> kafka@harsha.io>
> >> > > > > > > > > >> > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> >> > > > > > > > > >> > > > >> >> > >> >> Overall current design
> looks
> >> > > > good
> >> > > > > to
> >> > > > > > > > me. I
> >> > > > > > > > > >> am +1
> >> > > > > > > > > >> > > on
> >> > > > > > > > > >> > > > >> the
> >> > > > > > > > > >> > > > >> >> > KIP.
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as
> >> > well.
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57
> AM,
> >> > parth
> >> > > > > > > > brahmbhatt
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
> infinite
> >> > > > lifetime
> >> > > > > > > but I
> >> > > > > > > > > >> see the
> >> > > > > > > > > >> > > > >> Streaming
> >> > > > > > > > > >> > > > >> >> > use
> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use
> case I
> >> > > was
> >> > > > > > hoping
> >> > > > > > > > > >> there will
> >> > > > > > > > > >> > > > be
> >> > > > > > > > > >> > > > >> some
> >> > > > > > > > > >> > > > >> >> > notion
> >> > > > > > > > > >> > > > >> >> > >> >> > of
> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new
> >> > > > delegation
> >> > > > > > > tokens
> >> > > > > > > > > at
> >> > > > > > > > > >> fixed
> >> > > > > > > > > >> > > > >> interval
> >> > > > > > > > > >> > > > >> >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that
> is
> >> > not
> >> > > > the
> >> > > > > > case
> >> > > > > > > > for
> >> > > > > > > > > >> we can
> >> > > > > > > > > >> > > > >> discuss
> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing
> them self
> >> > > and
> >> > > > > the
> >> > > > > > > > > >> security
> >> > > > > > > > > >> > > > >> implications
> >> > > > > > > > > >> > > > >> >> > of the
> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch
> >> > tokens
> >> > > > > from
> >> > > > > > > > > >> zookeeper,
> >> > > > > > > > > >> > > > >> overall I
> >> > > > > > > > > >> > > > >> >> > think
> >> > > > > > > > > >> > > > >> >> > >> >> > its
> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely
> on our
> >> > > > > metadata
> >> > > > > > > > store
> >> > > > > > > > > >> and I
> >> > > > > > > > > >> > > > >> think we
> >> > > > > > > > > >> > > > >> >> > are
> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with
> all
> >> > the
> >> > > > > KIP-4
> >> > > > > > > > > >> improvements.
> >> > > > > > > > > >> > > > I
> >> > > > > > > > > >> > > > >> chose
> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the
> client
> >> > > > will
> >> > > > > > > still
> >> > > > > > > > > >> just talk
> >> > > > > > > > > >> > > > to
> >> > > > > > > > > >> > > > >> >> > broker , its
> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use
> zookeeper
> >> > > > which
> >> > > > > we
> >> > > > > > > > > >> already do
> >> > > > > > > > > >> > > > for a
> >> > > > > > > > > >> > > > >> lot
> >> > > > > > > > > >> > > > >> >> > of
> >> > > > > > > > > >> > > > >> >> > >> >> > other
> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development
> + and
> >> > > the
> >> > > > > > > ability
> >> > > > > > > > > so
> >> > > > > > > > > >> > > tokens
> >> > > > > > > > > >> > > > >> will
> >> > > > > > > > > >> > > > >> >> > survive
> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster
> >> > > failure.
> >> > > > > if a
> >> > > > > > > > > >> majority
> >> > > > > > > > > >> > > > agrees
> >> > > > > > > > > >> > > > >> the
> >> > > > > > > > > >> > > > >> >> > added
> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller
> >> > > forwarding
> >> > > > > > keys
> >> > > > > > > to
> >> > > > > > > > > all
> >> > > > > > > > > >> > > > broker is
> >> > > > > > > > > >> > > > >> >> > justified
> >> > > > > > > > > >> > > > >> >> > >> >> > as
> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security ,
> I am
> >> > > fine
> >> > > > > > with
> >> > > > > > > > that
> >> > > > > > > > > >> option
> >> > > > > > > > > >> > > > too.
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
> support SSL
> >> > > we
> >> > > > > can
> >> > > > > > > not
> >> > > > > > > > > >> store
> >> > > > > > > > > >> > > > master
> >> > > > > > > > > >> > > > >> keys
> >> > > > > > > > > >> > > > >> >> > in
> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will
> be
> >> > > exposed
> >> > > > > on
> >> > > > > > > > wire.
> >> > > > > > > > > To
> >> > > > > > > > > >> > > > support
> >> > > > > > > > > >> > > > >> >> > rotation
> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current
> clients is
> >> > > > > > > something I
> >> > > > > > > > > >> need to
> >> > > > > > > > > >> > > > put
> >> > > > > > > > > >> > > > >> more
> >> > > > > > > > > >> > > > >> >> > thought
> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
> assumes the
> >> > > > > rotation
> >> > > > > > > > will
> >> > > > > > > > > >> > > > invalidate
> >> > > > > > > > > >> > > > >> all
> >> > > > > > > > > >> > > > >> >> > current
> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also
> review
> >> > > and
> >> > > > > post
> >> > > > > > > > their
> >> > > > > > > > > >> > > comments
> >> > > > > > > > > >> > > > >> so we
> >> > > > > > > > > >> > > > >> >> > can
> >> > > > > > > > > >> > > > >> >> > >> >> > make
> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39
> AM,
> >> > > Ashish
> >> > > > > > Singh
> >> > > > > > > <
> >> > > > > > > > > >> > > > >> asingh@cloudera.com
> >> > > > > > > > > >> > > > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at
> 11:26 AM,
> >> > > > > Harsha <
> >> > > > > > > > > >> > > > kafka@harsha.io>
> >> > > > > > > > > >> > > > >> >> > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion
> >> > > threads
> >> > > > on
> >> > > > > > > this
> >> > > > > > > > > KIP.
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from
> >> > Jitendra
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large
> number of
> >> > > > > clients
> >> > > > > > > that
> >> > > > > > > > > are
> >> > > > > > > > > >> > > > running
> >> > > > > > > > > >> > > > >> all
> >> > > > > > > > > >> > > > >> >> > over the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate
> with
> >> > > Kafka
> >> > > > > > > > brokers,
> >> > > > > > > > > >> is very
> >> > > > > > > > > >> > > > >> similar
> >> > > > > > > > > >> > > > >> >> > to the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large
> number
> >> > of
> >> > > > > tasks
> >> > > > > > > > > running
> >> > > > > > > > > >> > > across
> >> > > > > > > > > >> > > > >> the
> >> > > > > > > > > >> > > > >> >> > cluster
> >> > > > > > > > > >> > > > >> >> > >> >> that
> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs
> >> > > > Namenode.
> >> > > > > > > > > >> Therefore, the
> >> > > > > > > > > >> > > > >> >> > delegation token
> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a
> good
> >> > fit
> >> > > > for
> >> > > > > > > this
> >> > > > > > > > > use
> >> > > > > > > > > >> case
> >> > > > > > > > > >> > > > as we
> >> > > > > > > > > >> > > > >> >> > have seen
> >> > > > > > > > > >> > > > >> >> > >> >> it
> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in
> HDFS
> >> > and
> >> > > > > YARN.
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is
> very
> >> > much
> >> > > > > > inline
> >> > > > > > > > with
> >> > > > > > > > > >> Hadoop
> >> > > > > > > > > >> > > > >> >> > approach. A few
> >> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to
> allow
> >> > > > > > infinite
> >> > > > > > > > > >> renewable
> >> > > > > > > > > >> > > > >> lifetime
> >> > > > > > > > > >> > > > >> >> > for a
> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a
> token
> >> > to a
> >> > > > max
> >> > > > > > > life
> >> > > > > > > > > time
> >> > > > > > > > > >> > > > (default
> >> > > > > > > > > >> > > > >> 7
> >> > > > > > > > > >> > > > >> >> > days). A
> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
> >> > believed
> >> > > to
> >> > > > > > > > increase
> >> > > > > > > > > >> with
> >> > > > > > > > > >> > > > time.
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite
> >> > > lifetime
> >> > > > > > might
> >> > > > > > > > not
> >> > > > > > > > > >> be the
> >> > > > > > > > > >> > > > best
> >> > > > > > > > > >> > > > >> idea.
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the
> tokens
> >> > are
> >> > > > > stored
> >> > > > > > > in
> >> > > > > > > > > >> zookeeper
> >> > > > > > > > > >> > > > as
> >> > > > > > > > > >> > > > >> well,
> >> > > > > > > > > >> > > > >> >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> can
> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is
> clever
> >> > > as
> >> > > > it
> >> > > > > > can
> >> > > > > > > > > allow
> >> > > > > > > > > >> > > > >> replacing the
> >> > > > > > > > > >> > > > >> >> > tokens
> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max
> life
> >> > > time,
> >> > > > > and
> >> > > > > > > > > clients
> >> > > > > > > > > >> can
> >> > > > > > > > > >> > > > >> download
> >> > > > > > > > > >> > > > >> >> > new
> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> shouldn't be
> >> > a
> >> > > > big
> >> > > > > > load
> >> > > > > > > > on
> >> > > > > > > > > >> > > zookeeper
> >> > > > > > > > > >> > > > >> as a
> >> > > > > > > > > >> > > > >> >> > client
> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token
> once in
> >> > > > > several
> >> > > > > > > > days.
> >> > > > > > > > > >> In this
> >> > > > > > > > > >> > > > >> approach
> >> > > > > > > > > >> > > > >> >> > you
> >> > > > > > > > > >> > > > >> >> > >> >> don't
> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on
> the
> >> > > token
> >> > > > > even
> >> > > > > > > for
> >> > > > > > > > > >> long
> >> > > > > > > > > >> > > > running
> >> > > > > > > > > >> > > > >> >> > clients.
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are
> >> > > generated
> >> > > > > > using
> >> > > > > > > a
> >> > > > > > > > > >> master
> >> > > > > > > > > >> > > key.
> >> > > > > > > > > >> > > > >> The
> >> > > > > > > > > >> > > > >> >> > master
> >> > > > > > > > > >> > > > >> >> > >> >> key
> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically
> >> > > changed.
> >> > > > > In
> >> > > > > > > > > Hadoop,
> >> > > > > > > > > >> the
> >> > > > > > > > > >> > > > >> default
> >> > > > > > > > > >> > > > >> >> > renewal
> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require
> brokers
> >> > > > > > maintaining
> >> > > > > > > a
> >> > > > > > > > > >> list of X
> >> > > > > > > > > >> > > > most
> >> > > > > > > > > >> > > > >> >> > recent
> >> > > > > > > > > >> > > > >> >> > >> >> master
> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to
> be
> >> > > > > persisted
> >> > > > > > > > > >> somewhere, as
> >> > > > > > > > > >> > > > if a
> >> > > > > > > > > >> > > > >> >> > broker
> >> > > > > > > > > >> > > > >> >> > >> >> goes
> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get
> that list
> >> > > > again
> >> > > > > > and
> >> > > > > > > > > >> storing
> >> > > > > > > > > >> > > > master
> >> > > > > > > > > >> > > > >> keys
> >> > > > > > > > > >> > > > >> >> > on ZK
> >> > > > > > > > > >> > > > >> >> > >> >> is
> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However,
> if a
> >> > > > broker
> >> > > > > > goes
> >> > > > > > > > > down
> >> > > > > > > > > >> then
> >> > > > > > > > > >> > > we
> >> > > > > > > > > >> > > > >> have
> >> > > > > > > > > >> > > > >> >> > much
> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
> client can
> >> > > > always
> >> > > > > > > > > >> > > re-authenticate
> >> > > > > > > > > >> > > > is
> >> > > > > > > > > >> > > > >> such
> >> > > > > > > > > >> > > > >> >> > >> >> events.
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a
> look at
> >> > > other
> >> > > > > > > > > >> alternatives
> >> > > > > > > > > >> > > this
> >> > > > > > > > > >> > > > >> list has
> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough
> proposal,
> >> > > > great
> >> > > > > > > work!"
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at
> 10:28
> >> > PM,
> >> > > > Gwen
> >> > > > > > > > Shapira
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
> Thanks!
> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at
> 9:25
> >> > PM,
> >> > > > > > Harsha <
> >> > > > > > > > > >> > > > kafka@harsha.io
> >> > > > > > > > > >> > > > >> >
> >> > > > > > > > > >> > > > >> >> > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any
> release
> >> > > > > vehicle
> >> > > > > > > but
> >> > > > > > > > > >> still the
> >> > > > > > > > > >> > > > >> work can
> >> > > > > > > > > >> > > > >> >> > move
> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> interested in
> >> > the
> >> > > > KIP
> >> > > > > > > > please
> >> > > > > > > > > >> do the
> >> > > > > > > > > >> > > > >> review and
> >> > > > > > > > > >> > > > >> >> > >> >> provide
> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at
> >> > 04:59
> >> > > > PM,
> >> > > > > > > Ismael
> >> > > > > > > > > >> Juma
> >> > > > > > > > > >> > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would
> be
> >> > good
> >> > > > to
> >> > > > > > have
> >> > > > > > > > > more
> >> > > > > > > > > >> time
> >> > > > > > > > > >> > > to
> >> > > > > > > > > >> > > > >> review
> >> > > > > > > > > >> > > > >> >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at
> >> > 12:55
> >> > > > AM,
> >> > > > > > Gwen
> >> > > > > > > > > >> Shapira <
> >> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
> depends on
> >> > > > KIP-43,
> >> > > > > > > which
> >> > > > > > > > > is
> >> > > > > > > > > >> > > > already a
> >> > > > > > > > > >> > > > >> bit
> >> > > > > > > > > >> > > > >> >> > of a
> >> > > > > > > > > >> > > > >> >> > >> >> risk
> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release -
> any
> >> > > chance
> >> > > > > we
> >> > > > > > > can
> >> > > > > > > > > >> delay
> >> > > > > > > > > >> > > > >> delegation
> >> > > > > > > > > >> > > > >> >> > tokens
> >> > > > > > > > > >> > > > >> >> > >> >> to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community
> >> > working
> >> > > > on a
> >> > > > > > > > release
> >> > > > > > > > > >> every
> >> > > > > > > > > >> > > 3
> >> > > > > > > > > >> > > > >> month,
> >> > > > > > > > > >> > > > >> >> > this
> >> > > > > > > > > >> > > > >> >> > >> >> is not
> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26,
> 2016 at
> >> > > 5:11
> >> > > > > PM,
> >> > > > > > > > Ashish
> >> > > > > > > > > >> Singh
> >> > > > > > > > > >> > > <
> >> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for
> the
> >> > > > awesome
> >> > > > > > > write
> >> > > > > > > > > up.
> >> > > > > > > > > >> > > > Following
> >> > > > > > > > > >> > > > >> our
> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it
> will
> >> > be
> >> > > > > easier
> >> > > > > > > to
> >> > > > > > > > > >> compare
> >> > > > > > > > > >> > > > >> various
> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together.
> I am
> >> > > > stating
> >> > > > > > > > below a
> >> > > > > > > > > >> few
> >> > > > > > > > > >> > > > >> >> > alternatives along
> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal)
> >> > Store
> >> > > > > > > Delegation
> >> > > > > > > > > >> Token,
> >> > > > > > > > > >> > > DT,
> >> > > > > > > > > >> > > > >> on ZK.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> > > authenticates
> >> > > > > > with a
> >> > > > > > > > > >> broker.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> client is
> >> > > > > > > > authenticated,
> >> > > > > > > > > >> it
> >> > > > > > > > > >> > > will
> >> > > > > > > > > >> > > > >> make a
> >> > > > > > > > > >> > > > >> >> > broker
> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> delegation
> >> > > > token.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
> >> > > generates
> >> > > > a
> >> > > > > > > shared
> >> > > > > > > > > >> secret
> >> > > > > > > > > >> > > > based
> >> > > > > > > > > >> > > > >> on
> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret
> >> > shared
> >> > > > > > between
> >> > > > > > > > all
> >> > > > > > > > > >> > > brokers,
> >> > > > > > > > > >> > > > >> >> > randomly
> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker
> stores
> >> > this
> >> > > > > token
> >> > > > > > in
> >> > > > > > > > its
> >> > > > > > > > > >> in
> >> > > > > > > > > >> > > > memory
> >> > > > > > > > > >> > > > >> cache.
> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the
> DelegationToken
> >> > > > > without
> >> > > > > > > the
> >> > > > > > > > > >> hmac in
> >> > > > > > > > > >> > > the
> >> > > > > > > > > >> > > > >> >> > zookeeper.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers
> will
> >> > > > have a
> >> > > > > > > cache
> >> > > > > > > > > >> backed
> >> > > > > > > > > >> > > by
> >> > > > > > > > > >> > > > >> >> > zookeeper so
> >> > > > > > > > > >> > > > >> >> > >> >> they
> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified
> >> > whenever
> >> > > a
> >> > > > > new
> >> > > > > > > > token
> >> > > > > > > > > is
> >> > > > > > > > > >> > > > >> generated and
> >> > > > > > > > > >> > > > >> >> > they
> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache
> whenever
> >> > > > token
> >> > > > > > > state
> >> > > > > > > > > >> changes.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker
> returns
> >> > the
> >> > > > > token
> >> > > > > > to
> >> > > > > > > > > >> Client.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> and
> >> > fixes
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable
> race
> >> > > > > condition,
> >> > > > > > > > client
> >> > > > > > > > > >> tries
> >> > > > > > > > > >> > > to
> >> > > > > > > > > >> > > > >> >> > authenticate
> >> > > > > > > > > >> > > > >> >> > >> >> with
> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to
> be
> >> > > > updated
> >> > > > > > with
> >> > > > > > > > the
> >> > > > > > > > > >> newly
> >> > > > > > > > > >> > > > >> generated
> >> > > > > > > > > >> > > > >> >> > DT.
> >> > > > > > > > > >> > > > >> >> > >> >> This
> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with
> making
> >> > > > > dtRequest
> >> > > > > > > > block
> >> > > > > > > > > >> until
> >> > > > > > > > > >> > > all
> >> > > > > > > > > >> > > > >> >> > brokers have
> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk
> barrier or
> >> > > > > similar
> >> > > > > > > > > >> mechanism
> >> > > > > > > > > >> > > can
> >> > > > > > > > > >> > > > be
> >> > > > > > > > > >> > > > >> used.
> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will
> >> > > increase
> >> > > > > > > > > complexity.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a
> static
> >> > > secret
> >> > > > > key
> >> > > > > > > > from
> >> > > > > > > > > >> config
> >> > > > > > > > > >> > > > >> file. Will
> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and
> uses a
> >> > > static
> >> > > > > > > secret
> >> > > > > > > > > >> key. It
> >> > > > > > > > > >> > > is
> >> > > > > > > > > >> > > > >> advised
> >> > > > > > > > > >> > > > >> >> > to
> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> This
> >> > can
> >> > > > be
> >> > > > > > > > avoided
> >> > > > > > > > > >> with
> >> > > > > > > > > >> > > > >> controller
> >> > > > > > > > > >> > > > >> >> > >> >> generating
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to
> brokers
> >> > > > > > > periodically.
> >> > > > > > > > > >> However,
> >> > > > > > > > > >> > > > >> this will
> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain
> certain
> >> > > counts
> >> > > > of
> >> > > > > > > > > >> secretKeys.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1)
> Have
> >> > > > > controller
> >> > > > > > > > > >> generate
> >> > > > > > > > > >> > > > >> delegation
> >> > > > > > > > > >> > > > >> >> > token.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> > > authenticates
> >> > > > > > with a
> >> > > > > > > > > >> broker.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> client is
> >> > > > > > > > authenticated,
> >> > > > > > > > > >> it
> >> > > > > > > > > >> > > will
> >> > > > > > > > > >> > > > >> make a
> >> > > > > > > > > >> > > > >> >> > broker
> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> delegation
> >> > > > token.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker
> forwards
> >> > the
> >> > > > > > request
> >> > > > > > > > to
> >> > > > > > > > > >> > > > controller.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller
> >> > > generates
> >> > > > a
> >> > > > > DT
> >> > > > > > > and
> >> > > > > > > > > >> > > broadcasts
> >> > > > > > > > > >> > > > >> to all
> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker
> stores
> >> > this
> >> > > > > token
> >> > > > > > in
> >> > > > > > > > its
> >> > > > > > > > > >> memory
> >> > > > > > > > > >> > > > >> cache.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller
> >> > responds
> >> > > > to
> >> > > > > > > > broker’s
> >> > > > > > > > > >> DT
> >> > > > > > > > > >> > > req.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker
> returns
> >> > the
> >> > > > > token
> >> > > > > > to
> >> > > > > > > > > >> Client.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> and
> >> > fixes
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will
> have to
> >> > add
> >> > > > new
> >> > > > > > > APIs
> >> > > > > > > > to
> >> > > > > > > > > >> > > support
> >> > > > > > > > > >> > > > >> >> > controller
> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top
> of
> >> > the
> >> > > > > > minimal
> >> > > > > > > > APIs
> >> > > > > > > > > >> that
> >> > > > > > > > > >> > > are
> >> > > > > > > > > >> > > > >> >> > currently
> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will
> also have
> >> > > to
> >> > > > > add
> >> > > > > > > APIs
> >> > > > > > > > > to
> >> > > > > > > > > >> > > support
> >> > > > > > > > > >> > > > >> the
> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new
> broker
> >> > > comes
> >> > > > up
> >> > > > > > it
> >> > > > > > > > will
> >> > > > > > > > > >> have
> >> > > > > > > > > >> > > to
> >> > > > > > > > > >> > > > >> get all
> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In
> catastrophic
> >> > > > > failures
> >> > > > > > > > where
> >> > > > > > > > > >> all
> >> > > > > > > > > >> > > > brokers
> >> > > > > > > > > >> > > > >> go
> >> > > > > > > > > >> > > > >> >> > down,
> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
> >> > > servers
> >> > > > > are
> >> > > > > > > > > >> restarted as
> >> > > > > > > > > >> > > > >> tokens
> >> > > > > > > > > >> > > > >> >> > are not
> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> happens,
> >> > then
> >> > > > > there
> >> > > > > > > are
> >> > > > > > > > > more
> >> > > > > > > > > >> > > > important
> >> > > > > > > > > >> > > > >> >> > things to
> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
> better
> >> > to
> >> > > > > > > > > >> re-authenticate.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2)
> Do not
> >> > > > > > distribute
> >> > > > > > > > DT
> >> > > > > > > > > to
> >> > > > > > > > > >> > > other
> >> > > > > > > > > >> > > > >> brokers
> >> > > > > > > > > >> > > > >> >> > at
> >> > > > > > > > > >> > > > >> >> > >> >> all.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
> >> > > authenticates
> >> > > > > > with a
> >> > > > > > > > > >> broker.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a
> client is
> >> > > > > > > > authenticated,
> >> > > > > > > > > >> it
> >> > > > > > > > > >> > > will
> >> > > > > > > > > >> > > > >> make a
> >> > > > > > > > > >> > > > >> >> > broker
> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a
> delegation
> >> > > > token.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
> >> > > generates
> >> > > > DT
> >> > > > > > of
> >> > > > > > > > > form,
> >> > > > > > > > > >> > > [hmac +
> >> > > > > > > > > >> > > > >> (owner,
> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime,
> id,
> >> > hmac,
> >> > > > > > > > > >> expirationTime)]
> >> > > > > > > > > >> > > and
> >> > > > > > > > > >> > > > >> passes
> >> > > > > > > > > >> > > > >> >> > back
> >> > > > > > > > > >> > > > >> >> > >> >> this
> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is
> generated
> >> > via
> >> > > > > > > > > >> {HMAC-SHA256(owner,
> >> > > > > > > > > >> > > > >> renewer,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime)
> >> > using
> >> > > > > > > > SecretKey}.
> >> > > > > > > > > >> Note
> >> > > > > > > > > >> > > that
> >> > > > > > > > > >> > > > >> all
> >> > > > > > > > > >> > > > >> >> > brokers
> >> > > > > > > > > >> > > > >> >> > >> >> have
> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then
> goes
> >> > to
> >> > > > any
> >> > > > > > > > broker
> >> > > > > > > > > >> and to
> >> > > > > > > > > >> > > > >> >> > authenticate
> >> > > > > > > > > >> > > > >> >> > >> >> sends
> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker
> recalculates
> >> > > hmac
> >> > > > > > using
> >> > > > > > > > > >> (owner,
> >> > > > > > > > > >> > > > >> renewer,
> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> expirationTime) info
> >> > > > from
> >> > > > > DT
> >> > > > > > > and
> >> > > > > > > > > its
> >> > > > > > > > > >> > > > >> SecretKey. If
> >> > > > > > > > > >> > > > >> >> > it
> >> > > > > > > > > >> > > > >> >> > >> >> matches
> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is
> >> > > > > authenticated.
> >> > > > > > > > Yes,
> >> > > > > > > > > >> it will
> >> > > > > > > > > >> > > > do
> >> > > > > > > > > >> > > > >> other
> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp
> expiry and
> >> > > > such.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret
> key
> >> > will
> >> > > > be
> >> > > > > > > > > generated
> >> > > > > > > > > >> by
> >> > > > > > > > > >> > > > >> controller
> >> > > > > > > > > >> > > > >> >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> passed
> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> and
> >> > fixes
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to
> delete a
> >> > DT?
> >> > > > > Yes,
> >> > > > > > > that
> >> > > > > > > > > is
> >> > > > > > > > > >> a
> >> > > > > > > > > >> > > > downside
> >> > > > > > > > > >> > > > >> >> > here.
> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with
> >> > > brokers
> >> > > > > > > > > maintaining
> >> > > > > > > > > >> a
> >> > > > > > > > > >> > > > >> blacklist of
> >> > > > > > > > > >> > > > >> >> > DTs,
> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed
> after
> >> > > > > expiry.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In
> catastrophic
> >> > > > > failures
> >> > > > > > > > where
> >> > > > > > > > > >> all
> >> > > > > > > > > >> > > > brokers
> >> > > > > > > > > >> > > > >> go
> >> > > > > > > > > >> > > > >> >> > down,
> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
> >> > > servers
> >> > > > > are
> >> > > > > > > > > >> restarted as
> >> > > > > > > > > >> > > > >> tokens
> >> > > > > > > > > >> > > > >> >> > are not
> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this
> happens,
> >> > then
> >> > > > > there
> >> > > > > > > are
> >> > > > > > > > > more
> >> > > > > > > > > >> > > > important
> >> > > > > > > > > >> > > > >> >> > things to
> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is
> better
> >> > to
> >> > > > > > > > > >> re-authenticate.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26,
> 2016 at
> >> > > > 1:58
> >> > > > > > PM,
> >> > > > > > > > > Parth
> >> > > > > > > > > >> > > > >> Brahmbhatt <
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > pbrahmbhatt@hortonworks.com>
> >> > > > > > > > wrote:
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed
> KIP-48 so
> >> > > we
> >> > > > > can
> >> > > > > > > > offer
> >> > > > > > > > > >> hadoop
> >> > > > > > > > > >> > > > like
> >> > > > > > > > > >> > > > >> >> > delegation
> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can
> review
> >> > > the
> >> > > > > > design
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends
> on
> >> > > KIP-43
> >> > > > > and
> >> > > > > > > we
> >> > > > > > > > > >> have also
> >> > > > > > > > > >> > > > >> >> > discussed an
> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design
> here<
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> > > > > > > > > >> > > > >> >> > >> >>
> >> > > > > > > > > >> > > > >> >> >
> >> > > > > > > > > >> > > > >>
> >> > > > > > > > > >> > > >
> >> > > > > > > > > >> > >
> >> > > > > > > > > >>
> >> > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > Regards,
> >> > > > > >
> >> > > > > > Rajini
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Regards,
> >> > > >
> >> > > > Rajini
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Liquan Pei
> >> Software Engineer, Confluent Inc
>
Re: [DISCUSS] KIP-48 Support for delegation tokens as an
authentication mechanism
Posted by Gwen Shapira <gw...@confluent.io>.
From what I can see, remaining questions are:
1. Who / how are tokens renewed? By original requester only? or using
Kerberos auth only?
2. Are tokens stored on each broker or in ZK?
3. How are tokens invalidated / expired?
4. Which encryption algorithm is used?
5. What is the impersonation proposal (it wasn't in the KIP but was
discussed in this thread)?
6. Do we need new ACLs, if so - for what actions?
Gwen
On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> Jun & Ismael,
> Unfortunately I couldn't attend the KIP meeting
> when delegation tokens discussed. Appreciate if
> you can update the thread if you have any
> further questions.
> Thanks,
> Harsha
>
> On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
>> It seems that the links to images in the KIP are broken.
>>
>> Liquan
>>
>> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
>> brahmbhatt.parth@gmail.com> wrote:
>>
>> > 110. What does getDelegationTokenAs mean?
>> > In the current proposal we only allow a user to get delegation token for
>> > the identity that it authenticated as using another mechanism, i.e. A user
>> > that authenticate using a keytab for principal user1@EXAMPLE.COM will get
>> > delegation tokens for that user only. In future I think we will have to
>> > extend support such that we allow some set of users (
>> > kafka-rest-user@EXAMPLE.COM, storm-nimbus@EXAMPLE.COM) to acquire
>> > delegation tokens on behalf of other users whose identity they have
>> > verified independently. Kafka brokers will have ACLs to control which
>> > users are allowed to impersonate other users and get tokens on behalf of
>> > them. Overall Impersonation is a whole different problem in my opinion and
>> > I think we can tackle it in separate KIP.
>> >
>> > 111. What's the typical rate of getting and renewing delegation tokens?
>> > Typically this should be very very low, 1 request per minute is a
>> > relatively high estimate. However it depends on the token expiration. I am
>> > less worried about the extra load it puts on controller vs the added
>> > complexity and the value it offers.
>> >
>> > Thanks
>> > Parth
>> >
>> >
>> >
>> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <is...@juma.me.uk> wrote:
>> >
>> > > Thanks Rajini. It would probably require a separate KIP as it will
>> > > introduce user visible changes. We could also update KIP-48 to have this
>> > > information, but it seems cleaner to do it separately. We can discuss
>> > that
>> > > in the KIP call today.
>> > >
>> > > Ismael
>> > >
>> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
>> > > rajinisivaram@googlemail.com> wrote:
>> > >
>> > > > Ismael,
>> > > >
>> > > > I have created a JIRA (
>> > https://issues.apache.org/jira/browse/KAFKA-3751)
>> > > > for adding SCRAM as a SASL mechanism. Would that need another KIP? If
>> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets
>> > > reviewed
>> > > > when the PR is ready?
>> > > >
>> > > > Thank you,
>> > > >
>> > > > Rajini
>> > > >
>> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <is...@juma.me.uk>
>> > wrote:
>> > > >
>> > > > > Thanks Rajini, SCRAM seems like a good candidate.
>> > > > >
>> > > > > Gwen had independently mentioned this as a SASL mechanism that might
>> > be
>> > > > > useful for Kafka and I have been meaning to check it in more detail.
>> > > Good
>> > > > > to know that you are willing to contribute an implementation. Maybe
>> > we
>> > > > > should file a separate JIRA for this?
>> > > > >
>> > > > > Ismael
>> > > > >
>> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
>> > > > > rajinisivaram@googlemail.com> wrote:
>> > > > >
>> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is a
>> > > better
>> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in SCRAM
>> > > > > > SaslServer or SaslClient, but I will be happy to add support in
>> > Kafka
>> > > > > since
>> > > > > > it would be a useful mechanism to support anyway.
>> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol for
>> > > > > > SCRAM-SHA-256.
>> > > > > >
>> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <ju...@confluent.io> wrote:
>> > > > > >
>> > > > > > > Parth,
>> > > > > > >
>> > > > > > > Thanks for the explanation. A couple of more questions.
>> > > > > > >
>> > > > > > > 110. What does getDelegationTokenAs mean?
>> > > > > > >
>> > > > > > > 111. What's the typical rate of getting and renewing delegation
>> > > > tokens?
>> > > > > > > That may have an impact on whether they should be directed to the
>> > > > > > > controller.
>> > > > > > >
>> > > > > > > Jun
>> > > > > > >
>> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
>> > > > > > > brahmbhatt.parth@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hi Jun,
>> > > > > > > >
>> > > > > > > > Thanks for reviewing.
>> > > > > > > >
>> > > > > > > > * We could add a Cluster action to add acls on who can request
>> > > > > > delegation
>> > > > > > > > tokens. I don't see the use case for that yet but down the line
>> > > > when
>> > > > > we
>> > > > > > > > start supporting getDelegationTokenAs it will be necessary.
>> > > > > > > > * Yes we recommend tokens to be only used/distributed over
>> > secure
>> > > > > > > channels.
>> > > > > > > > * Depending on what design we end up choosing Invalidation will
>> > > be
>> > > > > > > > responsibility of every broker or controller.
>> > > > > > > > * I am not sure if I documented somewhere that invalidation
>> > will
>> > > > > > directly
>> > > > > > > > go through zookeeper but that is not the intent. Invalidation
>> > > will
>> > > > > > either
>> > > > > > > > be request based or due to expiration. No direct zookeeper
>> > > > > interaction
>> > > > > > > from
>> > > > > > > > any client.
>> > > > > > > > * "Broker also stores the DelegationToken without the hmac in
>> > the
>> > > > > > > > zookeeper." : Sorry about the confusion. The sole purpose of
>> > > > > zookeeper
>> > > > > > in
>> > > > > > > > this design is as distribution channel for tokens between all
>> > > > brokers
>> > > > > > > and a
>> > > > > > > > layer that ensures only tokens that were generated by making a
>> > > > > request
>> > > > > > > to a
>> > > > > > > > broker will be accepted (more on this in second paragraph). The
>> > > > token
>> > > > > > > > consists of few elements (owner, renewer, uuid , expiration,
>> > > hmac)
>> > > > ,
>> > > > > > one
>> > > > > > > of
>> > > > > > > > which is the finally generated hmac but hmac it self is
>> > derivable
>> > > > if
>> > > > > > you
>> > > > > > > > have all the other elements of the token + secret key to
>> > generate
>> > > > > hmac.
>> > > > > > > > Given zookeeper does not provide SSL support we do not want the
>> > > > > entire
>> > > > > > > > token to be wire transferred to zookeeper as that will be an
>> > > > insecure
>> > > > > > > wire
>> > > > > > > > transfer. Instead we only store all the other elements of a
>> > > > > delegation
>> > > > > > > > tokens. Brokers can read these elements and because they also
>> > > have
>> > > > > > access
>> > > > > > > > to secret key they will be able to generate hmac on their end.
>> > > > > > > >
>> > > > > > > > One of the alternative proposed is to avoid zookeeper
>> > > altogether. A
>> > > > > > > Client
>> > > > > > > > will call broker with required information (owner, renwer,
>> > > > > expiration)
>> > > > > > > and
>> > > > > > > > get back (signed hmac, uuid). Broker won't store this in
>> > > zookeeper.
>> > > > > > From
>> > > > > > > > this point a client can contact any broker with all the
>> > > delegation
>> > > > > > token
>> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker will
>> > > > > regenerate
>> > > > > > > the
>> > > > > > > > hmac and as long as it matches with hmac presented by client ,
>> > > > broker
>> > > > > > > will
>> > > > > > > > allow the request to authenticate. Only problem with this
>> > > approach
>> > > > > is
>> > > > > > if
>> > > > > > > > the secret key is compromised any client can now generate
>> > random
>> > > > > tokens
>> > > > > > > and
>> > > > > > > > they will still be able to authenticate as any user they like.
>> > > with
>> > > > > > > > zookeeper we guarantee that only tokens acquired via a broker
>> > > > (using
>> > > > > > some
>> > > > > > > > auth scheme other than delegation token) will be accepted. We
>> > > need
>> > > > to
>> > > > > > > > discuss which proposal makes more sense and we can go over it
>> > in
>> > > > > > > tomorrow's
>> > > > > > > > meeting.
>> > > > > > > >
>> > > > > > > > Also, can you forward the invite to me?
>> > > > > > > >
>> > > > > > > > Thanks
>> > > > > > > > Parth
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <ju...@confluent.io>
>> > > > wrote:
>> > > > > > > >
>> > > > > > > > > Thanks for the KIP. A few comments.
>> > > > > > > > >
>> > > > > > > > > 100. This potentially can be useful for Kafka Connect and
>> > Kafka
>> > > > > rest
>> > > > > > > > proxy
>> > > > > > > > > where a worker agent will need to run a task on behalf of a
>> > > > client.
>> > > > > > We
>> > > > > > > > will
>> > > > > > > > > likely need to change how those services use Kafka clients
>> > > > > > > > > (producer/consumer). Instead of a shared client per worker,
>> > we
>> > > > will
>> > > > > > > need
>> > > > > > > > a
>> > > > > > > > > client per user task since the authentication happens at the
>> > > > > > connection
>> > > > > > > > > level. For Kafka Connect, the renewer will be the workers.
>> > So,
>> > > we
>> > > > > > > > probably
>> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy, the
>> > > > renewer
>> > > > > > can
>> > > > > > > > > probably just be the creator of the token.
>> > > > > > > > >
>> > > > > > > > > 101. Do we need new acl on who can request delegation tokens?
>> > > > > > > > >
>> > > > > > > > > 102. Do we recommend people to send delegation tokens in an
>> > > > > encrypted
>> > > > > > > > > channel?
>> > > > > > > > >
>> > > > > > > > > 103. Who is responsible for expiring tokens, every broker?
>> > > > > > > > >
>> > > > > > > > > 104. For invalidating tokens, would it be better to do it in
>> > a
>> > > > > > request
>> > > > > > > > > instead of going to ZK directly?
>> > > > > > > > >
>> > > > > > > > > 105. The terminology of client in the wiki sometimes refers
>> > to
>> > > > the
>> > > > > > end
>> > > > > > > > > client and some other times refers to the client using the
>> > > > > delegation
>> > > > > > > > > tokens. It would be useful to distinguish between the two.
>> > > > > > > > >
>> > > > > > > > > 106. Could you explain the sentence "Broker also stores the
>> > > > > > > > DelegationToken
>> > > > > > > > > without the hmac in the zookeeper." a bit more? I thought the
>> > > > > > > delegation
>> > > > > > > > > token is the hmac.
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > >
>> > > > > > > > > Jun
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <ju...@confluent.io>
>> > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi, Harsha,
>> > > > > > > > > >
>> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss this in
>> > > the
>> > > > > > > meeting
>> > > > > > > > > > tomorrow.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <ka...@harsha.io>
>> > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > >> Hi All,
>> > > > > > > > > >> Can we have a KIP meeting around this. The KIP
>> > is
>> > > > up
>> > > > > > for
>> > > > > > > > > >> sometime and if there are any questions lets
>> > > > quickly
>> > > > > > hash
>> > > > > > > > out
>> > > > > > > > > >> details.
>> > > > > > > > > >>
>> > > > > > > > > >> Thanks,
>> > > > > > > > > >> Harsha
>> > > > > > > > > >>
>> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt wrote:
>> > > > > > > > > >> > That is what the hadoop echo system uses so no good
>> > reason
>> > > > > > really.
>> > > > > > > > We
>> > > > > > > > > >> > could
>> > > > > > > > > >> > change it to whatever is the newest recommended standard
>> > > is.
>> > > > > > > > > >> >
>> > > > > > > > > >> > Thanks
>> > > > > > > > > >> > Parth
>> > > > > > > > > >> >
>> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
>> > > > > ismael@juma.me.uk
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > > >> >
>> > > > > > > > > >> > > Hi Parth,
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing this and
>> > > may
>> > > > > have
>> > > > > > > > > >> additional
>> > > > > > > > > >> > > questions later. The immediate question that came to
>> > > mind
>> > > > is
>> > > > > > our
>> > > > > > > > > >> choice of
>> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE in
>> > the
>> > > > IANA
>> > > > > > > > > Registry
>> > > > > > > > > >> of
>> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has been
>> > > moved
>> > > > > to
>> > > > > > > > > Historic
>> > > > > > > > > >> > > status:
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
>> > > > > > > > > >> > >
>> > > > > > > > >
>> > > > > >
>> > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > What is the reasoning behind that choice?
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > Thanks,
>> > > > > > > > > >> > > Ismael
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira <
>> > > > > > > gwen@confluent.io
>> > > > > > > > >
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> > >
>> > > > > > > > > >> > > > Also comments inline :)
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > > * I want to emphasize that even though delegation
>> > > > tokens
>> > > > > > > are a
>> > > > > > > > > >> Hadoop
>> > > > > > > > > >> > > > > innovation, I feel very strongly about not adding
>> > > > > > dependency
>> > > > > > > > on
>> > > > > > > > > >> Hadoop
>> > > > > > > > > >> > > > > when implementing delegation tokens for Kafka. The
>> > > KIP
>> > > > > > > doesn't
>> > > > > > > > > >> imply
>> > > > > > > > > >> > > > > such dependency, but if you can clarify...
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *No hadoop dependency.*
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will read
>> > the
>> > > > KIP
>> > > > > > and
>> > > > > > > > > panic
>> > > > > > > > > >> > > > three weeks before the next release...
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > > * Can we get delegation token at any time after
>> > > > > > > > authenticating?
>> > > > > > > > > >> only
>> > > > > > > > > >> > > > > immediately after?
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *As long as you are authenticated you can get
>> > > > delegation
>> > > > > > > > tokens.
>> > > > > > > > > >> We
>> > > > > > > > > >> > > need
>> > > > > > > > > >> > > > to
>> > > > > > > > > >> > > > > discuss if a client authenticated using delegation
>> > > > > token,
>> > > > > > > can
>> > > > > > > > > also
>> > > > > > > > > >> > > > acquire
>> > > > > > > > > >> > > > > delegation token again or not. Also there is the
>> > > > > question
>> > > > > > of
>> > > > > > > > do
>> > > > > > > > > we
>> > > > > > > > > >> > > allow
>> > > > > > > > > >> > > > > anyone to acquire delegation token or we want
>> > > specific
>> > > > > > ACLs
>> > > > > > > (I
>> > > > > > > > > >> think
>> > > > > > > > > >> > > its
>> > > > > > > > > >> > > > an
>> > > > > > > > > >> > > > > overkill.)*
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > I agree that ACLs is an overkill.
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > I think we are debating two options: Either require
>> > > > > Kerberos
>> > > > > > > > auth
>> > > > > > > > > >> for
>> > > > > > > > > >> > > > renewal or require non-owners to renew.
>> > > > > > > > > >> > > > I *think* the latter is simpler (it basically
>> > require
>> > > a
>> > > > > "job
>> > > > > > > > > master"
>> > > > > > > > > >> > > > to take responsibility for the renewal, it will have
>> > > its
>> > > > > own
>> > > > > > > > > >> identity
>> > > > > > > > > >> > > > anyway and I think this is the correct design
>> > pattern
>> > > > > > anyway.
>> > > > > > > > For
>> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate renewals?),
>> > but
>> > > > it
>> > > > > is
>> > > > > > > > hard
>> > > > > > > > > to
>> > > > > > > > > >> > > > debate simplicity without looking at the code
>> > changes
>> > > > > > > required.
>> > > > > > > > If
>> > > > > > > > > >> you
>> > > > > > > > > >> > > > have a draft of how the "require Kerberos" will look
>> > > in
>> > > > > > Kafka
>> > > > > > > > > code,
>> > > > > > > > > >> > > > I'll be happy to take a look.
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > > * My understanding is that tokens will propagate
>> > via
>> > > > ZK
>> > > > > > but
>> > > > > > > > > >> without
>> > > > > > > > > >> > > > > additional changes to UpdateMetadata protocol,
>> > > > correct?
>> > > > > > > > Clients
>> > > > > > > > > >> > > > > currently don't retry on SASL auth failure (IIRC),
>> > > but
>> > > > > > since
>> > > > > > > > the
>> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we will
>> > > need
>> > > > to
>> > > > > > > > retry a
>> > > > > > > > > >> bit
>> > > > > > > > > >> > > > > to avoid clients failing auth due to timing
>> > issues.
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *I am considering 2 alternatives right now. The
>> > > > current
>> > > > > > > > > documented
>> > > > > > > > > >> > > > approach
>> > > > > > > > > >> > > > > is zookeeper based and it does not require any
>> > > changes
>> > > > > to
>> > > > > > > > > >> > > UpdateMetadata
>> > > > > > > > > >> > > > > protocol. An alternative approach can remove
>> > > zookeeper
>> > > > > > > > > dependency
>> > > > > > > > > >> as
>> > > > > > > > > >> > > well
>> > > > > > > > > >> > > > > but we can discuss that in KIP discussion call.*
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping Jun to
>> > > > > > arrange a
>> > > > > > > > > call?
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just the
>> > > > > > controller
>> > > > > > > > > issue
>> > > > > > > > > >> the
>> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared
>> > secret.
>> > > > Not
>> > > > > > > sure
>> > > > > > > > if
>> > > > > > > > > >> we
>> > > > > > > > > >> > > > > want to continue the discussion here or on the
>> > > wiki. I
>> > > > > > think
>> > > > > > > > > that
>> > > > > > > > > >> we
>> > > > > > > > > >> > > > > can decouple the problem of "token distribution"
>> > > from
>> > > > > > > "shared
>> > > > > > > > > >> secret
>> > > > > > > > > >> > > > > distribution" and use the controller as the only
>> > > token
>> > > > > > > > generator
>> > > > > > > > > >> to
>> > > > > > > > > >> > > > > solve the second issue, while still using ZK async
>> > > to
>> > > > > > > > distribute
>> > > > > > > > > >> > > > > tokens.
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *As mentioned in the previous Email I am fine with
>> > > > that
>> > > > > > > > approach
>> > > > > > > > > >> as
>> > > > > > > > > >> > > long
>> > > > > > > > > >> > > > as
>> > > > > > > > > >> > > > > we agree that the extra complexity of
>> > > adding/updating
>> > > > > APIS
>> > > > > > > > adds
>> > > > > > > > > >> enough
>> > > > > > > > > >> > > > > value. The advantage with the controller approach
>> > is
>> > > > > > secret
>> > > > > > > > > >> rotation
>> > > > > > > > > >> > > can
>> > > > > > > > > >> > > > be
>> > > > > > > > > >> > > > > automated,frequent and would not require
>> > > deployment. *
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Can you detail the extra complexity (or point me to
>> > > the
>> > > > > > email
>> > > > > > > I
>> > > > > > > > > >> > > > missed?) - which Apis are required?
>> > > > > > > > > >> > > > As far as I can tell, clients can already find the
>> > > > > > controller
>> > > > > > > > from
>> > > > > > > > > >> > > > metadata. I'm a bit more concerned about controller
>> > > > load.
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos auth
>> > for
>> > > > > > > renwal, I
>> > > > > > > > > >> think
>> > > > > > > > > >> > > > > it mixes the transport layer the the request
>> > content
>> > > > in
>> > > > > a
>> > > > > > > > pretty
>> > > > > > > > > >> ugly
>> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is
>> > > better.
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *I feel this is a necessary evil. While this will
>> > > make
>> > > > > the
>> > > > > > > > kafka
>> > > > > > > > > >> code
>> > > > > > > > > >> > > > > pretty straight forward , forcing renewer to
>> > > > non-owner
>> > > > > > > pushes
>> > > > > > > > > >> the code
>> > > > > > > > > >> > > > > ugliness to client and makes it even harder to
>> > > > > > integrate. *
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > As mentioned before, I don't think the "renewal by
>> > > > other"
>> > > > > > > > approach
>> > > > > > > > > >> is
>> > > > > > > > > >> > > > that ugly for the clients we expect to use
>> > delegation
>> > > > > tokens
>> > > > > > > > since
>> > > > > > > > > >> > > > they will have an app-master of some sort who
>> > > requested
>> > > > > the
>> > > > > > > > token
>> > > > > > > > > to
>> > > > > > > > > >> > > > begin with.
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > The response for my question on how multiple
>> > > > identities
>> > > > > > will
>> > > > > > > > be
>> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we don't
>> > > > > > > > authenticate
>> > > > > > > > > >> each
>> > > > > > > > > >> > > > > request, we authenticate connections.
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > *We authenticate connections, and only when they
>> > are
>> > > > > being
>> > > > > > > > > >> established.
>> > > > > > > > > >> > > > Let
>> > > > > > > > > >> > > > > me try to phrase this as a question, in absence of
>> > > > > > > delegation
>> > > > > > > > > >> tokens if
>> > > > > > > > > >> > > > we
>> > > > > > > > > >> > > > > had to support the use case using user TGT's how
>> > > would
>> > > > > we
>> > > > > > do
>> > > > > > > > it?
>> > > > > > > > > >> My
>> > > > > > > > > >> > > point
>> > > > > > > > > >> > > > > was it would be no different with delegation
>> > tokens.
>> > > > The
>> > > > > > use
>> > > > > > > > > case
>> > > > > > > > > >> you
>> > > > > > > > > >> > > are
>> > > > > > > > > >> > > > > describing seems more like impersonation.*
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Yeah, I thought that one of the things that
>> > delegation
>> > > > > > tokens
>> > > > > > > > > >> handled.
>> > > > > > > > > >> > > > Maybe I got it wrong :)
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Thanks for the detailed answers.
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > Gwen
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > > > > Thanks
>> > > > > > > > > >> > > > > Parth
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen Shapira <
>> > > > > > > > > gwen@confluent.io
>> > > > > > > > > >> >
>> > > > > > > > > >> > > > wrote:
>> > > > > > > > > >> > > > >
>> > > > > > > > > >> > > > >> Hi Parth and Harsha,
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> Few more comments:
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * The API / RequestResponse section doesn't seem
>> > to
>> > > > > have
>> > > > > > > good
>> > > > > > > > > >> > > > >> description of the changes to the Kafka Protocol.
>> > > > > Sounds
>> > > > > > > like
>> > > > > > > > > >> you are
>> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
>> > > > > > RenewTokenRequest
>> > > > > > > > (and
>> > > > > > > > > >> > > > >> matching responses), without detailing the
>> > contents
>> > > > of
>> > > > > > the
>> > > > > > > > > >> requests
>> > > > > > > > > >> > > > >> and responses? Or rather, you show the class
>> > > > interface,
>> > > > > > but
>> > > > > > > > not
>> > > > > > > > > >> the
>> > > > > > > > > >> > > > >> underlying protocol. This could be seen as an
>> > > > > > > implementation
>> > > > > > > > > >> detail,
>> > > > > > > > > >> > > > >> but since the binary protocol is what we provide
>> > to
>> > > > > > > non-Java
>> > > > > > > > > >> clients,
>> > > > > > > > > >> > > > >> we need to show the changes there.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * getDelegationToken sounds like
>> > > > > > > > delegationTokenRequestHandler?
>> > > > > > > > > >> Is it
>> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client? Its
>> > > > > unclear...
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * I want to emphasize that even though delegation
>> > > > > tokens
>> > > > > > > are
>> > > > > > > > a
>> > > > > > > > > >> Hadoop
>> > > > > > > > > >> > > > >> innovation, I feel very strongly about not adding
>> > > > > > > dependency
>> > > > > > > > on
>> > > > > > > > > >> Hadoop
>> > > > > > > > > >> > > > >> when implementing delegation tokens for Kafka.
>> > The
>> > > > KIP
>> > > > > > > > doesn't
>> > > > > > > > > >> imply
>> > > > > > > > > >> > > > >> such dependency, but if you can clarify...
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * Can we get delegation token at any time after
>> > > > > > > > authenticating?
>> > > > > > > > > >> only
>> > > > > > > > > >> > > > >> immediately after?
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * My understanding is that tokens will propagate
>> > > via
>> > > > ZK
>> > > > > > but
>> > > > > > > > > >> without
>> > > > > > > > > >> > > > >> additional changes to UpdateMetadata protocol,
>> > > > correct?
>> > > > > > > > Clients
>> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure
>> > (IIRC),
>> > > > but
>> > > > > > > since
>> > > > > > > > > the
>> > > > > > > > > >> > > > >> tokens propagate between brokers asynch, we will
>> > > need
>> > > > > to
>> > > > > > > > retry
>> > > > > > > > > a
>> > > > > > > > > >> bit
>> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing
>> > issues.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * Strongly agreeing on clients not touching ZK
>> > > > directly
>> > > > > > :)
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having just the
>> > > > > > controller
>> > > > > > > > > >> issue the
>> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared
>> > > secret.
>> > > > > Not
>> > > > > > > sure
>> > > > > > > > > if
>> > > > > > > > > >> we
>> > > > > > > > > >> > > > >> want to continue the discussion here or on the
>> > > wiki.
>> > > > I
>> > > > > > > think
>> > > > > > > > > >> that we
>> > > > > > > > > >> > > > >> can decouple the problem of "token distribution"
>> > > from
>> > > > > > > "shared
>> > > > > > > > > >> secret
>> > > > > > > > > >> > > > >> distribution" and use the controller as the only
>> > > > token
>> > > > > > > > > generator
>> > > > > > > > > >> to
>> > > > > > > > > >> > > > >> solve the second issue, while still using ZK
>> > async
>> > > to
>> > > > > > > > > distribute
>> > > > > > > > > >> > > > >> tokens.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite lifetime
>> > of
>> > > > > > tokens
>> > > > > > > > (and
>> > > > > > > > > >> hoped
>> > > > > > > > > >> > > > >> to hear from others in the community) - but
>> > having
>> > > > the
>> > > > > > > option
>> > > > > > > > > and
>> > > > > > > > > >> > > > >> documenting it as less secure, allows users to
>> > > > > configure
>> > > > > > > > their
>> > > > > > > > > >> system
>> > > > > > > > > >> > > > >> as the wish.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * While I like the idea of forcing kerberos auth
>> > > for
>> > > > > > > renwal,
>> > > > > > > > I
>> > > > > > > > > >> think
>> > > > > > > > > >> > > > >> it mixes the transport layer the the request
>> > > content
>> > > > > in a
>> > > > > > > > > pretty
>> > > > > > > > > >> ugly
>> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner is
>> > > better.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> Things I'd still like to see:
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * More detailed explanation on what we plan to do
>> > > for
>> > > > > the
>> > > > > > > > java
>> > > > > > > > > >> clients
>> > > > > > > > > >> > > > >> specifically - new configuration? new APIs?
>> > > > > > > > > >> > > > >> The response for my question on how multiple
>> > > > identities
>> > > > > > > will
>> > > > > > > > be
>> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we
>> > don't
>> > > > > > > > authenticate
>> > > > > > > > > >> each
>> > > > > > > > > >> > > > >> request, we authenticate connections.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only used
>> > in
>> > > > the
>> > > > > > > Hadoop
>> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
>> > alternatives
>> > > in
>> > > > > > other
>> > > > > > > > > >> ecosystems
>> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether there
>> > are
>> > > > some
>> > > > > > > > > >> advantages
>> > > > > > > > > >> > > > >> there.
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> Gwen
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
>> > > > > kafka@harsha.io
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > > >> > > > >> > Hi Gwen,
>> > > > > > > > > >> > > > >> > Can you look at Parth's last reply.
>> > > Does
>> > > > > it
>> > > > > > > > answer
>> > > > > > > > > >> your
>> > > > > > > > > >> > > > >> > concerns.
>> > > > > > > > > >> > > > >> >
>> > > > > > > > > >> > > > >> > Thanks,
>> > > > > > > > > >> > > > >> > Harsha
>> > > > > > > > > >> > > > >> >
>> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
>> > > brahmbhatt
>> > > > > > wrote:
>> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki already
>> > has
>> > > > > > details
>> > > > > > > on
>> > > > > > > > > >> token
>> > > > > > > > > >> > > > >> >> expiration
>> > > > > > > > > >> > > > >> >> under token acquisition process
>> > > > > > > > > >> > > > >> >> <
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > >
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
>> > > > > > > > > >> > > > >> >.
>> > > > > > > > > >> > > > >> >> Current proposal is that tokens will expire
>> > > based
>> > > > > on a
>> > > > > > > > > server
>> > > > > > > > > >> side
>> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless
>> > renewed.
>> > > > > > Renewal
>> > > > > > > > is
>> > > > > > > > > >> only
>> > > > > > > > > >> > > > allowed
>> > > > > > > > > >> > > > >> >> until the max life time of token.
>> > Alternatively
>> > > we
>> > > > > > could
>> > > > > > > > > also
>> > > > > > > > > >> make
>> > > > > > > > > >> > > > that
>> > > > > > > > > >> > > > >> >> an
>> > > > > > > > > >> > > > >> >> optional param and the server side default can
>> > > > serve
>> > > > > > as
>> > > > > > > > the
>> > > > > > > > > >> upper
>> > > > > > > > > >> > > > bound.
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >> To your second point it will be done exactly
>> > the
>> > > > > same
>> > > > > > > way
>> > > > > > > > we
>> > > > > > > > > >> would
>> > > > > > > > > >> > > > >> >> support
>> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client will have
>> > > to
>> > > > > put
>> > > > > > > the
>> > > > > > > > > >> tokens it
>> > > > > > > > > >> > > > >> wants
>> > > > > > > > > >> > > > >> >> to use in the subject instance and call
>> > > > > > produce/consume
>> > > > > > > > > inside
>> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to keep
>> > > track
>> > > > of
>> > > > > > its
>> > > > > > > > own
>> > > > > > > > > >> > > > subject. I
>> > > > > > > > > >> > > > >> >> will have to look at the code to see if we
>> > > support
>> > > > > > this
>> > > > > > > > > >> feature
>> > > > > > > > > >> > > right
>> > > > > > > > > >> > > > >> now
>> > > > > > > > > >> > > > >> >> but my understanding is delegation token
>> > > shouldn't
>> > > > > > need
>> > > > > > > > any
>> > > > > > > > > >> special
>> > > > > > > > > >> > > > >> >> treatment as its just another type of
>> > Credential
>> > > > in
>> > > > > > the
>> > > > > > > > > >> subject.
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >> I would also like to know what is your opinion
>> > > > about
>> > > > > > > > > infinite
>> > > > > > > > > >> > > renewal
>> > > > > > > > > >> > > > >> (my
>> > > > > > > > > >> > > > >> >> recommendation is to not support this), tokens
>> > > > > > renewing
>> > > > > > > > them
>> > > > > > > > > >> > > self(my
>> > > > > > > > > >> > > > >> >> recommendation is to not support this) and
>> > most
>> > > > > > > > importantly
>> > > > > > > > > >> your
>> > > > > > > > > >> > > > choice
>> > > > > > > > > >> > > > >> >> between the alternatives listed on this thread
>> > > > > > > > > >> > > > >> >> <
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > >
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
>> > > > > > > > > >> > > > >> >
>> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2 minus
>> > > > > > > controller
>> > > > > > > > > >> > > > distributing
>> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >> Thanks
>> > > > > > > > > >> > > > >> >> Parth
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen Shapira <
>> > > > > > > > > >> gwen@confluent.io>
>> > > > > > > > > >> > > > wrote:
>> > > > > > > > > >> > > > >> >>
>> > > > > > > > > >> > > > >> >> > Harsha,
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I didn't
>> > see
>> > > > > your
>> > > > > > > > design
>> > > > > > > > > >> yet,
>> > > > > > > > > >> > > > but in
>> > > > > > > > > >> > > > >> >> > our proxy, we have a set of producers, which
>> > > > will
>> > > > > > > serve
>> > > > > > > > > >> multiple
>> > > > > > > > > >> > > > users
>> > > > > > > > > >> > > > >> >> > going through the proxy. Since these users
>> > > will
>> > > > > have
>> > > > > > > > > >> different
>> > > > > > > > > >> > > > >> >> > privileges, they'll need to authenticate
>> > > > > separately,
>> > > > > > > and
>> > > > > > > > > >> can't
>> > > > > > > > > >> > > > share a
>> > > > > > > > > >> > > > >> >> > token.
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >> >> > Am I missing anything?
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >> >> > Gwen
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha <
>> > > > > > > kafka@harsha.io
>> > > > > > > > >
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> > > > >> >> > > Gwen,
>> > > > > > > > > >> > > > >> >> > > On your second point. Can you
>> > > > > describe
>> > > > > > a
>> > > > > > > > > >> usecase
>> > > > > > > > > >> > > where
>> > > > > > > > > >> > > > >> >> > > mutliple clients ended up
>> > > sharing a
>> > > > > > > > producer
>> > > > > > > > > >> and
>> > > > > > > > > >> > > even
>> > > > > > > > > >> > > > if
>> > > > > > > > > >> > > > >> they
>> > > > > > > > > >> > > > >> >> > > do why can't they not use
>> > single
>> > > > > token
>> > > > > > > that
>> > > > > > > > > >> producer
>> > > > > > > > > >> > > > >> >> > > captures. Why would we need
>> > > > multiple
>> > > > > > > > clients
>> > > > > > > > > >> with
>> > > > > > > > > >> > > > >> different
>> > > > > > > > > >> > > > >> >> > > tokens sharing a single
>> > instance
>> > > of
>> > > > > > > > producer.
>> > > > > > > > > >> Also
>> > > > > > > > > >> > > in
>> > > > > > > > > >> > > > >> this
>> > > > > > > > > >> > > > >> >> > > case other clients have access
>> > > all
>> > > > > the
>> > > > > > > > tokens
>> > > > > > > > > >> no?
>> > > > > > > > > >> > > > >> >> > >
>> > > > > > > > > >> > > > >> >> > > Thanks,
>> > > > > > > > > >> > > > >> >> > > Harsha
>> > > > > > > > > >> > > > >> >> > >
>> > > > > > > > > >> > > > >> >> > >
>> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, Gwen
>> > > Shapira
>> > > > > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
>> > > > > > > > > >> > > > >> >> > >>
>> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in the
>> > > wiki:
>> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for delegation
>> > > > > tokens?
>> > > > > > > > > >> Renewal? How
>> > > > > > > > > >> > > > do we
>> > > > > > > > > >> > > > >> >> > >> revoke them?
>> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation tokens
>> > for
>> > > > > > "do-as"
>> > > > > > > > > (say,
>> > > > > > > > > >> > > submit
>> > > > > > > > > >> > > > >> Storm
>> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a producer
>> > > for
>> > > > > > every
>> > > > > > > > job
>> > > > > > > > > >> (we
>> > > > > > > > > >> > > can't
>> > > > > > > > > >> > > > >> share
>> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running on
>> > same
>> > > > > node),
>> > > > > > > > since
>> > > > > > > > > >> we
>> > > > > > > > > >> > > only
>> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is there a
>> > > plan
>> > > > > to
>> > > > > > > > change
>> > > > > > > > > >> this
>> > > > > > > > > >> > > for
>> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow
>> > > multiple
>> > > > > > users
>> > > > > > > > > with
>> > > > > > > > > >> > > > different
>> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
>> > > > > > > > > >> > > > >> >> > >>
>> > > > > > > > > >> > > > >> >> > >> Gwen
>> > > > > > > > > >> > > > >> >> > >>
>> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, parth
>> > > > > brahmbhatt
>> > > > > > > > > >> > > > >> >> > >> <br...@gmail.com> wrote:
>> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can
>> > other
>> > > > > > > committers
>> > > > > > > > > >> review?
>> > > > > > > > > >> > > > >> >> > >> >
>> > > > > > > > > >> > > > >> >> > >> > Thanks
>> > > > > > > > > >> > > > >> >> > >> > Parth
>> > > > > > > > > >> > > > >> >> > >> >
>> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM,
>> > Harsha <
>> > > > > > > > > >> kafka@harsha.io>
>> > > > > > > > > >> > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> >
>> > > > > > > > > >> > > > >> >> > >> >> Parth,
>> > > > > > > > > >> > > > >> >> > >> >> Overall current design looks
>> > > > good
>> > > > > to
>> > > > > > > > me. I
>> > > > > > > > > >> am +1
>> > > > > > > > > >> > > on
>> > > > > > > > > >> > > > >> the
>> > > > > > > > > >> > > > >> >> > KIP.
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as
>> > well.
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> > >> >> -Harsha
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 AM,
>> > parth
>> > > > > > > > brahmbhatt
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of infinite
>> > > > lifetime
>> > > > > > > but I
>> > > > > > > > > >> see the
>> > > > > > > > > >> > > > >> Streaming
>> > > > > > > > > >> > > > >> >> > use
>> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use case I
>> > > was
>> > > > > > hoping
>> > > > > > > > > >> there will
>> > > > > > > > > >> > > > be
>> > > > > > > > > >> > > > >> some
>> > > > > > > > > >> > > > >> >> > notion
>> > > > > > > > > >> > > > >> >> > >> >> > of
>> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new
>> > > > delegation
>> > > > > > > tokens
>> > > > > > > > > at
>> > > > > > > > > >> fixed
>> > > > > > > > > >> > > > >> interval
>> > > > > > > > > >> > > > >> >> > and
>> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that is
>> > not
>> > > > the
>> > > > > > case
>> > > > > > > > for
>> > > > > > > > > >> we can
>> > > > > > > > > >> > > > >> discuss
>> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing them self
>> > > and
>> > > > > the
>> > > > > > > > > >> security
>> > > > > > > > > >> > > > >> implications
>> > > > > > > > > >> > > > >> >> > of the
>> > > > > > > > > >> > > > >> >> > >> >> > same.
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch
>> > tokens
>> > > > > from
>> > > > > > > > > >> zookeeper,
>> > > > > > > > > >> > > > >> overall I
>> > > > > > > > > >> > > > >> >> > think
>> > > > > > > > > >> > > > >> >> > >> >> > its
>> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely on our
>> > > > > metadata
>> > > > > > > > store
>> > > > > > > > > >> and I
>> > > > > > > > > >> > > > >> think we
>> > > > > > > > > >> > > > >> >> > are
>> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with all
>> > the
>> > > > > KIP-4
>> > > > > > > > > >> improvements.
>> > > > > > > > > >> > > > I
>> > > > > > > > > >> > > > >> chose
>> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the client
>> > > > will
>> > > > > > > still
>> > > > > > > > > >> just talk
>> > > > > > > > > >> > > > to
>> > > > > > > > > >> > > > >> >> > broker , its
>> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use zookeeper
>> > > > which
>> > > > > we
>> > > > > > > > > >> already do
>> > > > > > > > > >> > > > for a
>> > > > > > > > > >> > > > >> lot
>> > > > > > > > > >> > > > >> >> > of
>> > > > > > > > > >> > > > >> >> > >> >> > other
>> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development + and
>> > > the
>> > > > > > > ability
>> > > > > > > > > so
>> > > > > > > > > >> > > tokens
>> > > > > > > > > >> > > > >> will
>> > > > > > > > > >> > > > >> >> > survive
>> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster
>> > > failure.
>> > > > > if a
>> > > > > > > > > >> majority
>> > > > > > > > > >> > > > agrees
>> > > > > > > > > >> > > > >> the
>> > > > > > > > > >> > > > >> >> > added
>> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller
>> > > forwarding
>> > > > > > keys
>> > > > > > > to
>> > > > > > > > > all
>> > > > > > > > > >> > > > broker is
>> > > > > > > > > >> > > > >> >> > justified
>> > > > > > > > > >> > > > >> >> > >> >> > as
>> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security , I am
>> > > fine
>> > > > > > with
>> > > > > > > > that
>> > > > > > > > > >> option
>> > > > > > > > > >> > > > too.
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not support SSL
>> > > we
>> > > > > can
>> > > > > > > not
>> > > > > > > > > >> store
>> > > > > > > > > >> > > > master
>> > > > > > > > > >> > > > >> keys
>> > > > > > > > > >> > > > >> >> > in
>> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will be
>> > > exposed
>> > > > > on
>> > > > > > > > wire.
>> > > > > > > > > To
>> > > > > > > > > >> > > > support
>> > > > > > > > > >> > > > >> >> > rotation
>> > > > > > > > > >> > > > >> >> > >> >> > without affecting current clients is
>> > > > > > > something I
>> > > > > > > > > >> need to
>> > > > > > > > > >> > > > put
>> > > > > > > > > >> > > > >> more
>> > > > > > > > > >> > > > >> >> > thought
>> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal assumes the
>> > > > > rotation
>> > > > > > > > will
>> > > > > > > > > >> > > > invalidate
>> > > > > > > > > >> > > > >> all
>> > > > > > > > > >> > > > >> >> > current
>> > > > > > > > > >> > > > >> >> > >> >> > tokens.
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also review
>> > > and
>> > > > > post
>> > > > > > > > their
>> > > > > > > > > >> > > comments
>> > > > > > > > > >> > > > >> so we
>> > > > > > > > > >> > > > >> >> > can
>> > > > > > > > > >> > > > >> >> > >> >> > make
>> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > Thanks
>> > > > > > > > > >> > > > >> >> > >> >> > Parth
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 AM,
>> > > Ashish
>> > > > > > Singh
>> > > > > > > <
>> > > > > > > > > >> > > > >> asingh@cloudera.com
>> > > > > > > > > >> > > > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at 11:26 AM,
>> > > > > Harsha <
>> > > > > > > > > >> > > > kafka@harsha.io>
>> > > > > > > > > >> > > > >> >> > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion
>> > > threads
>> > > > on
>> > > > > > > this
>> > > > > > > > > KIP.
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from
>> > Jitendra
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large number of
>> > > > > clients
>> > > > > > > that
>> > > > > > > > > are
>> > > > > > > > > >> > > > running
>> > > > > > > > > >> > > > >> all
>> > > > > > > > > >> > > > >> >> > over the
>> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate with
>> > > Kafka
>> > > > > > > > brokers,
>> > > > > > > > > >> is very
>> > > > > > > > > >> > > > >> similar
>> > > > > > > > > >> > > > >> >> > to the
>> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large number
>> > of
>> > > > > tasks
>> > > > > > > > > running
>> > > > > > > > > >> > > across
>> > > > > > > > > >> > > > >> the
>> > > > > > > > > >> > > > >> >> > cluster
>> > > > > > > > > >> > > > >> >> > >> >> that
>> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs
>> > > > Namenode.
>> > > > > > > > > >> Therefore, the
>> > > > > > > > > >> > > > >> >> > delegation token
>> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a good
>> > fit
>> > > > for
>> > > > > > > this
>> > > > > > > > > use
>> > > > > > > > > >> case
>> > > > > > > > > >> > > > as we
>> > > > > > > > > >> > > > >> >> > have seen
>> > > > > > > > > >> > > > >> >> > >> >> it
>> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in HDFS
>> > and
>> > > > > YARN.
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is very
>> > much
>> > > > > > inline
>> > > > > > > > with
>> > > > > > > > > >> Hadoop
>> > > > > > > > > >> > > > >> >> > approach. A few
>> > > > > > > > > >> > > > >> >> > >> >> > > > comments:
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to allow
>> > > > > > infinite
>> > > > > > > > > >> renewable
>> > > > > > > > > >> > > > >> lifetime
>> > > > > > > > > >> > > > >> >> > for a
>> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a token
>> > to a
>> > > > max
>> > > > > > > life
>> > > > > > > > > time
>> > > > > > > > > >> > > > (default
>> > > > > > > > > >> > > > >> 7
>> > > > > > > > > >> > > > >> >> > days). A
>> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
>> > believed
>> > > to
>> > > > > > > > increase
>> > > > > > > > > >> with
>> > > > > > > > > >> > > > time.
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite
>> > > lifetime
>> > > > > > might
>> > > > > > > > not
>> > > > > > > > > >> be the
>> > > > > > > > > >> > > > best
>> > > > > > > > > >> > > > >> idea.
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the tokens
>> > are
>> > > > > stored
>> > > > > > > in
>> > > > > > > > > >> zookeeper
>> > > > > > > > > >> > > > as
>> > > > > > > > > >> > > > >> well,
>> > > > > > > > > >> > > > >> >> > and
>> > > > > > > > > >> > > > >> >> > >> >> can
>> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is clever
>> > > as
>> > > > it
>> > > > > > can
>> > > > > > > > > allow
>> > > > > > > > > >> > > > >> replacing the
>> > > > > > > > > >> > > > >> >> > tokens
>> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max life
>> > > time,
>> > > > > and
>> > > > > > > > > clients
>> > > > > > > > > >> can
>> > > > > > > > > >> > > > >> download
>> > > > > > > > > >> > > > >> >> > new
>> > > > > > > > > >> > > > >> >> > >> >> tokens
>> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It shouldn't be
>> > a
>> > > > big
>> > > > > > load
>> > > > > > > > on
>> > > > > > > > > >> > > zookeeper
>> > > > > > > > > >> > > > >> as a
>> > > > > > > > > >> > > > >> >> > client
>> > > > > > > > > >> > > > >> >> > >> >> will
>> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token once in
>> > > > > several
>> > > > > > > > days.
>> > > > > > > > > >> In this
>> > > > > > > > > >> > > > >> approach
>> > > > > > > > > >> > > > >> >> > you
>> > > > > > > > > >> > > > >> >> > >> >> don't
>> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on the
>> > > token
>> > > > > even
>> > > > > > > for
>> > > > > > > > > >> long
>> > > > > > > > > >> > > > running
>> > > > > > > > > >> > > > >> >> > clients.
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are
>> > > generated
>> > > > > > using
>> > > > > > > a
>> > > > > > > > > >> master
>> > > > > > > > > >> > > key.
>> > > > > > > > > >> > > > >> The
>> > > > > > > > > >> > > > >> >> > master
>> > > > > > > > > >> > > > >> >> > >> >> key
>> > > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically
>> > > changed.
>> > > > > In
>> > > > > > > > > Hadoop,
>> > > > > > > > > >> the
>> > > > > > > > > >> > > > >> default
>> > > > > > > > > >> > > > >> >> > renewal
>> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require brokers
>> > > > > > maintaining
>> > > > > > > a
>> > > > > > > > > >> list of X
>> > > > > > > > > >> > > > most
>> > > > > > > > > >> > > > >> >> > recent
>> > > > > > > > > >> > > > >> >> > >> >> master
>> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to be
>> > > > > persisted
>> > > > > > > > > >> somewhere, as
>> > > > > > > > > >> > > > if a
>> > > > > > > > > >> > > > >> >> > broker
>> > > > > > > > > >> > > > >> >> > >> >> goes
>> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get that list
>> > > > again
>> > > > > > and
>> > > > > > > > > >> storing
>> > > > > > > > > >> > > > master
>> > > > > > > > > >> > > > >> keys
>> > > > > > > > > >> > > > >> >> > on ZK
>> > > > > > > > > >> > > > >> >> > >> >> is
>> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However, if a
>> > > > broker
>> > > > > > goes
>> > > > > > > > > down
>> > > > > > > > > >> then
>> > > > > > > > > >> > > we
>> > > > > > > > > >> > > > >> have
>> > > > > > > > > >> > > > >> >> > much
>> > > > > > > > > >> > > > >> >> > >> >> bigger
>> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and client can
>> > > > always
>> > > > > > > > > >> > > re-authenticate
>> > > > > > > > > >> > > > is
>> > > > > > > > > >> > > > >> such
>> > > > > > > > > >> > > > >> >> > >> >> events.
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a look at
>> > > other
>> > > > > > > > > >> alternatives
>> > > > > > > > > >> > > this
>> > > > > > > > > >> > > > >> list has
>> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough proposal,
>> > > > great
>> > > > > > > work!"
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at 10:28
>> > PM,
>> > > > Gwen
>> > > > > > > > Shapira
>> > > > > > > > > >> wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me. Thanks!
>> > > > > > > > > >> > > > >> >> > >> >> > > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at 9:25
>> > PM,
>> > > > > > Harsha <
>> > > > > > > > > >> > > > kafka@harsha.io
>> > > > > > > > > >> > > > >> >
>> > > > > > > > > >> > > > >> >> > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any release
>> > > > > vehicle
>> > > > > > > but
>> > > > > > > > > >> still the
>> > > > > > > > > >> > > > >> work can
>> > > > > > > > > >> > > > >> >> > move
>> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is interested in
>> > the
>> > > > KIP
>> > > > > > > > please
>> > > > > > > > > >> do the
>> > > > > > > > > >> > > > >> review and
>> > > > > > > > > >> > > > >> >> > >> >> provide
>> > > > > > > > > >> > > > >> >> > >> >> > > the
>> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at
>> > 04:59
>> > > > PM,
>> > > > > > > Ismael
>> > > > > > > > > >> Juma
>> > > > > > > > > >> > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would be
>> > good
>> > > > to
>> > > > > > have
>> > > > > > > > > more
>> > > > > > > > > >> time
>> > > > > > > > > >> > > to
>> > > > > > > > > >> > > > >> review
>> > > > > > > > > >> > > > >> >> > and
>> > > > > > > > > >> > > > >> >> > >> >> > > discuss
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at
>> > 12:55
>> > > > AM,
>> > > > > > Gwen
>> > > > > > > > > >> Shapira <
>> > > > > > > > > >> > > > >> >> > >> >> gwen@confluent.io>
>> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 depends on
>> > > > KIP-43,
>> > > > > > > which
>> > > > > > > > > is
>> > > > > > > > > >> > > > already a
>> > > > > > > > > >> > > > >> bit
>> > > > > > > > > >> > > > >> >> > of a
>> > > > > > > > > >> > > > >> >> > >> >> risk
>> > > > > > > > > >> > > > >> >> > >> >> > > for
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release - any
>> > > chance
>> > > > > we
>> > > > > > > can
>> > > > > > > > > >> delay
>> > > > > > > > > >> > > > >> delegation
>> > > > > > > > > >> > > > >> >> > tokens
>> > > > > > > > > >> > > > >> >> > >> >> to
>> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community
>> > working
>> > > > on a
>> > > > > > > > release
>> > > > > > > > > >> every
>> > > > > > > > > >> > > 3
>> > > > > > > > > >> > > > >> month,
>> > > > > > > > > >> > > > >> >> > this
>> > > > > > > > > >> > > > >> >> > >> >> is not
>> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, 2016 at
>> > > 5:11
>> > > > > PM,
>> > > > > > > > Ashish
>> > > > > > > > > >> Singh
>> > > > > > > > > >> > > <
>> > > > > > > > > >> > > > >> >> > >> >> > > asingh@cloudera.com>
>> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for the
>> > > > awesome
>> > > > > > > write
>> > > > > > > > > up.
>> > > > > > > > > >> > > > Following
>> > > > > > > > > >> > > > >> our
>> > > > > > > > > >> > > > >> >> > >> >> discussion
>> > > > > > > > > >> > > > >> >> > >> >> > > > from the
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it will
>> > be
>> > > > > easier
>> > > > > > > to
>> > > > > > > > > >> compare
>> > > > > > > > > >> > > > >> various
>> > > > > > > > > >> > > > >> >> > >> >> alternatives
>> > > > > > > > > >> > > > >> >> > >> >> > > > if they
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together. I am
>> > > > stating
>> > > > > > > > below a
>> > > > > > > > > >> few
>> > > > > > > > > >> > > > >> >> > alternatives along
>> > > > > > > > > >> > > > >> >> > >> >> > > with
>> > > > > > > > > >> > > > >> >> > >> >> > > > a the
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal)
>> > Store
>> > > > > > > Delegation
>> > > > > > > > > >> Token,
>> > > > > > > > > >> > > DT,
>> > > > > > > > > >> > > > >> on ZK.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> > > authenticates
>> > > > > > with a
>> > > > > > > > > >> broker.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
>> > > > > > > > authenticated,
>> > > > > > > > > >> it
>> > > > > > > > > >> > > will
>> > > > > > > > > >> > > > >> make a
>> > > > > > > > > >> > > > >> >> > broker
>> > > > > > > > > >> > > > >> >> > >> >> side
>> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
>> > > > token.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
>> > > generates
>> > > > a
>> > > > > > > shared
>> > > > > > > > > >> secret
>> > > > > > > > > >> > > > based
>> > > > > > > > > >> > > > >> on
>> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret
>> > shared
>> > > > > > between
>> > > > > > > > all
>> > > > > > > > > >> > > brokers,
>> > > > > > > > > >> > > > >> >> > randomly
>> > > > > > > > > >> > > > >> >> > >> >> > > generated
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker stores
>> > this
>> > > > > token
>> > > > > > in
>> > > > > > > > its
>> > > > > > > > > >> in
>> > > > > > > > > >> > > > memory
>> > > > > > > > > >> > > > >> cache.
>> > > > > > > > > >> > > > >> >> > >> >> Broker
>> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the DelegationToken
>> > > > > without
>> > > > > > > the
>> > > > > > > > > >> hmac in
>> > > > > > > > > >> > > the
>> > > > > > > > > >> > > > >> >> > zookeeper.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers will
>> > > > have a
>> > > > > > > cache
>> > > > > > > > > >> backed
>> > > > > > > > > >> > > by
>> > > > > > > > > >> > > > >> >> > zookeeper so
>> > > > > > > > > >> > > > >> >> > >> >> they
>> > > > > > > > > >> > > > >> >> > >> >> > > > will all
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified
>> > whenever
>> > > a
>> > > > > new
>> > > > > > > > token
>> > > > > > > > > is
>> > > > > > > > > >> > > > >> generated and
>> > > > > > > > > >> > > > >> >> > they
>> > > > > > > > > >> > > > >> >> > >> >> will
>> > > > > > > > > >> > > > >> >> > >> >> > > > update
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache whenever
>> > > > token
>> > > > > > > state
>> > > > > > > > > >> changes.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker returns
>> > the
>> > > > > token
>> > > > > > to
>> > > > > > > > > >> Client.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
>> > fixes
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable race
>> > > > > condition,
>> > > > > > > > client
>> > > > > > > > > >> tries
>> > > > > > > > > >> > > to
>> > > > > > > > > >> > > > >> >> > authenticate
>> > > > > > > > > >> > > > >> >> > >> >> with
>> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to be
>> > > > updated
>> > > > > > with
>> > > > > > > > the
>> > > > > > > > > >> newly
>> > > > > > > > > >> > > > >> generated
>> > > > > > > > > >> > > > >> >> > DT.
>> > > > > > > > > >> > > > >> >> > >> >> This
>> > > > > > > > > >> > > > >> >> > >> >> > > can
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with making
>> > > > > dtRequest
>> > > > > > > > block
>> > > > > > > > > >> until
>> > > > > > > > > >> > > all
>> > > > > > > > > >> > > > >> >> > brokers have
>> > > > > > > > > >> > > > >> >> > >> >> > > > updated
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk barrier or
>> > > > > similar
>> > > > > > > > > >> mechanism
>> > > > > > > > > >> > > can
>> > > > > > > > > >> > > > be
>> > > > > > > > > >> > > > >> used.
>> > > > > > > > > >> > > > >> >> > >> >> However,
>> > > > > > > > > >> > > > >> >> > >> >> > > > all such
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will
>> > > increase
>> > > > > > > > > complexity.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a static
>> > > secret
>> > > > > key
>> > > > > > > > from
>> > > > > > > > > >> config
>> > > > > > > > > >> > > > >> file. Will
>> > > > > > > > > >> > > > >> >> > >> >> require
>> > > > > > > > > >> > > > >> >> > >> >> > > yet
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and uses a
>> > > static
>> > > > > > > secret
>> > > > > > > > > >> key. It
>> > > > > > > > > >> > > is
>> > > > > > > > > >> > > > >> advised
>> > > > > > > > > >> > > > >> >> > to
>> > > > > > > > > >> > > > >> >> > >> >> rotate
>> > > > > > > > > >> > > > >> >> > >> >> > > > secret
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. This
>> > can
>> > > > be
>> > > > > > > > avoided
>> > > > > > > > > >> with
>> > > > > > > > > >> > > > >> controller
>> > > > > > > > > >> > > > >> >> > >> >> generating
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to brokers
>> > > > > > > periodically.
>> > > > > > > > > >> However,
>> > > > > > > > > >> > > > >> this will
>> > > > > > > > > >> > > > >> >> > >> >> require
>> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain certain
>> > > counts
>> > > > of
>> > > > > > > > > >> secretKeys.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1) Have
>> > > > > controller
>> > > > > > > > > >> generate
>> > > > > > > > > >> > > > >> delegation
>> > > > > > > > > >> > > > >> >> > token.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> > > authenticates
>> > > > > > with a
>> > > > > > > > > >> broker.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
>> > > > > > > > authenticated,
>> > > > > > > > > >> it
>> > > > > > > > > >> > > will
>> > > > > > > > > >> > > > >> make a
>> > > > > > > > > >> > > > >> >> > broker
>> > > > > > > > > >> > > > >> >> > >> >> side
>> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
>> > > > token.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker forwards
>> > the
>> > > > > > request
>> > > > > > > > to
>> > > > > > > > > >> > > > controller.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller
>> > > generates
>> > > > a
>> > > > > DT
>> > > > > > > and
>> > > > > > > > > >> > > broadcasts
>> > > > > > > > > >> > > > >> to all
>> > > > > > > > > >> > > > >> >> > >> >> brokers.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker stores
>> > this
>> > > > > token
>> > > > > > in
>> > > > > > > > its
>> > > > > > > > > >> memory
>> > > > > > > > > >> > > > >> cache.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller
>> > responds
>> > > > to
>> > > > > > > > broker’s
>> > > > > > > > > >> DT
>> > > > > > > > > >> > > req.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker returns
>> > the
>> > > > > token
>> > > > > > to
>> > > > > > > > > >> Client.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
>> > fixes
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will have to
>> > add
>> > > > new
>> > > > > > > APIs
>> > > > > > > > to
>> > > > > > > > > >> > > support
>> > > > > > > > > >> > > > >> >> > controller
>> > > > > > > > > >> > > > >> >> > >> >> pushing
>> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top of
>> > the
>> > > > > > minimal
>> > > > > > > > APIs
>> > > > > > > > > >> that
>> > > > > > > > > >> > > are
>> > > > > > > > > >> > > > >> >> > currently
>> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will also have
>> > > to
>> > > > > add
>> > > > > > > APIs
>> > > > > > > > > to
>> > > > > > > > > >> > > support
>> > > > > > > > > >> > > > >> the
>> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
>> > > > > > > > > >> > > > >> >> > >> >> > > > case,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new broker
>> > > comes
>> > > > up
>> > > > > > it
>> > > > > > > > will
>> > > > > > > > > >> have
>> > > > > > > > > >> > > to
>> > > > > > > > > >> > > > >> get all
>> > > > > > > > > >> > > > >> >> > >> >> delegation
>> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In catastrophic
>> > > > > failures
>> > > > > > > > where
>> > > > > > > > > >> all
>> > > > > > > > > >> > > > brokers
>> > > > > > > > > >> > > > >> go
>> > > > > > > > > >> > > > >> >> > down,
>> > > > > > > > > >> > > > >> >> > >> >> the
>> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
>> > > servers
>> > > > > are
>> > > > > > > > > >> restarted as
>> > > > > > > > > >> > > > >> tokens
>> > > > > > > > > >> > > > >> >> > are not
>> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens,
>> > then
>> > > > > there
>> > > > > > > are
>> > > > > > > > > more
>> > > > > > > > > >> > > > important
>> > > > > > > > > >> > > > >> >> > things to
>> > > > > > > > > >> > > > >> >> > >> >> > > worry
>> > > > > > > > > >> > > > >> >> > >> >> > > > about
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better
>> > to
>> > > > > > > > > >> re-authenticate.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2) Do not
>> > > > > > distribute
>> > > > > > > > DT
>> > > > > > > > > to
>> > > > > > > > > >> > > other
>> > > > > > > > > >> > > > >> brokers
>> > > > > > > > > >> > > > >> >> > at
>> > > > > > > > > >> > > > >> >> > >> >> all.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client
>> > > authenticates
>> > > > > > with a
>> > > > > > > > > >> broker.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is
>> > > > > > > > authenticated,
>> > > > > > > > > >> it
>> > > > > > > > > >> > > will
>> > > > > > > > > >> > > > >> make a
>> > > > > > > > > >> > > > >> >> > broker
>> > > > > > > > > >> > > > >> >> > >> >> side
>> > > > > > > > > >> > > > >> >> > >> >> > > > call to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation
>> > > > token.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker
>> > > generates
>> > > > DT
>> > > > > > of
>> > > > > > > > > form,
>> > > > > > > > > >> > > [hmac +
>> > > > > > > > > >> > > > >> (owner,
>> > > > > > > > > >> > > > >> >> > >> >> renewer,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime, id,
>> > hmac,
>> > > > > > > > > >> expirationTime)]
>> > > > > > > > > >> > > and
>> > > > > > > > > >> > > > >> passes
>> > > > > > > > > >> > > > >> >> > back
>> > > > > > > > > >> > > > >> >> > >> >> this
>> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is generated
>> > via
>> > > > > > > > > >> {HMAC-SHA256(owner,
>> > > > > > > > > >> > > > >> renewer,
>> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime)
>> > using
>> > > > > > > > SecretKey}.
>> > > > > > > > > >> Note
>> > > > > > > > > >> > > that
>> > > > > > > > > >> > > > >> all
>> > > > > > > > > >> > > > >> >> > brokers
>> > > > > > > > > >> > > > >> >> > >> >> have
>> > > > > > > > > >> > > > >> >> > >> >> > > > this
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then goes
>> > to
>> > > > any
>> > > > > > > > broker
>> > > > > > > > > >> and to
>> > > > > > > > > >> > > > >> >> > authenticate
>> > > > > > > > > >> > > > >> >> > >> >> sends
>> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker recalculates
>> > > hmac
>> > > > > > using
>> > > > > > > > > >> (owner,
>> > > > > > > > > >> > > > >> renewer,
>> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
>> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) info
>> > > > from
>> > > > > DT
>> > > > > > > and
>> > > > > > > > > its
>> > > > > > > > > >> > > > >> SecretKey. If
>> > > > > > > > > >> > > > >> >> > it
>> > > > > > > > > >> > > > >> >> > >> >> matches
>> > > > > > > > > >> > > > >> >> > >> >> > > > with
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is
>> > > > > authenticated.
>> > > > > > > > Yes,
>> > > > > > > > > >> it will
>> > > > > > > > > >> > > > do
>> > > > > > > > > >> > > > >> other
>> > > > > > > > > >> > > > >> >> > >> >> obvious
>> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp expiry and
>> > > > such.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret key
>> > will
>> > > > be
>> > > > > > > > > generated
>> > > > > > > > > >> by
>> > > > > > > > > >> > > > >> controller
>> > > > > > > > > >> > > > >> >> > and
>> > > > > > > > > >> > > > >> >> > >> >> passed
>> > > > > > > > > >> > > > >> >> > >> >> > > to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and
>> > fixes
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to delete a
>> > DT?
>> > > > > Yes,
>> > > > > > > that
>> > > > > > > > > is
>> > > > > > > > > >> a
>> > > > > > > > > >> > > > downside
>> > > > > > > > > >> > > > >> >> > here.
>> > > > > > > > > >> > > > >> >> > >> >> However,
>> > > > > > > > > >> > > > >> >> > >> >> > > > this can
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with
>> > > brokers
>> > > > > > > > > maintaining
>> > > > > > > > > >> a
>> > > > > > > > > >> > > > >> blacklist of
>> > > > > > > > > >> > > > >> >> > DTs,
>> > > > > > > > > >> > > > >> >> > >> >> DTs
>> > > > > > > > > >> > > > >> >> > >> >> > > > from this
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed after
>> > > > > expiry.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In catastrophic
>> > > > > failures
>> > > > > > > > where
>> > > > > > > > > >> all
>> > > > > > > > > >> > > > brokers
>> > > > > > > > > >> > > > >> go
>> > > > > > > > > >> > > > >> >> > down,
>> > > > > > > > > >> > > > >> >> > >> >> the
>> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if
>> > > servers
>> > > > > are
>> > > > > > > > > >> restarted as
>> > > > > > > > > >> > > > >> tokens
>> > > > > > > > > >> > > > >> >> > are not
>> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens,
>> > then
>> > > > > there
>> > > > > > > are
>> > > > > > > > > more
>> > > > > > > > > >> > > > important
>> > > > > > > > > >> > > > >> >> > things to
>> > > > > > > > > >> > > > >> >> > >> >> > > worry
>> > > > > > > > > >> > > > >> >> > >> >> > > > about
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better
>> > to
>> > > > > > > > > >> re-authenticate.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, 2016 at
>> > > > 1:58
>> > > > > > PM,
>> > > > > > > > > Parth
>> > > > > > > > > >> > > > >> Brahmbhatt <
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > pbrahmbhatt@hortonworks.com>
>> > > > > > > > wrote:
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed KIP-48 so
>> > > we
>> > > > > can
>> > > > > > > > offer
>> > > > > > > > > >> hadoop
>> > > > > > > > > >> > > > like
>> > > > > > > > > >> > > > >> >> > delegation
>> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can review
>> > > the
>> > > > > > design
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > >
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends on
>> > > KIP-43
>> > > > > and
>> > > > > > > we
>> > > > > > > > > >> have also
>> > > > > > > > > >> > > > >> >> > discussed an
>> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design here<
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > >
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
>> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
>> > > > > > > > > >> > > > >> >> > >> >> > > >
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > --
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
>> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
>> > > > > > > > > >> > > > >> >> > >> >> > >
>> > > > > > > > > >> > > > >> >> > >> >>
>> > > > > > > > > >> > > > >> >> >
>> > > > > > > > > >> > > > >>
>> > > > > > > > > >> > > >
>> > > > > > > > > >> > >
>> > > > > > > > > >>
>> > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Regards,
>> > > > > >
>> > > > > > Rajini
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Rajini
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Liquan Pei
>> Software Engineer, Confluent Inc