You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Guozhang Wang <wa...@gmail.com> on 2022/10/03 04:15:04 UTC
Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol
1. Yeah that makes sense. I originally concerned if there's any consistency
issues if the HB request with encoded topic IDs and the GPA/GIA RPCs with
encoded topic IDs use mappings from two different brokers, and one of them
is stale, but after thinking twice about it I think as long as we handle
e.g. `unknown topic ids` from HB responses then we should be fine.
3. Ack, in that case I think we also need to define "ConsumerGroupMetadata"
which is passed in `onAssignment`. We have a schema defined in the
persistent storage for different purposes, but not in the consumer APIs
yet. I think that would just include a single "Map<String, Metadata>"?
On Fri, Sep 30, 2022 at 5:51 AM David Jacot <dj...@confluent.io.invalid>
wrote:
> Hey Guozhang,
>
> Thanks for your comments.
>
> 1. That's a very good question. I think that it is better to separate
> the two concerns here. In my opinion, the GPA should only be used to
> feed the assignor. I see the GPA as a way to call a remote function so
> the GPA provides all its inputs to ensure that the assignor does not
> depend on any local state. Knowing this, I would not use the GPA to
> update the local state either. That is not its purpose. The consumer
> is responsible to update its local metadata cache either when it gets
> an unknown topic in the user-provided subscription or in the
> coordinator-provided assignment. The consumer has to do this anyway to
> get all topic metadata (e.g. leaders).
>
> 2. Interesting thought. Handling the translation is definitely
> possible. I wonder if there are cases where having the topic ids is
> beneficial. I am thinking about when a topic is deleted and recreated
> right away. In this case, the current member assignment would still
> have the old topic id and we would not be able to resolve it anymore.
> I suppose that we should directly remove it from the current
> assignment on the server side in this case and only add the new one.
> As I first thought, that seems OK. The major downside is that the user
> of the assignor would not know that the topic is different from what
> it may have cached locally.
>
> We could say that we don't expose topic ids externally for now in
> order to stay consistent with the other interfaces but it would be
> annoying to add it back if we want to expose topic ids in the future.
> I personally believe that we will have to expose them at some point.
> For instance, in the rebalance listener, it would be useful to know
> that a topic has been deleted and recreated. The only way to do this
> is to know the topic id. Let me think a little more about this and
> come back to you.
>
> 3. Sorry for this one. I was a typo. PartitionAssingor#assign should
> return GroupAssignment.
>
> Best,
> David
>
> On Thu, Sep 29, 2022 at 6:06 PM Guozhang Wang <wa...@gmail.com> wrote:
> >
> > Thanks David, I read through the updated wiki, and I have a couple
> > follow-up questions:
> >
> > 1. It seems we would have two potential sources to update the consumer's
> > internally maintained topic name -> topic ID mapping, which is used for
> > encoding subscribed topic Ids in the HB request, and offset commit / list
> > offset requests, etc. One is from metadata responses, and another one is
> > from the GPA (group prepare assignment). On the other hand, the assignor
> > APIs directly take parsed AssignmentMemberSpec and
> AssignmentTopicMetadata
> > from GPA responses directly instead of trying to consult the consumer's
> own
> > cached mapping. So just to clarify, consumers would also update its
> > internal mapping from the GPA responses as well, right?
> >
> > 2. I'm wondering if it's necessary to expose the topic IDs to the
> assignor
> > developers. What they need to do at the moment is to take the mapping
> from
> > AssignmentTopicMetadata, and translate their decided assignment into
> topic
> > IDs when returning the MemberAssignment. We can instead just let consumer
> > to do such translation for them internally and still let the assignor
> > developers to only deal with the topic names. Of course that means the
> > client side assignor APIs would be a bit further diverged from the
> > broker-side assignor such that:
> >
> > a) Topic metadata would be encoded just as String -> num.partitions;
> > b) AssignmentMemberSpec contains subscribedTopicNames and topicPartitions
> > as target assignments;
> > c) Returned MemberAssignment contains list of topicPartitions instead of
> > topicIdPartitions.
> >
> > Internally, consumers would convert the IDs to names when calling the
> > assign() function, and convert them back when sending the request; and
> > similarly it would convert the IDs to names when calling onAssignment()
> as
> > well.
> >
> > In short, I'm wondering if we should bite the bullet for the users,
> eating
> > more complexities internally than pushing that to users.
> >
> > 3. This is a nit one: the return type of `Assignment
> assign(AssignmentSpec
> > assignmentSpec);` seems not defined in the doc. I think it would just
> > include two fields, as:
> >
> > class Assignment {
> > GroupAssignment groupAssignment;
> >
> > ConsumerGroupMetadata consumerGroupMetadata;
> > }
> >
> > Is that right?
> >
> > Guozhang
> >
> > On Thu, Sep 29, 2022 at 1:26 AM David Jacot <djacot@confluent.io.invalid
> >
> > wrote:
> >
> > > Hi Guozhang,
> > >
> > > Thanks for your input.
> > >
> > > > 1) in HB request, when clients send the subscription, they are still
> > > > sending them in terms of topic names. That is, the consumer would
> not try
> > > > to keep all the mappings from topic names to topic ids for the
> subscribed
> > > > topics.
> > >
> > > This is a mistake of mine. The heartbeat request should use Topic Ids.
> > > I forgot to change it. The idea is that the client will get the
> > > metadata first to translate the topic names to topic ids and thus also
> > > have the mapping in its local metadata cache.
> > >
> > > > 2) in HB responses, the assignment's topics are encoded as IDs
> already.
> > > And
> > > > fetch requests can directly rely on that since they just need topic
> IDs
> > > and
> > > > do not care what actual topic names are.
> > >
> > > That's right.
> > >
> > > > 3) in prepare-assignment responses, we return subscriptions as topic
> IDs
> > > as
> > > > well.
> > >
> > > That's right.
> > >
> > > > In a word, the consumer does not try to maintain the mapping of topic
> > > names
> > > > to IDs at all, and even if they do, they will not expose it in e.g.
> > > > assignor APIs. But for client-side assignors like Streams, if they
> do not
> > > > maintain the mapping from topic IDs to topic names, certain logic
> such as
> > > > co-partitioning indeed cannot be executed.
> > >
> > > As I said, the consumer will have the mapping in its local metadata
> > > cache. It needs to get the partition metadata anyway and the topic id
> > > is part of it. However, I do agree that this mapping is not exposed.
> > >
> > > > So I think we'd need to consider 1) use topic names in subscriptions
> in
> > > the
> > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > PartitionAssignor#TopicMetadata#topicName.
> > >
> > > 1) makes sense to me. I think that we could actually provide both
> > > names and ids. Regarding 2), I think that I am slightly in favor of
> > > using both names and ids as well. Let me try to update the protocol
> > > and the interface towards this.
> > >
> > > Thanks,
> > > David
> > >
> > > On Wed, Sep 28, 2022 at 9:35 PM Guozhang Wang <wa...@gmail.com>
> wrote:
> > > >
> > > > Hello Jun/David,
> > > >
> > > > Just to chime in here for Jun's comment #50, today Streams does not
> have
> > > a
> > > > way to map topic names to topic IDs, and in its topology description
> > > topics
> > > > are encoded with names.
> > > >
> > > > What I originally thought was that, on the consumer side, we'd need
> to
> > > > maintain this mapping from topic names to topic IDs anyways, and
> Streams
> > > > should leverage on the consumer's functionality to get the
> corresponding
> > > > topic names for the given topic IDs. But when I read through the RPC
> > > again
> > > > I realized it may not be the case, since:
> > > >
> > > > 1) in HB request, when clients send the subscription, they are still
> > > > sending them in terms of topic names. That is, the consumer would
> not try
> > > > to keep all the mappings from topic names to topic ids for the
> subscribed
> > > > topics.
> > > > 2) in HB responses, the assignment's topics are encoded as IDs
> already.
> > > And
> > > > fetch requests can directly rely on that since they just need topic
> IDs
> > > and
> > > > do not care what actual topic names are.
> > > > 3) in prepare-assignment responses, we return subscriptions as topic
> IDs
> > > as
> > > > well.
> > > >
> > > > In a word, the consumer does not try to maintain the mapping of topic
> > > names
> > > > to IDs at all, and even if they do, they will not expose it in e.g.
> > > > assignor APIs. But for client-side assignors like Streams, if they
> do not
> > > > maintain the mapping from topic IDs to topic names, certain logic
> such as
> > > > co-partitioning indeed cannot be executed.
> > > >
> > > > So I think we'd need to consider 1) use topic names in subscriptions
> in
> > > the
> > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > PartitionAssignor#TopicMetadata#topicName.
> > > >
> > > > On Wed, Sep 28, 2022 at 1:26 AM David Jacot
> <djacot@confluent.io.invalid
> > > >
> > > > wrote:
> > > >
> > > > > Hi Jun,
> > > > >
> > > > > Thanks for your comments.
> > > > >
> > > > > 48. Okay. There is a typo in the text:
> > > > > > Note that the persisted target assignment of B already includes
> foo-2
> > > > > here
> > > > > but B does not get it yet because foo-2 is not free.
> > > > >
> > > > > It should be:
> > > > > > Note that the persisted member assignment of B already includes
> foo-2
> > > > > here
> > > > > but B does not get it yet because foo-2 is not free.
> > > > >
> > > > > The examples are showing the persisted state in the
> __consumer_offsets
> > > > > topics. When a member transitions to the next epoch, we directly
> > > > > persist the whole target assignment in order to reduce the number
> of
> > > > > writes and we manage dependencies in memory. So "B - epoch=2,
> > > > > partitions=[foo-2]" is actually correct from a persisted record
> > > > > perspective. However, the in-memory state would be different
> because
> > > > > foo-2 is not yet available as you pointed out. The in-memory state
> > > > > would like as follow:
> > > > >
> > > > > Group (epoch=2)
> > > > > A
> > > > > B
> > > > > Target Assignment (epoch=2)
> > > > > A - partitions=[foo-0, foo-1]
> > > > > B - partitions=[foo-2]
> > > > > Member Assignment
> > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2], pending
> partitions=[]
> > > > > B - epoch=2, partitions=[], pending partitions=[foo-2]
> > > > >
> > > > > I suppose that the confusion comes from this. I will go and convert
> > > > > all the examples to their in-memory representation in order to
> reduce
> > > > > the confusion.
> > > > >
> > > > > 50. Okay. I have to admit that I am not so familiar with the
> internals
> > > > > of KStreams. If I understand correctly, you're saying that KStreams
> > > > > may not have topic ids in its topology so topic names are missing
> here
> > > > > to match the topics. I was hoping that KStreams would have topic
> ids
> > > > > in the topology as well. If that is not the case, we can also add
> > > > > topic names on our side. I will check this.
> > > > >
> > > > > 53. It seems that I was not clear. Sorry for that. I did not mean
> to
> > > > > say that MemberAssignment.metadata is not used by the group
> > > > > coordinator. Let me re-explain the flow.
> > > > >
> > > > > PartitionAssignor.assign() returns an Assignment. Assignment
> contains
> > > > > MemberAssignments where each member has MemberAssignment.metadata.
> The
> > > > > Assignment is installed by the client with the
> > > > > ConsumerGroupInstallAssignment API and this overrides the current
> > > > > target assignment in the coordinator. Then the group coordinator
> uses
> > > > > it to reconcile members. Therefore the target assignment will
> > > > > eventually become the current member assignment and it will be
> > > > > delivered to the member, including the original
> > > > > MemberAssignment.metadata defines by the assignor.
> > > > >
> > > > > I hope that this helps to clarify how MemberAssignment.metadata is
> > > > > used afterwards.
> > > > >
> > > > > Best,
> > > > > David
> > > > >
> > > > > On Tue, Sep 27, 2022 at 10:31 PM Jun Rao <jun@confluent.io.invalid
> >
> > > wrote:
> > > > > > Hi, David,
> > > > > >
> > > > > > Thanks for the reply. A few more follow up questions.
> > > > > >
> > > > > > 48. Hmm, so member assignment is what the client sees. Then, I
> am not
> > > > > sure
> > > > > > about the following. The text says B cannot get foo-2 yet. But in
> > > Member
> > > > > > Assignment, B has foo-2.
> > > > > >
> > > > > > At this point B can transition to epoch 2 but cannot get foo-2
> until
> > > A
> > > > > > revokes it.
> > > > > >
> > > > > > Note that the persisted target assignment of B already includes
> foo-2
> > > > > here
> > > > > > but B does not get it yet because foo-2 is not free.
> > > > > >
> > > > > > Group (epoch=2)
> > > > > > A
> > > > > > B
> > > > > > Target Assignment (epoch=2)
> > > > > > A - partitions=[foo-0, foo-1]
> > > > > > B - partitions=[foo-2]
> > > > > > Member Assignment
> > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2]
> > > > > > B - epoch=2, partitions=[foo-2]
> > > > > >
> > > > > >
> > > > > > 50. Hmm, I thought KStreams doesn't use range assignor all the
> time.
> > > It
> > > > > > does the assignment based on the topology. For example, if you
> are
> > > > > joining
> > > > > > two topics A and B and deriving the join result in a 3rd topic C,
> > > which
> > > > > > will be aggregated. You want to use range to co-locate the
> partitions
> > > > > from
> > > > > > topic A and B. However, for topic C, the partition could
> potentially
> > > be
> > > > > > assigned in round robin. Without the topic name, KStreams won't
> know
> > > > > which
> > > > > > topics need to be co-located.
> > > > > >
> > > > > > 53. If MemberAssignment.metadata is not used by the group
> > > coordinator,
> > > > > > where is it being used?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Tue, Sep 27, 2022 at 1:20 PM Jason Gustafson
> > > > > <ja...@confluent.io.invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi David,
> > > > > > >
> > > > > > > Thanks for the response. Replied below:
> > > > > > >
> > > > > > > > I don't have a strong opinion on this one. However, it seems
> that
> > > > > this
> > > > > > > could be a pandora box if our subset turns out to not be
> sufficient
> > > > > > > for some existing users. We will have to add more to it and the
> > > > > > > compatibility story between versions could be a real
> nightmare. One
> > > > > > > thing that we could do for sure is disabling all unicode
> groups in
> > > the
> > > > > > > library. That seems possible and would already reduce the
> scope.
> > > > > > >
> > > > > > > Changes to the regex syntax seems not a major problem to me. We
> > > would
> > > > > bump
> > > > > > > the protocol version as we usually do. Actually that begs the
> same
> > > > > question
> > > > > > > about RE2. How stable is the syntax and does it differ
> depending
> > > on the
> > > > > > > version of the library? In general, I do agree that we'd be
> better
> > > off
> > > > > not
> > > > > > > having to manage the syntax ourselves if we can convince
> ourselves
> > > that
> > > > > > > compatibility won't be a problem and the client won't
> eventually
> > > need
> > > > > to
> > > > > > > have access to it.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Jason
> > > > > > >
> > > > > > > On Tue, Sep 27, 2022 at 6:21 AM David Jacot
> > > > > <dj...@confluent.io.invalid>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Luke,
> > > > > > > >
> > > > > > > > Thanks for your comments.
> > > > > > > >
> > > > > > > > Your suggestion makes sense. Jun's made the same one. We are
> also
> > > > > > > > discussing whether we should directly return the group state
> in
> > > the
> > > > > > > > response and drop the prepare assignment api. It seems that
> the
> > > > > > > > current consensus is to keep it though. If that stays like
> this,
> > > I
> > > > > > > > will apply your suggestion.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > David
> > > > > > > >
> > > > > > > > On Tue, Sep 27, 2022 at 10:40 AM Luke Chen <
> showuon@gmail.com>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi David,
> > > > > > > > >
> > > > > > > > > Thanks for your patience for answering all these questions!
> > > > > > > > >
> > > > > > > > > One more question from me:
> > > > > > > > > 1. When the selected consumer is going to be in client-side
> > > > > assigning,
> > > > > > > we
> > > > > > > > > will return COMPUTE_ASSIGNMENT in heartbeat response. I'm
> > > > > wondering if
> > > > > > > we
> > > > > > > > > could use a flag to indicate the assignment is required.
> The
> > > > > rationale
> > > > > > > is
> > > > > > > > > when returning error, we lose a chance to catch up with the
> > > latest
> > > > > > > target
> > > > > > > > > assignment or revoke partitions. Since we expected the
> record
> > > > > fetching
> > > > > > > is
> > > > > > > > > not blocked by the rebalance/assignment computing, we might
> > > want to
> > > > > > > allow
> > > > > > > > > the members to update the member assignment in each
> heartbeat.
> > > > > > > > >
> > > > > > > > > Luke
> > > > > > > > >
> > > > > > > > > On Tue, Sep 27, 2022 at 9:43 AM Jason Gustafson
> > > > > > > > <ja...@confluent.io.invalid>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hey David,
> > > > > > > > > >
> > > > > > > > > > Thanks for the response.
> > > > > > > > > >
> > > > > > > > > > > Do you think that we could have a case where two
> members
> > > with
> > > > > the
> > > > > > > > same
> > > > > > > > > > static id are active?
> > > > > > > > > >
> > > > > > > > > > I was just thinking about the handoff when we spin up a
> new
> > > node
> > > > > > > > before the
> > > > > > > > > > old one completes shutdown. It's probably ok to fence
> the old
> > > > > member
> > > > > > > > > > abruptly. I do recall we had a few issues with this
> handoff
> > > in
> > > > > the
> > > > > > > > past. It
> > > > > > > > > > was due to the fact that the replaced node might have
> been
> > > the
> > > > > > > leader.
> > > > > > > > > > Since we've removed the leader notion, perhaps this kind
> of
> > > issue
> > > > > > > won't
> > > > > > > > > > resurface.
> > > > > > > > > >
> > > > > > > > > > In regard to the RE2 library. I understand the intent to
> pass
> > > > > this
> > > > > > > > through
> > > > > > > > > > as a string without any validation so that we don't need
> the
> > > > > > > > dependence on
> > > > > > > > > > the client. I do wonder if that will be restrictive in
> > > practice.
> > > > > For
> > > > > > > > > > example, if a regex changes on the client side, would we
> > > need to
> > > > > > > > validate
> > > > > > > > > > against the current assignment to see which partitions to
> > > > > continue
> > > > > > > > > > fetching? Or would we instead invalidate the complete
> current
> > > > > > > > assignment?
> > > > > > > > > > Perhaps this is a rare case and we can do the latter.
> There
> > > > > could be
> > > > > > > > more
> > > > > > > > > > situations though.
> > > > > > > > > >
> > > > > > > > > > As an alternative, I wonder if we could define a
> minimalistic
> > > > > regex
> > > > > > > > syntax
> > > > > > > > > > (a subset of RE2 say) that could map more easily to regex
> > > support
> > > > > > > that
> > > > > > > > > > already exists in clients? For example, in this
> scenario, we
> > > are
> > > > > > > > matching
> > > > > > > > > > topics, so we only need to be able to represent topic
> > > characters
> > > > > and
> > > > > > > > not
> > > > > > > > > > the whole smorgasbord of whitespace and unicode options.
> > > Would it
> > > > > > > make
> > > > > > > > > > sense to have a reduced syntax which includes only
> standard
> > > topic
> > > > > > > > > > characters, wildcards, character classes, and
> repetitions?
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Jason
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Sep 22, 2022 at 9:02 AM Guozhang Wang <
> > > > > wangguoz@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi David, thanks for all the detailed explanations. I
> think
> > > > > they
> > > > > > > all
> > > > > > > > make
> > > > > > > > > > > sense. Just want to have a couple follow-ups here:
> > > > > > > > > > >
> > > > > > > > > > > > I don't really see the benefits here because server
> side
> > > > > > > assignors
> > > > > > > > > > > don't have metadata at all. They only assign
> > > topic-partitions.
> > > > > They
> > > > > > > > > > > are not supposed to generate metadata nor to receive
> > > metadata
> > > > > from
> > > > > > > > the
> > > > > > > > > > > members.
> > > > > > > > > > >
> > > > > > > > > > > Just to clarify I was asking about the `version` of the
> > > > > assignor
> > > > > > > > (i.e. up
> > > > > > > > > > > to what version that the client would support), and I
> do
> > > agree
> > > > > we
> > > > > > > > would
> > > > > > > > > > not
> > > > > > > > > > > need metadata. What I have in mind is that, for some
> > > specific
> > > > > > > > built-in
> > > > > > > > > > > broker-assignors, e.g. rack-aware assignors, if it's
> > > possible
> > > > > that
> > > > > > > > in a
> > > > > > > > > > > newer version we would have a hierarchical rack ID
> string
> > > > > format,
> > > > > > > > like
> > > > > > > > > > > "tier1-tier2" etc, but if some client has not upgraded
> > > their
> > > > > rack
> > > > > > > ID
> > > > > > > > > > > would still be in old format. In this case, the broker
> then
> > > > > needs
> > > > > > > to
> > > > > > > > > > choose
> > > > > > > > > > > the old versioned assignor. I'm probably making
> something
> > > up
> > > > > here
> > > > > > > for
> > > > > > > > > > rack
> > > > > > > > > > > aware assignors, but I'm wondering if in general such
> an
> > > > > > > > "auto-downgrade"
> > > > > > > > > > > behavior would be needed still for broker-side
> assignor,
> > > and
> > > > > if yes
> > > > > > > > would
> > > > > > > > > > > "version" still be useful.
> > > > > > > > > > >
> > > > > > > > > > > > Yeah, that's right. Within a rebalance,
> `onAssignment` is
> > > > > called
> > > > > > > > once
> > > > > > > > > > > when the member transitions to a new epoch. This one
> > > contains
> > > > > the
> > > > > > > > full
> > > > > > > > > > > metadata provided by the client side assignor. Then,
> > > > > `onAssignment`
> > > > > > > > > > > can be called max N times where N is the number of
> > > partitions
> > > > > > > pending
> > > > > > > > > > > revocation by other members. Let me try to clarify
> this in
> > > the
> > > > > KIP.
> > > > > > > > > > >
> > > > > > > > > > > Okay, my understanding is that the calling ordering of
> > > these
> > > > > > > > callbacks
> > > > > > > > > > > would be like the following:
> > > > > > > > > > >
> > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > onPartitionsRevoked(); // just once, since we do not
> > > really
> > > > > need
> > > > > > > > > > > to revoke incrementally.
> > > > > > > > > > >
> > > > > > > > > > > onAssignment(); // the first call, with epoch
> > > incremented
> > > > > > > > > > > onPartitionsAssigned(); // paired with the
> onAssignment
> > > > > > > > > > >
> > > > > > > > > > > onAssignment(); // the first onAssignment
> > > would
> > > > > bump
> > > > > > > up
> > > > > > > > the
> > > > > > > > > > > epoch, and the metadata reflected.
> > > > > > > > > > > onPartitionsAssigned(); // each time we get an
> additional
> > > > > > > > assignment,
> > > > > > > > > > we
> > > > > > > > > > > call onAssignment and then paired with an
> > > onPartitionsAssigned
> > > > > > > > > > > ...
> > > > > > > > > > > onAssignment();
> > > > > > > > > > > onPartitionsAssigned(); // on each of the
> onAssignment
> > > > > calls, the
> > > > > > > > > > encoded
> > > > > > > > > > > metadata would not change, only the incrementally added
> > > > > partitions
> > > > > > > be
> > > > > > > > > > > reflected
> > > > > > > > > > >
> > > > > > > > > > > Is that the case?
> > > > > > > > > > >
> > > > > > > > > > > I'm wondering if we would still call onAssignment just
> > > once,
> > > > > that
> > > > > > > > encodes
> > > > > > > > > > > all the assignment for this rebalance, including all
> the
> > > > > partitions
> > > > > > > > that
> > > > > > > > > > > should be assigned to the member but not yet assigned
> since
> > > > > they
> > > > > > > > have not
> > > > > > > > > > > been revoked by others. In that case the call ordering
> > > would
> > > > > be:
> > > > > > > > > > >
> > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > onPartitionsRevoked(); // just once
> > > > > > > > > > > onAssignment(); // just once, with epoch
> incremented,
> > > and
> > > > > > > metadata
> > > > > > > > > > > encoded changed, the "assignment" field also reflect
> the
> > > final
> > > > > > > target
> > > > > > > > > > > assignment
> > > > > > > > > > > onPartitionsAssigned(); // multiple times, which
> > > represent
> > > > > > > > > > incrementally
> > > > > > > > > > > added partitions
> > > > > > > > > > > ...
> > > > > > > > > > > onPartitionsAssigned();
> > > > > > > > > > >
> > > > > > > > > > > The motivation from this is that, most users would only
> > > > > implement
> > > > > > > the
> > > > > > > > > > > rebalance callback listeners and hence we'd definitely
> > > need to
> > > > > make
> > > > > > > > sure
> > > > > > > > > > > the semantics of that does not change much, and the
> time
> > > > > > > > > > > `onPartitionsAssigned` indicate the time when the
> > > partitions
> > > > > are
> > > > > > > > actually
> > > > > > > > > > > assigned to it; while for assignors, the
> `onAssignment` is
> > > > > used to
> > > > > > > > > > indicate
> > > > > > > > > > > what decision is made regarding for this member, i.e.
> when
> > > the
> > > > > > > > partitions
> > > > > > > > > > > are decided to be given to it, but not necessarily
> meant
> > > that
> > > > > it
> > > > > > > has
> > > > > > > > been
> > > > > > > > > > > given, since that time should be determined by the
> time of
> > > > > > > > > > > `onPartitionsAssigned`. The benefits though, would be
> that
> > > > > assignor
> > > > > > > > > > > implementers would not need to reason which
> `onAssignment`
> > > > > would be
> > > > > > > > the
> > > > > > > > > > > last one for this epoch.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Guozhang
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Guozhang
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Sep 22, 2022 at 2:20 AM David Jacot
> > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Guozhang,
> > > > > > > > > > > >
> > > > > > > > > > > > > 1) The client-side "PartitionAssignor#Assignment"
> has
> > > an
> > > > > Error
> > > > > > > > byte
> > > > > > > > > > > > field,
> > > > > > > > > > > > > while the broker-side
> "PartitionAssignor#Assignment"
> > > does
> > > > > not.
> > > > > > > > And
> > > > > > > > > > the
> > > > > > > > > > > > > rationale seems to be assuming that we should
> always be
> > > > > able to
> > > > > > > > do
> > > > > > > > > > the
> > > > > > > > > > > > > assignment at the broker-side assignor without
> errors.
> > > > > > > > Personally I
> > > > > > > > > > > think
> > > > > > > > > > > > > it's still potentially beneficial to add the Error
> > > field
> > > > > even
> > > > > > > for
> > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> where
> > > some
> > > > > > > > subscribed
> > > > > > > > > > > > > topics are not recognized with the current broker's
> > > > > metadata.
> > > > > > > > What do
> > > > > > > > > > > you
> > > > > > > > > > > > > think?
> > > > > > > > > > > >
> > > > > > > > > > > > Yeah, that seems reasonable. However, I wonder if it
> > > would be
> > > > > > > > better
> > > > > > > > > > > > to use an exception on the server side. This is what
> we
> > > > > usually
> > > > > > > do
> > > > > > > > for
> > > > > > > > > > > > server side plugins. On the client side, we use a
> field
> > > > > because
> > > > > > > the
> > > > > > > > > > > > errors are not defined in advance.
> > > > > > > > > > > >
> > > > > > > > > > > > Your comment also makes me think about what we
> should do
> > > > > when the
> > > > > > > > > > > > server side assignor fails. I suppose that we have to
> > > keep
> > > > > the
> > > > > > > > current
> > > > > > > > > > > > assignment until a new event occurs. For instance, in
> > > your
> > > > > > > example,
> > > > > > > > > > > > the coordinator would have to trigger a rebalance
> when
> > > > > > > unrecognized
> > > > > > > > > > > > topics are available. This would be part of the
> metadata
> > > > > > > > monitoring.
> > > > > > > > > > > >
> > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> additional
> > > > > fields
> > > > > > > > > > > > > reason/version/metadata compared with the
> broker-side
> > > > > > > > GroupMember. I
> > > > > > > > > > > > agree
> > > > > > > > > > > > > that broker-side assignor would not need
> > > reason/metadata
> > > > > since
> > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > > > > blackbox strings/bytes to the assignor, but what
> about
> > > > > version?
> > > > > > > > E.g.
> > > > > > > > > > is
> > > > > > > > > > > > it
> > > > > > > > > > > > > possible that we evolve our broker-side built-in
> > > assignor
> > > > > but
> > > > > > > > the old
> > > > > > > > > > > > > versioned clients would not be able to work with
> the
> > > new
> > > > > > > > version, in
> > > > > > > > > > > > which
> > > > > > > > > > > > > case we need to let the broker being aware of this
> and
> > > > > upgrade
> > > > > > > > its
> > > > > > > > > > > > behavior
> > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > >
> > > > > > > > > > > > I don't really see the benefits here because server
> side
> > > > > > > assignors
> > > > > > > > > > > > don't have metadata at all. They only assign
> > > > > topic-partitions.
> > > > > > > They
> > > > > > > > > > > > are not supposed to generate metadata nor to receive
> > > metadata
> > > > > > > from
> > > > > > > > the
> > > > > > > > > > > > members.
> > > > > > > > > > > >
> > > > > > > > > > > > > 3) Also related to 2) above, for the client-side
> > > > > "GroupMember",
> > > > > > > > > > instead
> > > > > > > > > > > > of
> > > > > > > > > > > > > including these three fields, what about just
> adding
> > > the
> > > > > > > > "Metadata"
> > > > > > > > > > > field
> > > > > > > > > > > > > class which has these three fields? Also, there
> are two
> > > > > > > > "Metadata"
> > > > > > > > > > > > > currently in the APIs, the first is a class that
> > > encodes
> > > > > > > > > > > > > reason/version/metadata, and the second is just the
> > > encoded
> > > > > > > > metadata
> > > > > > > > > > > > bytes.
> > > > > > > > > > > > > I'm wondering what about just naming the first as
> > > > > > > memberMetadata,
> > > > > > > > > > which
> > > > > > > > > > > > > then has a bytebuffer field Metadata, or instead
> > > naming the
> > > > > > > > second
> > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > >
> > > > > > > > > > > > That's a good point. Let me try to rationalize this
> > > interface
> > > > > > > > based on
> > > > > > > > > > > > your suggestions.
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > David
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Sep 13, 2022 at 9:21 PM Guozhang Wang <
> > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Just had a few more nit questions about the public
> > > APIs:
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1) The client-side "PartitionAssignor#Assignment"
> has
> > > an
> > > > > Error
> > > > > > > > byte
> > > > > > > > > > > > field,
> > > > > > > > > > > > > while the broker-side
> "PartitionAssignor#Assignment"
> > > does
> > > > > not.
> > > > > > > > And
> > > > > > > > > > the
> > > > > > > > > > > > > rationale seems to be assuming that we should
> always be
> > > > > able to
> > > > > > > > do
> > > > > > > > > > the
> > > > > > > > > > > > > assignment at the broker-side assignor without
> errors.
> > > > > > > > Personally I
> > > > > > > > > > > think
> > > > > > > > > > > > > it's still potentially beneficial to add the Error
> > > field
> > > > > even
> > > > > > > for
> > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> where
> > > some
> > > > > > > > subscribed
> > > > > > > > > > > > > topics are not recognized with the current broker's
> > > > > metadata.
> > > > > > > > What do
> > > > > > > > > > > you
> > > > > > > > > > > > > think?
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> additional
> > > > > fields
> > > > > > > > > > > > > reason/version/metadata compared with the
> broker-side
> > > > > > > > GroupMember. I
> > > > > > > > > > > > agree
> > > > > > > > > > > > > that broker-side assignor would not need
> > > reason/metadata
> > > > > since
> > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > > > > blackbox strings/bytes to the assignor, but what
> about
> > > > > version?
> > > > > > > > E.g.
> > > > > > > > > > is
> > > > > > > > > > > > it
> > > > > > > > > > > > > possible that we evolve our broker-side built-in
> > > assignor
> > > > > but
> > > > > > > > the old
> > > > > > > > > > > > > versioned clients would not be able to work with
> the
> > > new
> > > > > > > > version, in
> > > > > > > > > > > > which
> > > > > > > > > > > > > case we need to let the broker being aware of this
> and
> > > > > upgrade
> > > > > > > > its
> > > > > > > > > > > > behavior
> > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > > >
> > > > > > > > > > > > > 3) Also related to 2) above, for the client-side
> > > > > "GroupMember",
> > > > > > > > > > instead
> > > > > > > > > > > > of
> > > > > > > > > > > > > including these three fields, what about just
> adding
> > > the
> > > > > > > > "Metadata"
> > > > > > > > > > > field
> > > > > > > > > > > > > class which has these three fields? Also, there
> are two
> > > > > > > > "Metadata"
> > > > > > > > > > > > > currently in the APIs, the first is a class that
> > > encodes
> > > > > > > > > > > > > reason/version/metadata, and the second is just the
> > > encoded
> > > > > > > > metadata
> > > > > > > > > > > > bytes.
> > > > > > > > > > > > > I'm wondering what about just naming the first as
> > > > > > > memberMetadata,
> > > > > > > > > > which
> > > > > > > > > > > > > then has a bytebuffer field Metadata, or instead
> > > naming the
> > > > > > > > second
> > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Sep 13, 2022 at 12:08 PM Guozhang Wang <
> > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks for bringing this question up. I think the
> > > main
> > > > > > > > benefits as
> > > > > > > > > > > you
> > > > > > > > > > > > > > listed is 2) above if it stays; just to clarify,
> we
> > > would
> > > > > > > only
> > > > > > > > be
> > > > > > > > > > > able
> > > > > > > > > > > > to
> > > > > > > > > > > > > > save one round trip if the rebalance is still
> > > triggered
> > > > > by
> > > > > > > the
> > > > > > > > > > > broker;
> > > > > > > > > > > > if
> > > > > > > > > > > > > > the rebalance is triggered by the client then the
> > > > > > > > num.round.trips
> > > > > > > > > > are
> > > > > > > > > > > > the
> > > > > > > > > > > > > > same:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1) With GroupPrepareAssignment:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > T0: client decides to do a new assignment,
> suppose
> > > it has
> > > > > > > > already
> > > > > > > > > > > sent
> > > > > > > > > > > > a
> > > > > > > > > > > > > > HB and hence has to wait for it to return first
> since
> > > > > only
> > > > > > > one
> > > > > > > > > > > request
> > > > > > > > > > > > /
> > > > > > > > > > > > > > response can be inflight with the coordinator's
> > > socket.
> > > > > > > > > > > > > > T1: client receives the HB response, and then
> sends
> > > the
> > > > > > > > > > > > > > GroupPrepareAssignment request.
> > > > > > > > > > > > > > T2: the GroupPrepareAssignment response is
> returned.
> > > > > > > > > > > > > > T3: it calculates the new assignment, and sends a
> > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > request.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In total, two round trips.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2) Without GroupPrepareAssignment:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > T0: client decides to do a new assignment,
> suppose
> > > it has
> > > > > > > > already
> > > > > > > > > > > sent
> > > > > > > > > > > > a
> > > > > > > > > > > > > > HB and hence has to wait for it to return first
> since
> > > > > only
> > > > > > > one
> > > > > > > > > > > request
> > > > > > > > > > > > /
> > > > > > > > > > > > > > response can be inflight with the coordinator's
> > > socket.
> > > > > > > > > > > > > > T1: client receives the HB response, and then
> sends
> > > the
> > > > > new
> > > > > > > HB
> > > > > > > > > > > request
> > > > > > > > > > > > > > with the flag indicating a new rebalance needed..
> > > > > > > > > > > > > > T2: the HB response with the optional member
> metadata
> > > > > map is
> > > > > > > > > > > returned.
> > > > > > > > > > > > > > T3: it calculates the new assignment, and sends a
> > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > request.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In total, two round trips as well.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -----------------------------
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So to complete the full picture here, we'd need
> to
> > > modify
> > > > > > > both
> > > > > > > > HB
> > > > > > > > > > > > request
> > > > > > > > > > > > > > and response so that the client can also
> indicate a
> > > new
> > > > > > > > rebalance
> > > > > > > > > > via
> > > > > > > > > > > > the
> > > > > > > > > > > > > > HB request as well, right?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Assuming all above is true, I think it's okay to
> > > merge
> > > > > the
> > > > > > > > > > > > > > GroupPrepareAssignment into HB given that we can
> > > make the
> > > > > > > > > > additional
> > > > > > > > > > > > fields
> > > > > > > > > > > > > > encoding the full member (subscription) metadata
> and
> > > > > topic
> > > > > > > > metadata
> > > > > > > > > > > as
> > > > > > > > > > > > > > optional fields.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Mon, Sep 12, 2022 at 5:22 AM David Jacot
> > > > > > > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Hi all,
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> During an offline conversation, someone asked
> why we
> > > > > need
> > > > > > > the
> > > > > > > > > > > > > >> ConsumerGroupPrepareAssignment API and suggested
> > > that we
> > > > > > > could
> > > > > > > > > > > instead
> > > > > > > > > > > > > >> provide the group state in the heartbeat
> response.
> > > This
> > > > > has
> > > > > > > a
> > > > > > > > few
> > > > > > > > > > > > > >> advantages: 1) it does not require using a
> special
> > > error
> > > > > > > code
> > > > > > > > to
> > > > > > > > > > > > > >> signal that a new assignment is required as the
> > > signal
> > > > > would
> > > > > > > > be
> > > > > > > > > > the
> > > > > > > > > > > > > >> provided group state; 2) it removes one round
> trip
> > > when
> > > > > a
> > > > > > > > client
> > > > > > > > > > > side
> > > > > > > > > > > > > >> assignor is used. The downside is that it makes
> the
> > > > > > > heartbeat
> > > > > > > > > > > > > >> response's definition quite large. I recall
> that I
> > > went
> > > > > with
> > > > > > > > the
> > > > > > > > > > > > > >> current approach due to this.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Providing the group state in the heartbeat
> response
> > > is
> > > > > > > > appealing.
> > > > > > > > > > > What
> > > > > > > > > > > > > >> do you guys think?
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Best,
> > > > > > > > > > > > > >> David
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On Mon, Sep 12, 2022 at 2:17 PM David Jacot <
> > > > > > > > djacot@confluent.io>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > Hi Guozhang,
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > 1. I have added a reference to the relevant
> > > chapter
> > > > > > > instead
> > > > > > > > of
> > > > > > > > > > > > > >> > repeating the whole thing. Does that work for
> you?
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > 2. The "Rebalance Triggers" section you are
> > > referring
> > > > > to
> > > > > > > is
> > > > > > > > > > about
> > > > > > > > > > > > when
> > > > > > > > > > > > > >> > a rebalance should be triggered for the
> > > non-upgraded
> > > > > > > members
> > > > > > > > > > using
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> > old protocol. The section mentions that a
> > > rebalance
> > > > > must
> > > > > > > be
> > > > > > > > > > > > triggered
> > > > > > > > > > > > > >> > when a new assignment is installed. This
> implies
> > > that
> > > > > the
> > > > > > > > group
> > > > > > > > > > > > epoch
> > > > > > > > > > > > > >> > was updated either by a native member or a
> > > > > non-upgraded
> > > > > > > > member.
> > > > > > > > > > > For
> > > > > > > > > > > > > >> > the latter, the JoinGroup request would be the
> > > > > trigger. I
> > > > > > > > have
> > > > > > > > > > > > added a
> > > > > > > > > > > > > >> > reference to the relevant chapter in the
> > > "JoinGroup
> > > > > > > > Handling"
> > > > > > > > > > > > section
> > > > > > > > > > > > > >> > as well. Does that make sense?
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > Thanks,
> > > > > > > > > > > > > >> > David
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > On Fri, Sep 9, 2022 at 10:35 PM Guozhang Wang
> <
> > > > > > > > > > wangguoz@gmail.com
> > > > > > > > > > > >
> > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > Hello David,
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > Alright I think that's sufficient. Just to
> make
> > > that
> > > > > > > > clear in
> > > > > > > > > > > the
> > > > > > > > > > > > doc,
> > > > > > > > > > > > > >> > > could we update:
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > 1) the heartbeat request handling section,
> > > stating
> > > > > when
> > > > > > > > > > > > coordinator
> > > > > > > > > > > > > >> will
> > > > > > > > > > > > > >> > > trigger rebalance based on the HB's member
> > > metadata
> > > > > /
> > > > > > > > reason?
> > > > > > > > > > > > > >> > > 2) the "Rebalance Triggers" section to
> include
> > > what
> > > > > we
> > > > > > > > > > described
> > > > > > > > > > > > in
> > > > > > > > > > > > > >> "Group
> > > > > > > > > > > > > >> > > Epoch - Trigger a rebalance" section as
> well?
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > Guozhang
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > On Fri, Sep 9, 2022 at 1:28 AM David Jacot
> > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > Hi Guozhang,
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > I thought that the assignor will always be
> > > > > consulted
> > > > > > > > when
> > > > > > > > > > the
> > > > > > > > > > > > next
> > > > > > > > > > > > > >> > > > heartbeat request is constructed. In other
> > > words,
> > > > > > > > > > > > > >> > > > `PartitionAssignor#metadata` will be
> called
> > > for
> > > > > every
> > > > > > > > > > > heartbeat.
> > > > > > > > > > > > > >> This
> > > > > > > > > > > > > >> > > > gives the opportunity for the assignor to
> > > enforce
> > > > > a
> > > > > > > > > > rebalance
> > > > > > > > > > > by
> > > > > > > > > > > > > >> > > > setting the reason to a non-zero value or
> by
> > > > > changing
> > > > > > > > the
> > > > > > > > > > > > bytes. Do
> > > > > > > > > > > > > >> > > > you think that this is not sufficient?
> Are you
> > > > > > > > concerned by
> > > > > > > > > > > the
> > > > > > > > > > > > > >> delay?
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > Best,
> > > > > > > > > > > > > >> > > > David
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > On Fri, Sep 9, 2022 at 7:10 AM Guozhang
> Wang <
> > > > > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > Hello David,
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > One of Jun's comments make me thinking:
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > >> > > > > In this case, a new assignment is
> triggered
> > > by
> > > > > the
> > > > > > > > client
> > > > > > > > > > > side
> > > > > > > > > > > > > >> > > > > assignor. When constructing the HB, the
> > > consumer
> > > > > > > will
> > > > > > > > > > always
> > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > >> > > > > the client side assignor and propagate
> the
> > > > > > > > information to
> > > > > > > > > > > the
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > coordinator. In other words, we don't
> expect
> > > > > users
> > > > > > > to
> > > > > > > > call
> > > > > > > > > > > > > >> > > > > Consumer#enforceRebalance anymore.
> > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > As I looked at the current
> > > PartitionAssignor's
> > > > > > > > interface,
> > > > > > > > > > we
> > > > > > > > > > > > > >> actually do
> > > > > > > > > > > > > >> > > > > not have a way yet to instruct how to
> > > construct
> > > > > the
> > > > > > > > next
> > > > > > > > > > HB
> > > > > > > > > > > > > >> request, e.g.
> > > > > > > > > > > > > >> > > > > when the assignor wants to enforce a new
> > > > > rebalance
> > > > > > > > with a
> > > > > > > > > > > new
> > > > > > > > > > > > > >> assignment,
> > > > > > > > > > > > > >> > > > > we'd need some customizable APIs inside
> the
> > > > > > > > > > > PartitionAssignor
> > > > > > > > > > > > to
> > > > > > > > > > > > > >> indicate
> > > > > > > > > > > > > >> > > > > the next HB telling broker about so.
> WDYT
> > > about
> > > > > > > adding
> > > > > > > > > > such
> > > > > > > > > > > an
> > > > > > > > > > > > > >> API on the
> > > > > > > > > > > > > >> > > > > PartitionAssignor?
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > Guozhang
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > On Tue, Sep 6, 2022 at 6:09 AM David
> Jacot
> > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > Hi Jun,
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > I have updated the KIP to include your
> > > > > feedback. I
> > > > > > > > have
> > > > > > > > > > > also
> > > > > > > > > > > > > >> tried to
> > > > > > > > > > > > > >> > > > > > clarify the parts which were not
> cleared.
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > Best,
> > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > On Fri, Sep 2, 2022 at 4:18 PM David
> > > Jacot <
> > > > > > > > > > > > djacot@confluent.io
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Thanks for your feedback. Let me
> start
> > > by
> > > > > > > > answering
> > > > > > > > > > your
> > > > > > > > > > > > > >> questions
> > > > > > > > > > > > > >> > > > > > > inline and I will update the KIP
> next
> > > week.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall, the
> main
> > > > > benefits
> > > > > > > > of
> > > > > > > > > > the
> > > > > > > > > > > > KIP
> > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > efficient
> > > > > > > > support of
> > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > I would also add that the KIP
> removes
> > > the
> > > > > global
> > > > > > > > sync
> > > > > > > > > > > > barrier
> > > > > > > > > > > > > >> in the
> > > > > > > > > > > > > >> > > > > > > protocol which is essential to
> improve
> > > group
> > > > > > > > stability
> > > > > > > > > > > and
> > > > > > > > > > > > > >> > > > > > > scalability, and the KIP also
> > > simplifies the
> > > > > > > > client by
> > > > > > > > > > > > moving
> > > > > > > > > > > > > >> most of
> > > > > > > > > > > > > >> > > > > > > the logic to the server side.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 30. ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> singleton.
> > > Do we
> > > > > plan
> > > > > > > > to
> > > > > > > > > > > > support
> > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > >> > > > > > > > of the partition assignor in the
> > > > > consumers?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Definitely. The group coordinator
> will
> > > use
> > > > > the
> > > > > > > > > > assignor
> > > > > > > > > > > > used
> > > > > > > > > > > > > >> by a
> > > > > > > > > > > > > >> > > > > > > majority of the members. This
> allows the
> > > > > group
> > > > > > > to
> > > > > > > > move
> > > > > > > > > > > > from
> > > > > > > > > > > > > >> one
> > > > > > > > > > > > > >> > > > > > > assignor to another by a roll. This
> is
> > > > > explained
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > > > >> Assignor
> > > > > > > > > > > > > >> > > > > > > Selection chapter.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> explain
> > > > > whether
> > > > > > > > it's
> > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > >> > > > > > > > request or the scenarios when it
> > > needs to
> > > > > be
> > > > > > > > filled?
> > > > > > > > > > > For
> > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > >> > > > > > > > clear to me when TopicPartitions
> > > needs to
> > > > > be
> > > > > > > > filled.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > The client is expected to set those
> > > fields
> > > > > in
> > > > > > > > case of
> > > > > > > > > > a
> > > > > > > > > > > > > >> connection
> > > > > > > > > > > > > >> > > > > > > issue (e.g. timeout) or when the
> fields
> > > have
> > > > > > > > changed
> > > > > > > > > > > since
> > > > > > > > > > > > > >> the last
> > > > > > > > > > > > > >> > > > > > > HB. The server populates those
> fields as
> > > > > long as
> > > > > > > > the
> > > > > > > > > > > > member
> > > > > > > > > > > > > >> is not
> > > > > > > > > > > > > >> > > > > > > fully reconciled - the member should
> > > > > acknowledge
> > > > > > > > that
> > > > > > > > > > it
> > > > > > > > > > > > has
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > expected epoch and assignment. I
> will
> > > > > clarify
> > > > > > > > this in
> > > > > > > > > > > the
> > > > > > > > > > > > KIP.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> protocol,
> > > the
> > > > > rack
> > > > > > > > > > > affinity
> > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > >> > > > > > > > and the broker is only considered
> > > during
> > > > > > > > fetching,
> > > > > > > > > > but
> > > > > > > > > > > > not
> > > > > > > > > > > > > >> during
> > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> Sometimes,
> > > once
> > > > > the
> > > > > > > > > > > assignment
> > > > > > > > > > > > is
> > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > >> > > > > > > > no opportunity for read affinity
> > > because
> > > > > no
> > > > > > > > replicas
> > > > > > > > > > > of
> > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > wondering
> > > > > if we
> > > > > > > > should
> > > > > > > > > > > use
> > > > > > > > > > > > > >> this
> > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > >> > > > > > > > to address this by including rack
> in
> > > > > > > > GroupMember.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > That's an interesting idea. I don't
> see
> > > any
> > > > > > > issue
> > > > > > > > with
> > > > > > > > > > > > adding
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > rack
> > > > > > > > > > > > > >> > > > > > > to the members. I will do so.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> it's
> > > > > useful to
> > > > > > > > know
> > > > > > > > > > how
> > > > > > > > > > > > busy
> > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> event
> > > loop
> > > > > > > model,
> > > > > > > > it
> > > > > > > > > > > seems
> > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > >> > > > > > > > a metric that tracks the fraction
> of
> > > the
> > > > > time
> > > > > > > > the
> > > > > > > > > > > event
> > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > That's a great idea. I will add it.
> > > Thanks.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > coordinator
> > > > > > > > failover
> > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > >> > > > > > > > does it need to trigger the check
> if
> > > any
> > > > > group
> > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> matching
> > > topic?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Sure. When the new group coordinator
> > > takes
> > > > > over,
> > > > > > > > it
> > > > > > > > > > has
> > > > > > > > > > > > to:
> > > > > > > > > > > > > >> > > > > > > * Setup the session timeouts.
> > > > > > > > > > > > > >> > > > > > > * Trigger a new assignment if a
> client
> > > side
> > > > > > > > assignor
> > > > > > > > > > is
> > > > > > > > > > > > used.
> > > > > > > > > > > > > >> We
> > > > > > > > > > > > > >> > > > don't
> > > > > > > > > > > > > >> > > > > > > store the information about the
> member
> > > > > selected
> > > > > > > > to run
> > > > > > > > > > > the
> > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > >> > > > > > > so we have to start a new one.
> > > > > > > > > > > > > >> > > > > > > * Update the topics metadata,
> verify the
> > > > > > > wildcard
> > > > > > > > > > > > > >> subscriptions, and
> > > > > > > > > > > > > >> > > > > > > trigger a rebalance if needed.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 34. ConsumerGroupMetadataValue,
> > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > >> > > > > > > > ConsumerGroupMemberMetadataValue:
> > > Could we
> > > > > > > > document
> > > > > > > > > > > what
> > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > >> > > > > > > > reflects? For example, does the
> epoch
> > > in
> > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> about the
> > > > > one in
> > > > > > > > > > > > > >> > > > > > > >
> ConsumerGroupPartitionMetadataValue
> > > and
> > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Sure. I will clarify that but it is
> > > always
> > > > > the
> > > > > > > > latest
> > > > > > > > > > > > group
> > > > > > > > > > > > > >> epoch.
> > > > > > > > > > > > > >> > > > > > > When the group state is updated, the
> > > group
> > > > > epoch
> > > > > > > > is
> > > > > > > > > > > > bumped so
> > > > > > > > > > > > > >> we use
> > > > > > > > > > > > > >> > > > > > > that one for all the change records
> > > related
> > > > > to
> > > > > > > the
> > > > > > > > > > > update.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator will
> ensure
> > > > > that
> > > > > > > the
> > > > > > > > > > > > following
> > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > >> > > > > > > > met: ... All members exists." It's
> > > > > possible
> > > > > > > for
> > > > > > > > a
> > > > > > > > > > > member
> > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > That's right. Here I meant that the
> > > members
> > > > > > > > provided
> > > > > > > > > > by
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > >> > > > > > > in the assignment must exist in the
> > > group.
> > > > > The
> > > > > > > > > > assignor
> > > > > > > > > > > > can
> > > > > > > > > > > > > >> not make
> > > > > > > > > > > > > >> > > > > > > up new member ids.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> with a
> > > > > member
> > > > > > > > epoch
> > > > > > > > > > > > equals to
> > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what member
> id
> > > > > would be
> > > > > > > > used?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > A member is expected to abandon all
> its
> > > > > > > > partitions and
> > > > > > > > > > > > > >> rejoins when
> > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > >> > > > > > > receives the FENCED_MEMBER_EPOCH
> error.
> > > In
> > > > > this
> > > > > > > > case,
> > > > > > > > > > > the
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > > coordinator will have removed the
> member
> > > > > from
> > > > > > > the
> > > > > > > > > > group.
> > > > > > > > > > > > The
> > > > > > > > > > > > > >> member
> > > > > > > > > > > > > >> > > > > > > can rejoin the group with the same
> > > member
> > > > > id but
> > > > > > > > with
> > > > > > > > > > 0
> > > > > > > > > > > as
> > > > > > > > > > > > > >> epoch. Let
> > > > > > > > > > > > > >> > > > > > > me see if I can clarify this in the
> KIP.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> have
> > > the
> > > > > > > ability
> > > > > > > > to
> > > > > > > > > > > > trigger a
> > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> reason
> > > or
> > > > > by
> > > > > > > > updating
> > > > > > > > > > > the
> > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to be
> > > > > conflicting
> > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > In this case, a new assignment is
> > > triggered
> > > > > by
> > > > > > > the
> > > > > > > > > > > client
> > > > > > > > > > > > side
> > > > > > > > > > > > > >> > > > > > > assignor. When constructing the HB,
> the
> > > > > consumer
> > > > > > > > will
> > > > > > > > > > > > always
> > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > >> > > > > > > the client side assignor and
> propagate
> > > the
> > > > > > > > information
> > > > > > > > > > > to
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > > coordinator. In other words, we
> don't
> > > expect
> > > > > > > > users to
> > > > > > > > > > > call
> > > > > > > > > > > > > >> > > > > > > Consumer#enforceRebalance anymore.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples are
> > > nice.
> > > > > But
> > > > > > > the
> > > > > > > > > > > section
> > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > >> > > > > > > > 38.1 When the group transitions to
> > > epoch
> > > > > 2, B
> > > > > > > > > > > > immediately
> > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> which
> > > seems
> > > > > > > > > > incorrect.
> > > > > > > > > > > > > >> > > > > > > > 38.2 When the group transitions to
> > > epoch
> > > > > 3, C
> > > > > > > > seems
> > > > > > > > > > to
> > > > > > > > > > > > get
> > > > > > > > > > > > > >> into
> > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to epoch
> 3, C
> > > > > still
> > > > > > > > has A -
> > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Sorry for that! I will revise them.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of consumers:
> Do
> > > we
> > > > > > > support
> > > > > > > > the
> > > > > > > > > > > > upgrade
> > > > > > > > > > > > > >> from
> > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > We will support upgrading from the
> > > consumer
> > > > > > > > protocol
> > > > > > > > > > > > version
> > > > > > > > > > > > > >> 3,
> > > > > > > > > > > > > >> > > > > > > introduced in KIP-792. KIP-792 is
> not
> > > > > > > implemented
> > > > > > > > yet
> > > > > > > > > > so
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> earliest
> > > > > > > > > > > > > >> > > > > > > version is unknown at the moment.
> This
> > > is
> > > > > > > > explained in
> > > > > > > > > > > the
> > > > > > > > > > > > > >> migration
> > > > > > > > > > > > > >> > > > > > > plan chapter.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Thanks again for your feedback,
> Jun. I
> > > will
> > > > > > > > update the
> > > > > > > > > > > KIP
> > > > > > > > > > > > > >> based on
> > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > >> > > > > > > next week.
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > Best,
> > > > > > > > > > > > > >> > > > > > > David
> > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > On Thu, Sep 1, 2022 at 9:07 PM Jun
> Rao
> > > > > > > > > > > > > >> <ju...@confluent.io.invalid>
> > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > Hi, David,
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall, the
> main
> > > > > benefits
> > > > > > > > of
> > > > > > > > > > the
> > > > > > > > > > > > KIP
> > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > efficient
> > > > > > > > support of
> > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 30. ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> singleton.
> > > Do we
> > > > > plan
> > > > > > > > to
> > > > > > > > > > > > support
> > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > >> > > > > > > > of the partition assignor in the
> > > > > consumers?
> > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> explain
> > > > > whether
> > > > > > > > it's
> > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > >> > > > > > > > request or the scenarios when it
> > > needs to
> > > > > be
> > > > > > > > filled?
> > > > > > > > > > > For
> > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > >> > > > > > > > clear to me when TopicPartitions
> > > needs to
> > > > > be
> > > > > > > > filled.
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> protocol,
> > > the
> > > > > rack
> > > > > > > > > > > affinity
> > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > >> > > > > > > > and the broker is only considered
> > > during
> > > > > > > > fetching,
> > > > > > > > > > but
> > > > > > > > > > > > not
> > > > > > > > > > > > > >> during
> > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> Sometimes,
> > > once
> > > > > the
> > > > > > > > > > > assignment
> > > > > > > > > > > > is
> > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > >> > > > > > > > no opportunity for read affinity
> > > because
> > > > > no
> > > > > > > > replicas
> > > > > > > > > > > of
> > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > wondering
> > > > > if we
> > > > > > > > should
> > > > > > > > > > > use
> > > > > > > > > > > > > >> this
> > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > >> > > > > > > > to address this by including rack
> in
> > > > > > > > GroupMember.
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> it's
> > > > > useful to
> > > > > > > > know
> > > > > > > > > > how
> > > > > > > > > > > > busy
> > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> event
> > > loop
> > > > > > > model,
> > > > > > > > it
> > > > > > > > > > > seems
> > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > >> > > > > > > > a metric that tracks the fraction
> of
> > > the
> > > > > time
> > > > > > > > the
> > > > > > > > > > > event
> > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > coordinator
> > > > > > > > failover
> > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > >> > > > > > > > does it need to trigger the check
> if
> > > any
> > > > > group
> > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> matching
> > > topic?
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 34. ConsumerGroupMetadataValue,
> > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > >> > > > > > > > ConsumerGroupMemberMetadataValue:
> > > Could we
> > > > > > > > document
> > > > > > > > > > > what
> > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > >> > > > > > > > reflects? For example, does the
> epoch
> > > in
> > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> about the
> > > > > one in
> > > > > > > > > > > > > >> > > > > > > >
> ConsumerGroupPartitionMetadataValue
> > > and
> > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator will
> ensure
> > > > > that
> > > > > > > the
> > > > > > > > > > > > following
> > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > >> > > > > > > > met: ... All members exists." It's
> > > > > possible
> > > > > > > for
> > > > > > > > a
> > > > > > > > > > > member
> > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> with a
> > > > > member
> > > > > > > > epoch
> > > > > > > > > > > > equals to
> > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what member
> id
> > > > > would be
> > > > > > > > used?
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> have
> > > the
> > > > > > > ability
> > > > > > > > to
> > > > > > > > > > > > trigger a
> > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> reason
> > > or
> > > > > by
> > > > > > > > updating
> > > > > > > > > > > the
> > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to be
> > > > > conflicting
> > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples are
> > > nice.
> > > > > But
> > > > > > > the
> > > > > > > > > > > section
> > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > >> > > > > > > > 38.1 When the group transitions to
> > > epoch
> > > > > 2, B
> > > > > > > > > > > > immediately
> > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> which
> > > seems
> > > > > > > > > > incorrect.
> > > > > > > > > > > > > >> > > > > > > > 38.2 When the group transitions to
> > > epoch
> > > > > 3, C
> > > > > > > > seems
> > > > > > > > > > to
> > > > > > > > > > > > get
> > > > > > > > > > > > > >> into
> > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to epoch
> 3, C
> > > > > still
> > > > > > > > has A -
> > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of consumers:
> Do
> > > we
> > > > > > > support
> > > > > > > > the
> > > > > > > > > > > > upgrade
> > > > > > > > > > > > > >> from
> > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > Thanks,
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > Jun
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > On Mon, Aug 29, 2022 at 9:20 AM
> David
> > > > > Jacot
> > > > > > > > > > > > > >> > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > >> > > > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > Hi all,
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > The KIP states that we will
> > > > > re-implement the
> > > > > > > > > > > > coordinator
> > > > > > > > > > > > > >> in
> > > > > > > > > > > > > >> > > > Java. I
> > > > > > > > > > > > > >> > > > > > > > > discussed this offline with a
> few
> > > folks
> > > > > and
> > > > > > > > folks
> > > > > > > > > > > are
> > > > > > > > > > > > > >> concerned
> > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > >> > > > > > > > > we could introduce many
> regressions
> > > in
> > > > > the
> > > > > > > old
> > > > > > > > > > > > protocol
> > > > > > > > > > > > > >> if we do
> > > > > > > > > > > > > >> > > > so.
> > > > > > > > > > > > > >> > > > > > > > > Therefore, I am going to remove
> this
> > > > > > > statement
> > > > > > > > > > from
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> KIP. It
> > > > > > > > > > > > > >> > > > is an
> > > > > > > > > > > > > >> > > > > > > > > implementation detail after all
> so
> > > it
> > > > > does
> > > > > > > not
> > > > > > > > > > have
> > > > > > > > > > > > to be
> > > > > > > > > > > > > >> > > > decided at
> > > > > > > > > > > > > >> > > > > > > > > this stage. We will likely
> start by
> > > > > trying
> > > > > > > to
> > > > > > > > > > > > refactor the
> > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > >> > > > > > > > > implementation as a first step.
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > Cheers,
> > > > > > > > > > > > > >> > > > > > > > > David
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > On Mon, Aug 29, 2022 at 3:52 PM
> > > David
> > > > > Jacot
> > > > > > > <
> > > > > > > > > > > > > >> djacot@confluent.io
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > Hi Luke,
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> machine
> > > are:
> > > > > > > > "Empty,
> > > > > > > > > > > > assigning,
> > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in Consumer
> > > Group
> > > > > States
> > > > > > > > > > > section,
> > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > This sentence does not refer
> to
> > > those
> > > > > > > group
> > > > > > > > > > states
> > > > > > > > > > > > but
> > > > > > > > > > > > > >> rather
> > > > > > > > > > > > > >> > > > to a
> > > > > > > > > > > > > >> > > > > > > > > > state machine replication
> (SMR).
> > > This
> > > > > > > > refers to
> > > > > > > > > > > the
> > > > > > > > > > > > > >> entire
> > > > > > > > > > > > > >> > > > state of
> > > > > > > > > > > > > >> > > > > > > > > > group coordinator which is
> > > replicated
> > > > > via
> > > > > > > > the
> > > > > > > > > > log
> > > > > > > > > > > > > >> layer. I will
> > > > > > > > > > > > > >> > > > > > > > > > clarify this in the KIP.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean "each
> > > state
> > > > > > > machine
> > > > > > > > is
> > > > > > > > > > > > modelled
> > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > The idea is to follow a model
> > > similar
> > > > > to
> > > > > > > > the new
> > > > > > > > > > > > quorum
> > > > > > > > > > > > > >> > > > > > controller. We
> > > > > > > > > > > > > >> > > > > > > > > > will have N threads to process
> > > events.
> > > > > > > Each
> > > > > > > > > > > > > >> __consumer_offsets
> > > > > > > > > > > > > >> > > > > > > > > > partition is assigned to a
> unique
> > > > > thread
> > > > > > > > and all
> > > > > > > > > > > the
> > > > > > > > > > > > > >> events
> > > > > > > > > > > > > >> > > > (e.g.
> > > > > > > > > > > > > >> > > > > > > > > > requests, callbacks, etc.) are
> > > > > processed
> > > > > > > by
> > > > > > > > this
> > > > > > > > > > > > > >> thread. This
> > > > > > > > > > > > > >> > > > > > simplify
> > > > > > > > > > > > > >> > > > > > > > > > concurrency and will enable
> us to
> > > do
> > > > > > > > simulation
> > > > > > > > > > > > testing
> > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > >> > > > > > > > > > coordinator.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a state
> > > machine
> > > > > per
> > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > consumer
> > > > > group"
> > > > > > > > owned
> > > > > > > > > > > by
> > > > > > > > > > > > a
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > coordinator
> > > > > owns 2
> > > > > > > > > > > consumer
> > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> will we
> > > > > have 1
> > > > > > > > state
> > > > > > > > > > > > machine
> > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > See 1.1. The confusion comes
> from
> > > > > there, I
> > > > > > > > > > think.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > *is
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> number
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > >> > > > > > > > > > > to run the state machines.
> But
> > > I'm
> > > > > > > > wondering
> > > > > > > > > > if
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the state of
> > > each
> > > > > > > consumer
> > > > > > > > > > group
> > > > > > > > > > > > (or
> > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no heavy
> > > > > computation,
> > > > > > > > why
> > > > > > > > > > > > should we
> > > > > > > > > > > > > >> need
> > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > See 1.2. The idea is to have
> an
> > > > > ability to
> > > > > > > > shard
> > > > > > > > > > > the
> > > > > > > > > > > > > >> > > > processing as
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > computation could be heavy.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer session
> > > timeout,
> > > > > why
> > > > > > > > does
> > > > > > > > > > the
> > > > > > > > > > > > > >> default
> > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > >> > > > > > > > > > > locate between min (45s) and
> > > > > max(60s)? I
> > > > > > > > > > thought
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> lower/upper
> > > > > bound
> > > > > > > of
> > > > > > > > it,
> > > > > > > > > > > no?
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > group.consumer.session.timeout.ms
> > > > > int
> > > > > > > > 30s The
> > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> consumer
> > > > > group
> > > > > > > > > > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.min.session.timeout.ms
> > > > > > > > int 45s
> > > > > > > > > > > The
> > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.max.session.timeout.ms
> > > > > > > > int 60s
> > > > > > > > > > > The
> > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > This is indeed a mistake. The
> > > default
> > > > > > > > session
> > > > > > > > > > > > timeout
> > > > > > > > > > > > > >> should
> > > > > > > > > > > > > >> > > > be 45s
> > > > > > > > > > > > > >> > > > > > > > > > (the current default).
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server side
> > > > > assignor
> > > > > > > are
> > > > > > > > > > > [range,
> > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > assignor.
> > > > > I'd
> > > > > > > > like to
> > > > > > > > > > > > know
> > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > >> > > > > > > > > > > thought usually users will
> > > choose
> > > > > > > uniform
> > > > > > > > > > > assignor
> > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> distribution.
> > > Any
> > > > > > > other
> > > > > > > > > > reason
> > > > > > > > > > > > we
> > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> List
> > > range,
> > > > > > > > uniform
> > > > > > > > > > The
> > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > The order on the server side
> has
> > > no
> > > > > > > > influence
> > > > > > > > > > > > because
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > >> > > > > > must
> > > > > > > > > > > > > >> > > > > > > > > > chose the selector that he
> wants
> > > to
> > > > > use.
> > > > > > > > There
> > > > > > > > > > is
> > > > > > > > > > > no
> > > > > > > > > > > > > >> default
> > > > > > > > > > > > > >> > > > in the
> > > > > > > > > > > > > >> > > > > > > > > > current proposal. If the
> assignor
> > > is
> > > > > not
> > > > > > > > > > specified
> > > > > > > > > > > > by
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > client,
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > request is rejected. The
> default
> > > > > client
> > > > > > > > value
> > > > > > > > > > for
> > > > > > > > > > > > > >> > > > > > > > > > `group.remote.assignor` is
> > > `uniform`
> > > > > > > though.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > Thanks for your very good
> > > comments,
> > > > > Luke.
> > > > > > > I
> > > > > > > > hope
> > > > > > > > > > > > that my
> > > > > > > > > > > > > >> > > > answers
> > > > > > > > > > > > > >> > > > > > help
> > > > > > > > > > > > > >> > > > > > > > > > to clarify things. I will
> update
> > > the
> > > > > KIP
> > > > > > > as
> > > > > > > > well
> > > > > > > > > > > > based
> > > > > > > > > > > > > >> on your
> > > > > > > > > > > > > >> > > > > > > > > > feedback.
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > Cheers,
> > > > > > > > > > > > > >> > > > > > > > > > David
> > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > On Mon, Aug 22, 2022 at 9:29
> AM
> > > Luke
> > > > > Chen
> > > > > > > <
> > > > > > > > > > > > > >> showuon@gmail.com>
> > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > Hi David,
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > Thanks for the update.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > Some more questions:
> > > > > > > > > > > > > >> > > > > > > > > > > 1. In Group Coordinator
> > > section, you
> > > > > > > > > > mentioned:
> > > > > > > > > > > > > >> > > > > > > > > > > > The new group coordinator
> will
> > > > > have a
> > > > > > > > state
> > > > > > > > > > > > machine
> > > > > > > > > > > > > >> per
> > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets*
> partitions,
> > > > > where
> > > > > > > > each
> > > > > > > > > > > state
> > > > > > > > > > > > > >> machine is
> > > > > > > > > > > > > >> > > > > > modelled
> > > > > > > > > > > > > >> > > > > > > > > as an
> > > > > > > > > > > > > >> > > > > > > > > > > event loop. Those state
> machines
> > > > > will be
> > > > > > > > > > > executed
> > > > > > > > > > > > in
> > > > > > > > > > > > > >> > > > > > > > > > > *group.coordinator.threads*
> > > threads.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> machine
> > > are:
> > > > > > > > "Empty,
> > > > > > > > > > > > assigning,
> > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in Consumer
> > > Group
> > > > > States
> > > > > > > > > > > section,
> > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean "each
> > > state
> > > > > > > machine
> > > > > > > > is
> > > > > > > > > > > > modelled
> > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a state
> > > machine
> > > > > per
> > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > consumer
> > > > > group"
> > > > > > > > owned
> > > > > > > > > > > by
> > > > > > > > > > > > a
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > coordinator
> > > > > owns 2
> > > > > > > > > > > consumer
> > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> will we
> > > > > have 1
> > > > > > > > state
> > > > > > > > > > > > machine
> > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > *is
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> number
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > >> > > > > > > > > > > to run the state machines.
> But
> > > I'm
> > > > > > > > wondering
> > > > > > > > > > if
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the state of
> > > each
> > > > > > > consumer
> > > > > > > > > > group
> > > > > > > > > > > > (or
> > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no heavy
> > > > > computation,
> > > > > > > > why
> > > > > > > > > > > > should we
> > > > > > > > > > > > > >> need
> > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 2. For the default value in
> the
> > > new
> > > > > > > > configs:
> > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer session
> > > timeout,
> > > > > why
> > > > > > > > does
> > > > > > > > > > the
> > > > > > > > > > > > > >> default
> > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > >> > > > > > > > > > > locate between min (45s) and
> > > > > max(60s)? I
> > > > > > > > > > thought
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> lower/upper
> > > > > bound
> > > > > > > of
> > > > > > > > it,
> > > > > > > > > > > no?
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > group.consumer.session.timeout.ms
> > > > > int
> > > > > > > > 30s The
> > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> consumer
> > > > > group
> > > > > > > > > > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.min.session.timeout.ms
> > > > > > > > int 45s
> > > > > > > > > > > The
> > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.max.session.timeout.ms
> > > > > > > > int 60s
> > > > > > > > > > > The
> > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server side
> > > > > assignor
> > > > > > > are
> > > > > > > > > > > [range,
> > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > assignor.
> > > > > I'd
> > > > > > > > like to
> > > > > > > > > > > > know
> > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > >> > > > > > > > > > > thought usually users will
> > > choose
> > > > > > > uniform
> > > > > > > > > > > assignor
> > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> distribution.
> > > Any
> > > > > > > other
> > > > > > > > > > reason
> > > > > > > > > > > > we
> > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> List
> > > range,
> > > > > > > > uniform
> > > > > > > > > > The
> > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > Thank you.
> > > > > > > > > > > > > >> > > > > > > > > > > Luke
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Aug 22, 2022 at
> 2:10 PM
> > > Luke
> > > > > > > Chen
> > > > > > > > <
> > > > > > > > > > > > > >> showuon@gmail.com
> > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > > Hi Sagar,
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > > I have some thoughts about
> > > Kafka
> > > > > > > Connect
> > > > > > > > > > > > > >> integrating with
> > > > > > > > > > > > > >> > > > > > KIP-848,
> > > > > > > > > > > > > >> > > > > > > > > but I
> > > > > > > > > > > > > >> > > > > > > > > > > > think we should have a
> > > separate
> > > > > > > > discussion
> > > > > > > > > > > > thread
> > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > >> > > > Kafka
> > > > > > > > > > > > > >> > > > > > > > > Connect
> > > > > > > > > > > > > >> > > > > > > > > > > > KIP: Integrating Kafka
> Connect
> > > > > With
> > > > > > > New
> > > > > > > > > > > Consumer
> > > > > > > > > > > > > >> Rebalance
> > > > > > > > > > > > > >> > > > > > Protocol
> > > > > > > > > > > > > >> > > > > > > > > [1],
> > > > > > > > > > > > > >> > > > > > > > > > > > and let this discussion
> thread
> > > > > focus
> > > > > > > on
> > > > > > > > > > > consumer
> > > > > > > > > > > > > >> rebalance
> > > > > > > > > > > > > >> > > > > > protocol,
> > > > > > > > > > > > > >> > > > > > > > > WDYT?
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > > [1]
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > > Thank you.
> > > > > > > > > > > > > >> > > > > > > > > > > > Luke
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Aug 12, 2022 at
> 9:31
> > > PM
> > > > > Sagar
> > > > > > > <
> > > > > > > > > > > > > >> > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> Thank you Guozhang/David
> for
> > > the
> > > > > > > > feedback.
> > > > > > > > > > > > Looks
> > > > > > > > > > > > > >> like
> > > > > > > > > > > > > >> > > > there's
> > > > > > > > > > > > > >> > > > > > > > > agreement on
> > > > > > > > > > > > > >> > > > > > > > > > > >> using separate APIs for
> > > Connect.
> > > > > I
> > > > > > > > would
> > > > > > > > > > > > revisit
> > > > > > > > > > > > > >> the doc
> > > > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > >> > > > > > see
> > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > >> > > > > > > > > > > >> changes are to be made.
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> Thanks!
> > > > > > > > > > > > > >> > > > > > > > > > > >> Sagar.
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> On Tue, Aug 9, 2022 at
> 7:11
> > > PM
> > > > > David
> > > > > > > > Jacot
> > > > > > > > > > > > > >> > > > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > >> > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Hi Sagar,
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Thanks for the feedback
> > > and the
> > > > > > > > document.
> > > > > > > > > > > > That's
> > > > > > > > > > > > > >> really
> > > > > > > > > > > > > >> > > > > > helpful. I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > will take a look at it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Overall, it seems to me
> > > that
> > > > > both
> > > > > > > > Connect
> > > > > > > > > > > > and the
> > > > > > > > > > > > > >> > > > Consumer
> > > > > > > > > > > > > >> > > > > > could
> > > > > > > > > > > > > >> > > > > > > > > share
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the same underlying
> > > "engine".
> > > > > The
> > > > > > > > main
> > > > > > > > > > > > > >> difference is
> > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > Consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > assigns
> topic-partitions to
> > > > > members
> > > > > > > > > > whereas
> > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > >> > > > assigns
> > > > > > > > > > > > > >> > > > > > tasks
> > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > workers. I see two
> ways to
> > > move
> > > > > > > > forward:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) We extend the new
> > > proposed
> > > > > APIs
> > > > > > > to
> > > > > > > > > > > support
> > > > > > > > > > > > > >> different
> > > > > > > > > > > > > >> > > > > > resource
> > > > > > > > > > > > > >> > > > > > > > > types
> > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g. partitions,
> tasks,
> > > > > etc.); or
> > > > > > > > > > > > > >> > > > > > > > > > > >> > 2) We use new dedicated
> > > APIs
> > > > > for
> > > > > > > > Connect.
> > > > > > > > > > > The
> > > > > > > > > > > > > >> dedicated
> > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > >> > > > > > > > > would be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > similar to the new
> ones but
> > > > > > > > different on
> > > > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > content/resources and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > they would rely on the
> same
> > > > > engine
> > > > > > > > on the
> > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > >> > > > side.
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > I personally lean
> towards
> > > 2)
> > > > > > > because
> > > > > > > > I am
> > > > > > > > > > > > not a
> > > > > > > > > > > > > >> fan of
> > > > > > > > > > > > > >> > > > > > > > > overcharging
> > > > > > > > > > > > > >> > > > > > > > > > > >> > APIs to serve different
> > > > > purposes.
> > > > > > > > That
> > > > > > > > > > > being
> > > > > > > > > > > > > >> said, I am
> > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > >> > > > > > > > > opposed to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) if we can find an
> > > elegant
> > > > > way to
> > > > > > > > do
> > > > > > > > > > it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > I think that we can
> > > continue to
> > > > > > > > discuss
> > > > > > > > > > it
> > > > > > > > > > > > here
> > > > > > > > > > > > > >> for now
> > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > >> > > > > > order
> > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > ensure that this KIP is
> > > > > compatible
> > > > > > > > with
> > > > > > > > > > > what
> > > > > > > > > > > > we
> > > > > > > > > > > > > >> will do
> > > > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > >> > > > > > > > > Connect in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the future.
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Best,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > David
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > On Mon, Aug 8, 2022 at
> > > 2:41 PM
> > > > > > > David
> > > > > > > > > > Jacot
> > > > > > > > > > > <
> > > > > > > > > > > > > >> > > > > > djacot@confluent.io>
> > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > Hi all,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > I am back from
> vacation.
> > > I
> > > > > will
> > > > > > > go
> > > > > > > > > > > through
> > > > > > > > > > > > and
> > > > > > > > > > > > > >> address
> > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > >> > > > > > > > > comments
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > in the coming days.
> > > Thanks
> > > > > for
> > > > > > > your
> > > > > > > > > > > > feedback.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > Cheers,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > David
> > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Aug 3, 2022
> at
> > > 10:05
> > > > > PM
> > > > > > > > Gregory
> > > > > > > > > > > > Harris
> > > > > > > > > > > > > >> <
> > > > > > > > > > > > > >> > > > > > > > > gharris1727@gmail.com
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Hey All!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Thanks for the KIP,
> > > it's
> > > > > > > > wonderful to
> > > > > > > > > > > see
> > > > > > > > > > > > > >> > > > cooperative
> > > > > > > > > > > > > >> > > > > > > > > rebalancing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > making it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > down the stack!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > I had a few
> questions:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 1. The 'Rejected
> > > > > Alternatives'
> > > > > > > > > > section
> > > > > > > > > > > > > >> describes how
> > > > > > > > > > > > > >> > > > > > member
> > > > > > > > > > > > > >> > > > > > > > > epoch
> > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > advance in step
> with
> > > the
> > > > > group
> > > > > > > > epoch
> > > > > > > > > > > and
> > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > >> > > > > > epoch
> > > > > > > > > > > > > >> > > > > > > > > values. I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > that this is a good
> > > idea
> > > > > for
> > > > > > > the
> > > > > > > > > > > reasons
> > > > > > > > > > > > > >> described
> > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > KIP. When
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > protocol is
> > > incrementally
> > > > > > > > assigning
> > > > > > > > > > > > > >> partitions to a
> > > > > > > > > > > > > >> > > > > > worker,
> > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > >> > > > > > > > > > > >> member
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > epoch does each
> > > incremental
> > > > > > > > > > assignment
> > > > > > > > > > > > use?
> > > > > > > > > > > > > >> Are
> > > > > > > > > > > > > >> > > > member
> > > > > > > > > > > > > >> > > > > > epochs
> > > > > > > > > > > > > >> > > > > > > > > > > >> re-used,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > a single member
> epoch
> > > can
> > > > > > > > correspond
> > > > > > > > > > to
> > > > > > > > > > > > > >> multiple
> > > > > > > > > > > > > >> > > > > > different
> > > > > > > > > > > > > >> > > > > > > > > > > >> > (monotonically
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > larger)
> assignments?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 2. Is the
> Assignor's
> > > > > 'Reason'
> > > > > > > > field
> > > > > > > > > > > > opaque
> > > > > > > > > > > > > >> to the
> > > > > > > > > > > > > >> > > > group
> > > > > > > > > > > > > >> > > > > > > > > > > >> coordinator? If
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > not, should custom
> > > > > client-side
> > > > > > > > > > assignor
> > > > > > > > > > > > > >> > > > implementations
> > > > > > > > > > > > > >> > > > > > > > > interact
> > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Reason field, and
> how
> > > is
> > > > > its
> > > > > > > > common
> > > > > > > > > > > > meaning
> > > > > > > > > > > > > >> agreed
> > > > > > > > > > > > > >> > > > > > upon? If
> > > > > > > > > > > > > >> > > > > > > > > so, what
> > > > > > > > > > > > > >> > > > > > > > > > > >> > is the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > benefit of a
> distinct
> > > > > Reason
> > > > > > > > field
> > > > > > > > > > over
> > > > > > > > > > > > > >> including
> > > > > > > > > > > > > >> > > > such
> > > > > > > > > > > > > >> > > > > > > > > functionality
> > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > opaque metadata?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 3. The following is
> > > > > included in
> > > > > > > > the
> > > > > > > > > > > KIP:
> > > > > > > > > > > > > >> "Thanks to
> > > > > > > > > > > > > >> > > > > > this, the
> > > > > > > > > > > > > >> > > > > > > > > input
> > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client side
> assignor is
> > > > > > > entirely
> > > > > > > > > > driven
> > > > > > > > > > > > by
> > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > >> > > > > > > > > coordinator.
> > > > > > > > > > > > > >> > > > > > > > > > > >> The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > consumer is no
> longer
> > > > > > > > responsible for
> > > > > > > > > > > > > >> maintaining
> > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > >> > > > > > state
> > > > > > > > > > > > > >> > > > > > > > > besides
> > > > > > > > > > > > > >> > > > > > > > > > > >> its
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assigned
> partitions."
> > > Does
> > > > > this
> > > > > > > > mean
> > > > > > > > > > > > that the
> > > > > > > > > > > > > >> > > > > > client-side
> > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > > > >> MAY
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > incorporate
> additional
> > > > > > > > non-Metadata
> > > > > > > > > > > state
> > > > > > > > > > > > > >> (such as
> > > > > > > > > > > > > >> > > > > > partition
> > > > > > > > > > > > > >> > > > > > > > > > > >> > throughput,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > cpu/memory metrics,
> > > config
> > > > > > > > topics,
> > > > > > > > > > > etc),
> > > > > > > > > > > > or
> > > > > > > > > > > > > >> that
> > > > > > > > > > > > > >> > > > > > additional
> > > > > > > > > > > > > >> > > > > > > > > > > >> > non-Metadata
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > state SHOULD NOT be
> > > used?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 4. I see that
> there are
> > > > > > > separate
> > > > > > > > > > > classes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > org.apache.kafka.server.group.consumer.PartitionAssignor
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > >> > > >
> > > > > org.apache.kafka.clients.consumer.PartitionAssignor
> > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > >> > > > > > > > > seem to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > overlap
> significantly.
> > > Is
> > > > > it
> > > > > > > > possible
> > > > > > > > > > > for
> > > > > > > > > > > > > >> these two
> > > > > > > > > > > > > >> > > > > > > > > implementations
> > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > unified? This would
> > > serve
> > > > > to
> > > > > > > > promote
> > > > > > > > > > > > feature
> > > > > > > > > > > > > >> parity
> > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > >> > > > > > > > > server-side
> > > > > > > > > > > > > >> > > > > > > > > > > >> and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client-side
> assignors,
> > > and
> > > > > > > would
> > > > > > > > also
> > > > > > > > > > > > > >> facilitate
> > > > > > > > > > > > > >> > > > > > operational
> > > > > > > > > > > > > >> > > > > > > > > > > >> > flexibility in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > certain
> situations. For
> > > > > > > example,
> > > > > > > > if a
> > > > > > > > > > > > > >> server-side
> > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > has some
> > > > > > > > > > > > > >> > > > > > > > > > > >> > poor
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > behavior and needs
> a
> > > patch,
> > > > > > > > deploying
> > > > > > > > > > > the
> > > > > > > > > > > > > >> patched
> > > > > > > > > > > > > >> > > > > > assignor to
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and switching one
> > > consumer
> > > > > > > group
> > > > > > > > to a
> > > > > > > > > > > > > >> client-side
> > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > may be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > faster
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and less risky than
> > > > > patching
> > > > > > > all
> > > > > > > > of
> > > > > > > > > > the
> > > > > > > > > > > > > >> brokers.
> > > > > > > > > > > > > >> > > > With
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > currently
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > proposed distinct
> > > APIs, a
> > > > > > > > non-trivial
> > > > > > > > > > > > > >> > > > reimplementation
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > >> > > > > > > > > > > >> to be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assembled, and if
> the
> > > two
> > > > > APIs
> > > > > > > > have
> > > > > > > > > > > > diverged
> > > > > > > > > > > > > >> > > > > > significantly,
> > > > > > > > > > > > > >> > > > > > > > > then it
> > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > possible that a
> > > > > > > reimplementation
> > > > > > > > > > would
> > > > > > > > > > > > not be
> > > > > > > > > > > > > >> > > > possible.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > --
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Greg Harris
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> gharris1727@gmail.com
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> github.com/gharris1727
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Aug 3,
> 2022 at
> > > > > 8:39 AM
> > > > > > > > Sagar
> > > > > > > > > > <
> > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Hi
> Guozhang/David,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > I created a
> > > confluence
> > > > > page
> > > > > > > to
> > > > > > > > > > > discuss
> > > > > > > > > > > > how
> > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > change
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > based on the new
> > > > > rebalance
> > > > > > > > > > protocol.
> > > > > > > > > > > > > >> Here's the
> > > > > > > > > > > > > >> > > > page:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > It's also pretty
> > > longish
> > > > > and
> > > > > > > I
> > > > > > > > have
> > > > > > > > > > > > tried
> > > > > > > > > > > > > >> to keep
> > > > > > > > > > > > > >> > > > a
> > > > > > > > > > > > > >> > > > > > format
> > > > > > > > > > > > > >> > > > > > > > > > > >> similar to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > KIP-848. Let me
> know
> > > > > what you
> > > > > > > > > > think.
> > > > > > > > > > > > Also,
> > > > > > > > > > > > > >> do you
> > > > > > > > > > > > > >> > > > > > think this
> > > > > > > > > > > > > >> > > > > > > > > > > >> should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > moved to a
> separate
> > > > > > > discussion
> > > > > > > > > > thread
> > > > > > > > > > > > or
> > > > > > > > > > > > > >> is this
> > > > > > > > > > > > > >> > > > one
> > > > > > > > > > > > > >> > > > > > fine?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Sagar.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul 26,
> 2022
> > > at
> > > > > 7:37
> > > > > > > AM
> > > > > > > > > > > Sagar <
> > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Hello Guozhang,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thank you so
> much
> > > for
> > > > > the
> > > > > > > > doc on
> > > > > > > > > > > > Kafka
> > > > > > > > > > > > > >> Streams.
> > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > >> > > > > > > > > would do
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > analysis and
> come
> > > up
> > > > > with
> > > > > > > > such a
> > > > > > > > > > > > > >> document.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Sagar.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue, Jul 26,
> > > 2022 at
> > > > > > > 4:47
> > > > > > > > AM
> > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > >> Wang <
> > > > > > > > > > > > > >> > > > > > > > > > > >> wangguoz@gmail.com>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Hello Sagar,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> It would be
> great
> > > if
> > > > > you
> > > > > > > > could
> > > > > > > > > > > come
> > > > > > > > > > > > > >> back with
> > > > > > > > > > > > > >> > > > some
> > > > > > > > > > > > > >> > > > > > > > > analysis on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > how to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> implement the
> > > Connect
> > > > > side
> > > > > > > > > > > > integration
> > > > > > > > > > > > > >> with
> > > > > > > > > > > > > >> > > > the new
> > > > > > > > > > > > > >> > > > > > > > > protocol;
> > > > > > > > > > > > > >> > > > > > > > > > > >> so
> > > > > > > > > > > > > >> > > > > > > > > > > >> > far
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> besides
> > > leveraging on
> > > > > the
> > > > > > > > new
> > > > > > > > > > > > "protocol
> > > > > > > > > > > > > >> type"
> > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > >> > > > > > did not
> > > > > > > > > > > > > >> > > > > > > > > yet
> > > > > > > > > > > > > >> > > > > > > > > > > >> think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > through
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the Connect
> side
> > > > > > > > > > implementations.
> > > > > > > > > > > > For
> > > > > > > > > > > > > >> Streams
> > > > > > > > > > > > > >> > > > > > here's a
> > > > > > > > > > > > > >> > > > > > > > > draft of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> integration
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> plan:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> just FYI for
> your
> > > > > analysis
> > > > > > > > on
> > > > > > > > > > > > Connect.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> On Tue, Jul
> 19,
> > > 2022
> > > > > at
> > > > > > > > 10:48 PM
> > > > > > > > > > > > Sagar <
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> sagarmeansocean@gmail.com
> > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Hi David,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thank you
> for
> > > your
> > > > > > > > response.
> > > > > > > > > > The
> > > > > > > > > > > > > >> reason I
> > > > > > > > > > > > > >> > > > thought
> > > > > > > > > > > > > >> > > > > > > > > connect can
> > > > > > > > > > > > > >> > > > > > > > > > > >> > also fit
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> into
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > this new
> scheme
> > > is
> > > > > that
> > > > > > > > even
> > > > > > > > > > > > today the
> > > > > > > > > > > > > >> > > > connect
> > > > > > > > > > > > > >> > > > > > uses a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > WorkerCoordinator
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > extending
> from
> > > > > > > > > > > > AbstractCoordinator to
> > > > > > > > > > > > > >> empower
> > > > > > > > > > > > > >> > > > > > > > > rebalances of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > tasks/connectors.
> > > > > The
> > > > > > > > > > > > > >> WorkerCoordinator sets
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> protocolType()
> > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> connect
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > and uses the
> > > > > metadata()
> > > > > > > > method
> > > > > > > > > > > by
> > > > > > > > > > > > > >> plumbing
> > > > > > > > > > > > > >> > > > into
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > JoinGroupRequestProtocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > I think the
> > > changes
> > > > > to
> > > > > > > > support
> > > > > > > > > > > > > >> connect would
> > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > >> > > > > > > > > similar at a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > high
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > level
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the changes
> in
> > > > > streams
> > > > > > > > mainly
> > > > > > > > > > > > because
> > > > > > > > > > > > > >> of the
> > > > > > > > > > > > > >> > > > > > Client
> > > > > > > > > > > > > >> > > > > > > > > side
> > > > > > > > > > > > > >> > > > > > > > > > > >> > assignors
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > being
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > used in
> both.
> > > At an
> > > > > > > > > > > implementation
> > > > > > > > > > > > > >> level, we
> > > > > > > > > > > > > >> > > > > > might
> > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > >> > > > > > > > > > > >> make
> > > > > > > > > > > > > >> > > > > > > > > > > >> > a lot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > changes to
> get
> > > onto
> > > > > this
> > > > > > > > new
> > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > >> > > > protocol
> > > > > > > > > > > > > >> > > > > > like
> > > > > > > > > > > > > >> > > > > > > > > > > >> enhancing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > JoinGroup
> > > > > > > > request/response and
> > > > > > > > > > > > > >> SyncGroup and
> > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > ConsumerGroupHeartbeat
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > API etc
> again on
> > > > > similar
> > > > > > > > lines
> > > > > > > > > > > to
> > > > > > > > > > > > > >> streams (or
> > > > > > > > > > > > > >> > > > > > there
> > > > > > > > > > > > > >> > > > > > > > > might be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> deviations). I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > would try to
> > > > > perform a
> > > > > > > > > > detailed
> > > > > > > > > > > > > >> analysis of
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > same
> > > > > > > > > > > > > >> > > > > > > > > and we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > can have
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > separate
> > > discussion
> > > > > > > > thread for
> > > > > > > > > > > > that
> > > > > > > > > > > > > >> as that
> > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > >> > > > > > > > > derail this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > discussion
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > thread. Let
> me
> > > know
> > > > > if
> > > > > > > > that
> > > > > > > > > > > sounds
> > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > >> > > > you.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thanks!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Sagar.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > On Fri, Jul
> 15,
> > > > > 2022 at
> > > > > > > > 5:47
> > > > > > > > > > PM
> > > > > > > > > > > > David
> > > > > > > > > > > > > >> Jacot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Hi Sagar,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Thanks for
> > > your
> > > > > > > > comments.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 1) Yes.
> That
> > > > > refers to
> > > > > > > > > > > > > >> `Assignment#error`.
> > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > >> > > > > > > > > can
> > > > > > > > > > > > > >> > > > > > > > > > > >> > mention it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 2) The
> idea
> > > is to
> > > > > > > > > > transition C
> > > > > > > > > > > > from
> > > > > > > > > > > > > >> his
> > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > >> > > > > > > > > assignment
> > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > his
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > target
> > > assignment
> > > > > when
> > > > > > > > he
> > > > > > > > > > can
> > > > > > > > > > > > move
> > > > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > epoch 3.
> > > > > > > > > > > > > >> > > > > > When
> > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > happens,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > assignment
> > > > > is
> > > > > > > > updated
> > > > > > > > > > > and
> > > > > > > > > > > > > >> persisted
> > > > > > > > > > > > > >> > > > > > with all
> > > > > > > > > > > > > >> > > > > > > > > its
> > > > > > > > > > > > > >> > > > > > > > > > > >> > assigned
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > partitions
> > > even if
> > > > > > > they
> > > > > > > > are
> > > > > > > > > > > not
> > > > > > > > > > > > all
> > > > > > > > > > > > > >> revoked
> > > > > > > > > > > > > >> > > > > > yet. In
> > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > >> > > > > > > > > > > >> > words, the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > assignment
> > > > > > > > becomes
> > > > > > > > > > the
> > > > > > > > > > > > target
> > > > > > > > > > > > > >> > > > > > assignment.
> > > > > > > > > > > > > >> > > > > > > > > This is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > basically
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> optimization
> > > to
> > > > > avoid
> > > > > > > > having
> > > > > > > > > > > to
> > > > > > > > > > > > > >> write all
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > changes to
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > log.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > examples
> are
> > > > > based on
> > > > > > > > the
> > > > > > > > > > > > persisted
> > > > > > > > > > > > > >> state
> > > > > > > > > > > > > >> > > > so I
> > > > > > > > > > > > > >> > > > > > > > > understand
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > confusion.
> > > Let me
> > > > > see
> > > > > > > > if I
> > > > > > > > > > can
> > > > > > > > > > > > > >> improve
> > > > > > > > > > > > > >> > > > this in
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > description.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 3)
> Regarding
> > > > > Connect,
> > > > > > > it
> > > > > > > > > > could
> > > > > > > > > > > > > >> reuse the
> > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > >> > > > > > > > > with a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > client side
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > assignor
> if it
> > > > > fits in
> > > > > > > > the
> > > > > > > > > > > > > >> protocol. The
> > > > > > > > > > > > > >> > > > > > assignment
> > > > > > > > > > > > > >> > > > > > > > > is
> > > > > > > > > > > > > >> > > > > > > > > > > >> about
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > topicid-partitions +
> > > > > > > > > > metadata,
> > > > > > > > > > > > could
> > > > > > > > > > > > > >> > > > Connect
> > > > > > > > > > > > > >> > > > > > fit
> > > > > > > > > > > > > >> > > > > > > > > into this?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Best,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > David
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > On Fri,
> Jul
> > > 15,
> > > > > 2022
> > > > > > > at
> > > > > > > > 1:55
> > > > > > > > > > > PM
> > > > > > > > > > > > > >> Sagar <
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> sagarmeansocean@gmail.com>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Hi
> David,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Thanks
> for
> > > the
> > > > > KIP.
> > > > > > > I
> > > > > > > > just
> > > > > > > > > > > had
> > > > > > > > > > > > > >> minor
> > > > > > > > > > > > > >> > > > > > observations:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 1) In
> the
> > > > > Assignment
> > > > > > > > Error
> > > > > > > > > > > > > >> section in
> > > > > > > > > > > > > >> > > > Client
> > > > > > > > > > > > > >> > > > > > Side
> > > > > > > > > > > > > >> > > > > > > > > mode
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Assignment
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > process,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > you
> > > mentioned
> > > > > => `In
> > > > > > > > this
> > > > > > > > > > > > case,
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > >> > > > > > side
> > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > >> > > > > > > > > > > >> can
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > return
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > error
> to the
> > > > > group
> > > > > > > > > > > > coordinator`.
> > > > > > > > > > > > > >> In this
> > > > > > > > > > > > > >> > > > > > case are
> > > > > > > > > > > > > >> > > > > > > > > you
> > > > > > > > > > > > > >> > > > > > > > > > > >> > referring to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Assignor
> > > > > returning
> > > > > > > an
> > > > > > > > > > > > > >> AssignmentError
> > > > > > > > > > > > > >> > > > that's
> > > > > > > > > > > > > >> > > > > > > > > listed down
> > > > > > > > > > > > > >> > > > > > > > > > > >> > towards
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > end?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > If yes,
> do
> > > you
> > > > > think
> > > > > > > > it
> > > > > > > > > > > would
> > > > > > > > > > > > > >> make sense
> > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > >> > > > > > > > > mention this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > explicitly
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > here?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 2) In
> the
> > > Case
> > > > > > > Studies
> > > > > > > > > > > > section, I
> > > > > > > > > > > > > >> have a
> > > > > > > > > > > > > >> > > > > > slight
> > > > > > > > > > > > > >> > > > > > > > > > > >> confusion,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > sure
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> if
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > others
> have
> > > the
> > > > > > > same.
> > > > > > > > > > > Consider
> > > > > > > > > > > > > >> this step:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > When B
> > > > > heartbeats,
> > > > > > > the
> > > > > > > > > > group
> > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > >> > > > > > > > > transitions him
> > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > epoch
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 3
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > because
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > B has no
> > > > > partitions
> > > > > > > to
> > > > > > > > > > > > revoke. It
> > > > > > > > > > > > > >> > > > persists
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > change and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > reply.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> Group
> > > > > (epoch=3)
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> A
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> B
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> C
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> Target
> > > > > > > Assignment
> > > > > > > > > > > > (epoch=3)
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> A -
> > > > > > > > > > > partitions=[foo-0]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> B -
> > > > > > > > > > > partitions=[foo-2]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> C -
> > > > > > > > > > > partitions=[foo-1]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> Member
> > > > > > > Assignment
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> A -
> > > > > epoch=2,
> > > > > > > > > > > > > >> partitions=[foo-0,
> > > > > > > > > > > > > >> > > > > > foo-1]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> B -
> > > > > epoch=3,
> > > > > > > > > > > > > >> partitions=[foo-2]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> C -
> > > > > epoch=3,
> > > > > > > > > > > > > >> partitions=[foo-1]
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > When C
> > > > > heartbeats,
> > > > > > > it
> > > > > > > > > > > > transitions
> > > > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > epoch 3
> > > > > > > > > > > > > >> > > > > > but
> > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > >> > > > > > > > > > > >> get
> > > > > > > > > > > > > >> > > > > > > > > > > >> > foo-1
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> yet.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Here,it's
> > > > > mentioned
> > > > > > > > that
> > > > > > > > > > > > member C
> > > > > > > > > > > > > >> can't
> > > > > > > > > > > > > >> > > > get
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > foo-1
> > > > > > > > > > > > > >> > > > > > > > > > > >> > partition
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > yet,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > based
> on the
> > > > > > > > description
> > > > > > > > > > > > above,
> > > > > > > > > > > > > >> it seems
> > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > >> > > > > > > > > already has
> > > > > > > > > > > > > >> > > > > > > > > > > >> it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Do you
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > would be
> > > better
> > > > > to
> > > > > > > > remove
> > > > > > > > > > it
> > > > > > > > > > > > and
> > > > > > > > > > > > > >> > > > populate it
> > > > > > > > > > > > > >> > > > > > only
> > > > > > > > > > > > > >> > > > > > > > > when it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > actually
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> gets
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > I see
> this
> > > in a
> > > > > lot
> > > > > > > of
> > > > > > > > > > other
> > > > > > > > > > > > > >> places, so
> > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > >> > > > > > I
> > > > > > > > > > > > > >> > > > > > > > > > > >> understood it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > incorrectly
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Regarding
> > > > > connect ,
> > > > > > > it
> > > > > > > > > > might
> > > > > > > > > > > > be
> > > > > > > > > > > > > >> out of
> > > > > > > > > > > > > >> > > > scope
> > > > > > > > > > > > > >> > > > > > of
> > > > > > > > > > > > > >> > > > > > > > > this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > discussion,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > from
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > what I
> > > > > understood it
> > > > > > > > would
> > > > > > > > > > > > > >> probably be
> > > > > > > > > > > > > >> > > > > > running in
> > > > > > > > > > > > > >> > > > > > > > > client
> > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> assignor
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > mode
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > even on
> the
> > > new
> > > > > > > > rebalance
> > > > > > > > > > > > > >> protocol as it
> > > > > > > > > > > > > >> > > > has
> > > > > > > > > > > > > >> > > > > > its
> > > > > > > > > > > > > >> > > > > > > > > own
> > > > > > > > > > > > > >> > > > > > > > > > > >> Custom
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > Assignors(Eager
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > and
> > > > > > > > > > IncrementalCooperative).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Thanks!
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Sagar.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > On Fri,
> Jul
> > > 15,
> > > > > 2022
> > > > > > > > at
> > > > > > > > > > 5:00
> > > > > > > > > > > > PM
> > > > > > > > > > > > > >> David
> > > > > > > > > > > > > >> > > > Jacot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Thanks
> > > > > Hector! Our
> > > > > > > > goal
> > > > > > > > > > is
> > > > > > > > > > > > to
> > > > > > > > > > > > > >> move
> > > > > > > > > > > > > >> > > > forward
> > > > > > > > > > > > > >> > > > > > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > specialized API
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> instead of
> > > > > relying
> > > > > > > > on
> > > > > > > > > > one
> > > > > > > > > > > > > >> generic API.
> > > > > > > > > > > > > >> > > > For
> > > > > > > > > > > > > >> > > > > > > > > Connect, we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > can apply
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > exact
> same
> > > > > pattern
> > > > > > > > and
> > > > > > > > > > > > > >> reuse/share the
> > > > > > > > > > > > > >> > > > core
> > > > > > > > > > > > > >> > > > > > > > > > > >> > implementation on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > server
> > > side.
> > > > > For
> > > > > > > the
> > > > > > > > > > > schema
> > > > > > > > > > > > > >> registry, I
> > > > > > > > > > > > > >> > > > > > think
> > > > > > > > > > > > > >> > > > > > > > > that we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consider
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> having a
> > > > > tailored
> > > > > > > > API to
> > > > > > > > > > > do
> > > > > > > > > > > > > >> simple
> > > > > > > > > > > > > >> > > > > > > > > membership/leader
> > > > > > > > > > > > > >> > > > > > > > > > > >> > election.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Best,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > David
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > On
> Fri,
> > > Jul
> > > > > 15,
> > > > > > > > 2022 at
> > > > > > > > > > > > 10:22
> > > > > > > > > > > > > >> AM Ismael
> > > > > > > > > > > > > >> > > > > > Juma <
> > > > > > > > > > > > > >> > > > > > > > > > > >> > ismael@juma.me.uk
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> Three
> > > quick
> > > > > > > > comments:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 1.
> > > Regarding
> > > > > > > > > > > > > >> java.util.regex.Pattern
> > > > > > > > > > > > > >> > > > vs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > com.google.re2j.Pattern,
> > > > > > > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > document the
> > > > > > > > > > differences
> > > > > > > > > > > > in
> > > > > > > > > > > > > >> more
> > > > > > > > > > > > > >> > > > detail
> > > > > > > > > > > > > >> > > > > > before
> > > > > > > > > > > > > >> > > > > > > > > > > >> deciding
> > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > way
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> or
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> another.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > That
> > > said,
> > > > > if
> > > > > > > > people
> > > > > > > > > > > pass
> > > > > > > > > > > > > >> > > > > > > > > java.util.regex.Pattern,
> > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > >> > > > > > > > > > > >> > expect
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > their
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > semantics
> > > > > to be
> > > > > > > > > > honored.
> > > > > > > > > > > > If
> > > > > > > > > > > > > >> we are
> > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > >> > > > > > > > > something
> > > > > > > > > > > > > >> > > > > > > > > > > >> > different,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> then
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> should
> > > > > consider
> > > > > > > > adding
> > > > > > > > > > > an
> > > > > > > > > > > > > >> overload
> > > > > > > > > > > > > >> > > > with
> > > > > > > > > > > > > >> > > > > > our own
> > > > > > > > > > > > > >> > > > > > > > > > > >> Pattern
> > > > > > > > > > > > > >> > > > > > > > > > > >> > class
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > (I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > don't
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> think
> > > we'd
> > > > > want
> > > > > > > to
> > > > > > > > > > > expose
> > > > > > > > > > > > > >> re2j's at
> > > > > > > > > > > > > >> > > > this
> > > > > > > > > > > > > >> > > > > > > > > point).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 2.
> > > Regarding
> > > > > > > topic
> > > > > > > > > > ids,
> > > > > > > > > > > > any
> > > > > > > > > > > > > >> major new
> > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > >> > > > > > > > > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > integrate
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > it
> and
> > > > > should
> > > > > > > > handle
> > > > > > > > > > the
> > > > > > > > > > > > topic
> > > > > > > > > > > > > >> > > > > > recreation case
> > > > > > > > > > > > > >> > > > > > > > > > > >> > correctly.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That's
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > main
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> part we
> > > > > need to
> > > > > > > > > > handle.
> > > > > > > > > > > I
> > > > > > > > > > > > > >> agree with
> > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > >> > > > > > > > > that we'd
> > > > > > > > > > > > > >> > > > > > > > > > > >> > want to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > add
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > to
> the
> > > > > relevant
> > > > > > > > > > > protocols
> > > > > > > > > > > > > >> that don't
> > > > > > > > > > > > > >> > > > > > have it
> > > > > > > > > > > > > >> > > > > > > > > yet and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > that we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > can
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > probably
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> focus
> > > on the
> > > > > > > > internals
> > > > > > > > > > > > versus
> > > > > > > > > > > > > >> adding
> > > > > > > > > > > > > >> > > > new
> > > > > > > > > > > > > >> > > > > > APIs
> > > > > > > > > > > > > >> > > > > > > > > to the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> (unless
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > we
> find
> > > that
> > > > > > > > adding
> > > > > > > > > > new
> > > > > > > > > > > > APIs
> > > > > > > > > > > > > >> is
> > > > > > > > > > > > > >> > > > required
> > > > > > > > > > > > > >> > > > > > for
> > > > > > > > > > > > > >> > > > > > > > > > > >> reasonable
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> semantics).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 3.
> I am
> > > > > still
> > > > > > > not
> > > > > > > > sure
> > > > > > > > > > > > about
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > >> > > > > > > > > storing
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> configs.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It's
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > powerful for
> > > > > > > > configs
> > > > > > > > > > to
> > > > > > > > > > > be
> > > > > > > > > > > > > >> > > > centralized
> > > > > > > > > > > > > >> > > > > > in the
> > > > > > > > > > > > > >> > > > > > > > > > > >> metadata
> > > > > > > > > > > > > >> > > > > > > > > > > >> > log for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > various
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> reasons
> > > > > > > > (auditability,
> > > > > > > > > > > > > >> visibility,
> > > > > > > > > > > > > >> > > > > > consistency,
> > > > > > > > > > > > > >> > > > > > > > > > > >> etc.).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Similarly, I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > am
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > sure
> > > about
> > > > > > > > > > automatically
> > > > > > > > > > > > > >> deleting
> > > > > > > > > > > > > >> > > > > > configs in a
> > > > > > > > > > > > > >> > > > > > > > > way
> > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > they
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cannot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > recovered. A
> > > > > > > good
> > > > > > > > > > > property
> > > > > > > > > > > > > >> for modern
> > > > > > > > > > > > > >> > > > > > systems
> > > > > > > > > > > > > >> > > > > > > > > is to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > minimize
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > number
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > unrecoverable
> > > > > > > data
> > > > > > > > > > loss
> > > > > > > > > > > > > >> scenarios.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> Ismael
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > On
> Wed,
> > > Jul
> > > > > 13,
> > > > > > > > 2022
> > > > > > > > > > at
> > > > > > > > > > > > 3:47
> > > > > > > > > > > > > >> PM David
> > > > > > > > > > > > > >> > > > > > Jacot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> Thanks
> > > > > > > > Guozhang. My
> > > > > > > > > > > > answers
> > > > > > > > > > > > > >> are
> > > > > > > > > > > > > >> > > > below:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> 1)
> > > the
> > > > > > > > migration
> > > > > > > > > > > path,
> > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> flag
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> to
> > > > > enable
> > > > > > > the
> > > > > > > > new
> > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > protocols /
> > > > > > > > rpcs
> > > > > > > > > > and
> > > > > > > > > > > > old
> > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > group.
> > > > > How
> > > > > > > the
> > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > using
> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> new
> > > > > protocol
> > > > > > > > is
> > > > > > > > > > > > something
> > > > > > > > > > > > > >> we
> > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > Noted. I
> > > > > just
> > > > > > > > > > > published
> > > > > > > > > > > > a
> > > > > > > > > > > > > >> new
> > > > > > > > > > > > > >> > > > version
> > > > > > > > > > > > > >> > > > > > of KIP
> > > > > > > > > > > > > >> > > > > > > > > which
> > > > > > > > > > > > > >> > > > > > > > > > > >> > includes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> more
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > details
> > > > > about
> > > > > > > > this.
> > > > > > > > > > > See
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> > > > "Supporting
> > > > > > > > > > > > > >> > > > > > > > > Online
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Upgrade"
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> and
> > > the
> > > > > > > > > > > "Compatibility,
> > > > > > > > > > > > > >> > > > Deprecation,
> > > > > > > > > > > > > >> > > > > > and
> > > > > > > > > > > > > >> > > > > > > > > Migration
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Plan". I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > I
> > > have to
> > > > > > > think
> > > > > > > > > > > through
> > > > > > > > > > > > a
> > > > > > > > > > > > > >> few cases
> > > > > > > > > > > > > >> > > > > > now but
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > overall idea
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > mechanism
> > > > > > > > should be
> > > > > > > > > > > > > >> understandable.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> 2)
> > > the
> > > > > usage
> > > > > > > > of
> > > > > > > > > > > topic
> > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > used
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > as
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > part of
> > > > > RPCs
> > > > > > > > and
> > > > > > > > > > > admin
> > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > public
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> APIs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> to
> > > > > consumers
> > > > > > > > yet.
> > > > > > > > > > I
> > > > > > > > > > > > think
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > client
> > > > > to be
> > > > > > > > > > > > maintaining
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > leverage
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> on
> > > all
> > > > > the
> > > > > > > > > > augmented
> > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids;
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > secondly,
> > > > > > > > should
> > > > > > > > > > we
> > > > > > > > > > > > ever
> > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > public
> > > > > APIs
> > > > > > > as
> > > > > > > > > > well
> > > > > > > > > > > > (both
> > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> rebalance
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > listener for
> > > > > > > > cases
> > > > > > > > > > > > like
> > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> deletion-and-recreation).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > a)
> > > > > Assuming
> > > > > > > > that we
> > > > > > > > > > > > would
> > > > > > > > > > > > > >> include
> > > > > > > > > > > > > >> > > > > > converting
> > > > > > > > > > > > > >> > > > > > > > > all
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > offsets
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > related
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> RPCs
> > > to
> > > > > using
> > > > > > > > topic
> > > > > > > > > > > ids
> > > > > > > > > > > > in
> > > > > > > > > > > > > >> this
> > > > > > > > > > > > > >> > > > KIP,
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > would be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> able
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> fully
> > > > > operate
> > > > > > > > with
> > > > > > > > > > > topic
> > > > > > > > > > > > > >> ids. That
> > > > > > > > > > > > > >> > > > > > being
> > > > > > > > > > > > > >> > > > > > > > > said, it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > still has
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > provide
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> the
> > > topics
> > > > > > > > names in
> > > > > > > > > > > > various
> > > > > > > > > > > > > >> APIs so
> > > > > > > > > > > > > >> > > > > > having a
> > > > > > > > > > > > > >> > > > > > > > > > > >> mapping
> > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> seems
> > > > > > > > inevitable to
> > > > > > > > > > > me.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> b) I
> > > don't
> > > > > > > have
> > > > > > > > a
> > > > > > > > > > > strong
> > > > > > > > > > > > > >> opinion on
> > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > >> > > > > > > > > Here I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > wonder if
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > goes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > beyond the
> > > > > > > > scope of
> > > > > > > > > > > this
> > > > > > > > > > > > > >> KIP. I
> > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > >> > > > > > rather
> > > > > > > > > > > > > >> > > > > > > > > focus
> > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > internals
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> here
> > > and
> > > > > we
> > > > > > > can
> > > > > > > > > > > consider
> > > > > > > > > > > > > >> this
> > > > > > > > > > > > > >> > > > > > separately if
> > > > > > > > > > > > > >> > > > > > > > > we see
> > > > > > > > > > > > > >> > > > > > > > > > > >> > value in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> doing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> Coming
> > > > > back to
> > > > > > > > > > > Ismael's
> > > > > > > > > > > > > >> point about
> > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > >> > > > > > > > > topic ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> ConsumerGroupHeartbeatRequest, I
> > > > > > > > > > > > > >> > > > think
> > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > >> > > > > > > > > there
> > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > advantage
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > favour of
> > > > > it.
> > > > > > > > The
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > >> will
> > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> opportunity
> > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validate
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> the
> > > topics
> > > > > > > > exists
> > > > > > > > > > > before
> > > > > > > > > > > > > >> passing
> > > > > > > > > > > > > >> > > > them
> > > > > > > > > > > > > >> > > > > > into
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > rebalance
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > protocol.
> > > > > > > > Obviously,
> > > > > > > > > > > the
> > > > > > > > > > > > > >> > > > coordinator
> > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > >> > > > > > > > > also
> > > > > > > > > > > > > >> > > > > > > > > > > >> notice
> > > > > > > > > > > > > >> > > > > > > > > > > >> > it but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > does
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> not
> > > really
> > > > > > > have
> > > > > > > > a
> > > > > > > > > > way
> > > > > > > > > > > to
> > > > > > > > > > > > > >> reject an
> > > > > > > > > > > > > >> > > > > > invalid
> > > > > > > > > > > > > >> > > > > > > > > topic in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> response.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> I'm
> > > > > agreeing
> > > > > > > > with
> > > > > > > > > > > > David
> > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > necessary
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > deprecate
> > > > > > > the
> > > > > > > > > > > > subscribe
> > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > Pattern
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > while
> > > > > just
> > > > > > > > > > > documenting
> > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected by
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > server.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > Since
> > > > > the
> > > > > > > > > > > incompatible
> > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > overloaded
> > > > > > > > > > `String`
> > > > > > > > > > > > based
> > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> various
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > invalid
> > > > > > > > regexes.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> That
> > > could
> > > > > > > > work. I
> > > > > > > > > > > have
> > > > > > > > > > > > to
> > > > > > > > > > > > > >> look at
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > differences
> > > > > > > > > > > > > >> > > > > > > > > > > >> > between
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > two
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > engines to
> > > > > > > > better
> > > > > > > > > > > > > >> understand the
> > > > > > > > > > > > > >> > > > > > potential
> > > > > > > > > > > > > >> > > > > > > > > issues.
> > > > > > > > > > > > > >> > > > > > > > > > > >> My
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> understanding
> > > is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> that
> > > would
> > > > > > > work
> > > > > > > > for
> > > > > > > > > > > all
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> basic
> > > > > > > > > > > > > >> > > > > > regular
> > > > > > > > > > > > > >> > > > > > > > > > > >> > expressions. The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> differences
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > between
> > > > > the
> > > > > > > two
> > > > > > > > are
> > > > > > > > > > > > mainly
> > > > > > > > > > > > > >> about
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > various
> > > > > > > > > > > > > >> > > > > > > > > > > >> character
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> classes. I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> wonder
> > > > > what
> > > > > > > > other
> > > > > > > > > > > people
> > > > > > > > > > > > > >> think
> > > > > > > > > > > > > >> > > > about
> > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> Best,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> David
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > On
> > > Tue,
> > > > > Jul
> > > > > > > 12,
> > > > > > > > 2022
> > > > > > > > > > > at
> > > > > > > > > > > > > >> 11:28 PM
> > > > > > > > > > > > > >> > > > > > Guozhang
> > > > > > > > > > > > > >> > > > > > > > > Wang <
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > wangguoz@gmail.com
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > Thanks
> > > > > > > David!
> > > > > > > > I
> > > > > > > > > > > think
> > > > > > > > > > > > on
> > > > > > > > > > > > > >> the high
> > > > > > > > > > > > > >> > > > > > level
> > > > > > > > > > > > > >> > > > > > > > > there are
> > > > > > > > > > > > > >> > > > > > > > > > > >> > two meta
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > points we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > need
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> to
> > > > > > > concretize
> > > > > > > > a
> > > > > > > > > > bit
> > > > > > > > > > > > more:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> 1)
> > > the
> > > > > > > > migration
> > > > > > > > > > > path,
> > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> flag
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> to
> > > > > enable
> > > > > > > the
> > > > > > > > new
> > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > protocols /
> > > > > > > > rpcs
> > > > > > > > > > and
> > > > > > > > > > > > old
> > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > group.
> > > > > How
> > > > > > > the
> > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > using
> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> new
> > > > > protocol
> > > > > > > > is
> > > > > > > > > > > > something
> > > > > > > > > > > > > >> we
> > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> 2)
> > > the
> > > > > usage
> > > > > > > > of
> > > > > > > > > > > topic
> > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > used
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > as
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > part of
> > > > > RPCs
> > > > > > > > and
> > > > > > > > > > > admin
> > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > public
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> APIs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> to
> > > > > consumers
> > > > > > > > yet.
> > > > > > > > > > I
> > > > > > > > > > > > think
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > client
> > > > > to be
> > > > > > > > > > > > maintaining
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > leverage
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> on
> > > all
> > > > > the
> > > > > > > > > > augmented
> > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids;
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > secondly,
> > > > > > > > should
> > > > > > > > > > we
> > > > > > > > > > > > ever
> > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > public
> > > > > APIs
> > > > > > > as
> > > > > > > > > > well
> > > > > > > > > > > > (both
> > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> rebalance
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > listener for
> > > > > > > > cases
> > > > > > > > > > > > like
> > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> deletion-and-recreation).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> I'm
> > > > > agreeing
> > > > > > > > with
> > > > > > > > > > > > David
> > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > necessary
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > deprecate
> > > > > > > the
> > > > > > > > > > > > subscribe
> > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > Pattern
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > while
> > > > > just
> > > > > > > > > > > documenting
> > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected by
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > server.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > Since
> > > > > the
> > > > > > > > > > > incompatible
> > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > overloaded
> > > > > > > > > > `String`
> > > > > > > > > > > > based
> > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> various
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > invalid
> > > > > > > > regexes.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > Guozhang
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> On
> > > Tue,
> > > > > Jul
> > > > > > > > 12,
> > > > > > > > > > 2022
> > > > > > > > > > > > at
> > > > > > > > > > > > > >> 5:23 AM
> > > > > > > > > > > > > >> > > > > > David Jacot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > Hi
> > > > > Ismael,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > Thanks for
> > > > > > > > your
> > > > > > > > > > > > > >> feedback. Let
> > > > > > > > > > > > > >> > > > me
> > > > > > > > > > > > > >> > > > > > answer
> > > > > > > > > > > > > >> > > > > > > > > your
> > > > > > > > > > > > > >> > > > > > > > > > > >> > questions
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > inline.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 1. I
> > > > > > > think
> > > > > > > > > > it's
> > > > > > > > > > > > > >> premature to
> > > > > > > > > > > > > >> > > > > > talk about
> > > > > > > > > > > > > >> > > > > > > > > > > >> target
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > versions
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > deprecation
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > removal
> > > > > > > > of the
> > > > > > > > > > > > > >> existing group
> > > > > > > > > > > > > >> > > > > > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> Unlike
> > > > > > > > > > > > > >> > > > > > > > > > > >> > KRaft,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> affects a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> core
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > client
> > > > > > > > > > protocol
> > > > > > > > > > > > and
> > > > > > > > > > > > > >> hence
> > > > > > > > > > > > > >> > > > > > > > > deprecation/removal
> > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > heavily
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > dependent
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > how
> > > > > > > > quickly
> > > > > > > > > > > > > >> applications
> > > > > > > > > > > > > >> > > > migrate
> > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > That
> > > > > makes
> > > > > > > > > > sense.
> > > > > > > > > > > I
> > > > > > > > > > > > will
> > > > > > > > > > > > > >> > > > remove it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 2.
> > > > > The
> > > > > > > KIP
> > > > > > > > > > says
> > > > > > > > > > > we
> > > > > > > > > > > > > >> intend to
> > > > > > > > > > > > > >> > > > > > release
> > > > > > > > > > > > > >> > > > > > > > > this in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > 4.x, but
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wasn't
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > made
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > clear
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > why. If
> > > > > > > we
> > > > > > > > > > added
> > > > > > > > > > > > that
> > > > > > > > > > > > > >> as a
> > > > > > > > > > > > > >> > > > way to
> > > > > > > > > > > > > >> > > > > > > > > estimate
> > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > we'd
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > deprecate
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> remove
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > the
> > > > > > > group
> > > > > > > > > > > > protocol, I
> > > > > > > > > > > > > >> also
> > > > > > > > > > > > > >> > > > > > suggest
> > > > > > > > > > > > > >> > > > > > > > > removing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > this part.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Let me
> > > > > > > > explain
> > > > > > > > > > my
> > > > > > > > > > > > > >> reasoning. As
> > > > > > > > > > > > > >> > > > > > > > > explained, I
> > > > > > > > > > > > > >> > > > > > > > > > > >> plan
> > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rewrite
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > coordinator
> > > > > > > > in
> > > > > > > > > > > Java
> > > > > > > > > > > > > >> while we
> > > > > > > > > > > > > >> > > > > > implement
> > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> This
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > that
> > > > > the
> > > > > > > > > > internals
> > > > > > > > > > > > will
> > > > > > > > > > > > > >> be
> > > > > > > > > > > > > >> > > > slightly
> > > > > > > > > > > > > >> > > > > > > > > different
> > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> threading
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> model).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > Therefore, I
> > > > > > > > > > > wanted
> > > > > > > > > > > > to
> > > > > > > > > > > > > >> tighten
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > switch from
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the old
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > coordinator
> > > > > > > > to
> > > > > > > > > > the
> > > > > > > > > > > > new
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > >> > > > > > > > > to a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > major
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> release.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > alternative
> > > > > > > > > > would
> > > > > > > > > > > > be to
> > > > > > > > > > > > > >> use a
> > > > > > > > > > > > > >> > > > flag
> > > > > > > > > > > > > >> > > > > > to do
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > switch
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instead
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> relying
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > on the
> > > > > > > > software
> > > > > > > > > > > > upgrade.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 3.
> > > > > We
> > > > > > > > need to
> > > > > > > > > > > > flesh
> > > > > > > > > > > > > >> out the
> > > > > > > > > > > > > >> > > > > > details of
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > migration
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> story.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > sounds
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > like
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > we're
> > > > > > > > saying
> > > > > > > > > > we
> > > > > > > > > > > > will
> > > > > > > > > > > > > >> support
> > > > > > > > > > > > > >> > > > > > online
> > > > > > > > > > > > > >> > > > > > > > > > > >> migrations.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > correct?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > We
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > explain
> > > > > > > > this
> > > > > > > > > > in
> > > > > > > > > > > > > >> detail. It
> > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > >> > > > > > also
> > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > >> > > > > > > > > > > >> > as a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> separate
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > KIP,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > if
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> it's
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > easier.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Yes,
> > > > > we
> > > > > > > will
> > > > > > > > > > > support
> > > > > > > > > > > > > >> online
> > > > > > > > > > > > > >> > > > > > migrations
> > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > group.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > a
> > > > > group
> > > > > > > > using
> > > > > > > > > > the
> > > > > > > > > > > > old
> > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > >> > > > > > > > > able to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > switch to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Let me
> > > > > > > > briefly
> > > > > > > > > > > > explain
> > > > > > > > > > > > > >> how that
> > > > > > > > > > > > > >> > > > > > will work
> > > > > > > > > > > > > >> > > > > > > > > > > >> though.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > It is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > basically
> a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > four
> > > > > step
> > > > > > > > > > process:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > 1. The
> > > > > > > > cluster
> > > > > > > > > > > must
> > > > > > > > > > > > be
> > > > > > > > > > > > > >> > > > upgraded or
> > > > > > > > > > > > > >> > > > > > > > > rolled to a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > software
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > supporting
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > new
> > > > > group
> > > > > > > > > > > > coordinator.
> > > > > > > > > > > > > >> Both the
> > > > > > > > > > > > > >> > > > > > old and
> > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> coordinator
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > will
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > support
> > > > > > > the
> > > > > > > > old
> > > > > > > > > > > > > >> protocol and
> > > > > > > > > > > > > >> > > > rely
> > > > > > > > > > > > > >> > > > > > on the
> > > > > > > > > > > > > >> > > > > > > > > same
> > > > > > > > > > > > > >> > > > > > > > > > > >> > persisted
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > metadata
> so
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > they
> > > > > can
> > > > > > > > work
> > > > > > > > > > > > together.
> > > > > > > > > > > > > >> This
> > > > > > > > > > > > > >> > > > point
> > > > > > > > > > > > > >> > > > > > is an
> > > > > > > > > > > > > >> > > > > > > > > > > >> offline
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> migration.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > We
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > cannot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > do
> > > > > this
> > > > > > > one
> > > > > > > > live
> > > > > > > > > > > > > >> because it
> > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > >> > > > > > require
> > > > > > > > > > > > > >> > > > > > > > > > > >> > shutting down
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > current
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > coordinator
> > > > > > > > and
> > > > > > > > > > > > > >> starting up the
> > > > > > > > > > > > > >> > > > > > new one
> > > > > > > > > > > > > >> > > > > > > > > and
> > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > would
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cause
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > unavailabilities.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > 2. The
> > > > > > > > cluster's
> > > > > > > > > > > > > >> metadata
> > > > > > > > > > > > > >> > > > > > version/IBP
> > > > > > > > > > > > > >> > > > > > > > > must be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> X
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > order
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > enable
> > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > >> protocol.
> > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > >> > > > > > cannot
> > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 1)
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > terminated
> > > > > > > > > > because
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> old
> > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > >> > > > > > > > > doesn't
> > > > > > > > > > > > > >> > > > > > > > > > > >> > support
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > protocol.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > 3. The
> > > > > > > > consumers
> > > > > > > > > > > > must be
> > > > > > > > > > > > > >> > > > upgraded
> > > > > > > > > > > > > >> > > > > > to a
> > > > > > > > > > > > > >> > > > > > > > > version
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > supporting
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > online
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > migration
> > > > > > > > (must
> > > > > > > > > > > have
> > > > > > > > > > > > > >> KIP-792).
> > > > > > > > > > > > > >> > > > If
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > already
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > there.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > Nothing
> > > > > > > > must be
> > > > > > > > > > > > done at
> > > > > > > > > > > > > >> this
> > > > > > > > > > > > > >> > > > point.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > 4. The
> > > > > > > > consumers
> > > > > > > > > > > > must
> > > > > > > > > > > > > >> be rolled
> > > > > > > > > > > > > >> > > > > > with the
> > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > >> > > > > > > > > > > >> > flag
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> turned
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > on.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > consumer
> > > > > > > > group
> > > > > > > > > > is
> > > > > > > > > > > > > >> automatically
> > > > > > > > > > > > > >> > > > > > > > > converted when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > using
> > > > > the
> > > > > > > > new
> > > > > > > > > > > > protocol
> > > > > > > > > > > > > >> joins
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > group.
> > > > > > > > > > > > > >> > > > > > > > > While
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > members
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > using the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > old
> > > > > > > > protocol are
> > > > > > > > > > > > being
> > > > > > > > > > > > > >> > > > upgraded,
> > > > > > > > > > > > > >> > > > > > the old
> > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > >> > > > > > > > > > > >> > is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> proxied
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > into
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > new
> > > > > one.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Let me
> > > > > > > > clarify
> > > > > > > > > > all
> > > > > > > > > > > > of
> > > > > > > > > > > > > >> this in
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > KIP.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 4.
> > > > > I am
> > > > > > > > happy
> > > > > > > > > > > > that we
> > > > > > > > > > > > > >> are
> > > > > > > > > > > > > >> > > > > > pushing the
> > > > > > > > > > > > > >> > > > > > > > > pattern
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> subscriptions
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > server,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > but
> > > > > it
> > > > > > > > seems
> > > > > > > > > > > like
> > > > > > > > > > > > > >> there
> > > > > > > > > > > > > >> > > > could be
> > > > > > > > > > > > > >> > > > > > some
> > > > > > > > > > > > > >> > > > > > > > > tricky
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> compatibility
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> issues.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> Will
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > have a
> > > > > > > > > > mechanism
> > > > > > > > > > > > for
> > > > > > > > > > > > > >> users to
> > > > > > > > > > > > > >> > > > > > detect
> > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > update
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > their
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> regex
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > before
> > > > > > > > > > switching
> > > > > > > > > > > > to
> > > > > > > > > > > > > >> the new
> > > > > > > > > > > > > >> > > > > > protocol?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I
> > > > > think
> > > > > > > > that I
> > > > > > > > > > am
> > > > > > > > > > > a
> > > > > > > > > > > > bit
> > > > > > > > > > > > > >> more
> > > > > > > > > > > > > >> > > > > > optimistic
> > > > > > > > > > > > > >> > > > > > > > > than
> > > > > > > > > > > > > >> > > > > > > > > > > >> you
> > > > > > > > > > > > > >> > > > > > > > > > > >> > on this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > point. I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > believe
> > > > > > > > that the
> > > > > > > > > > > > > >> majority of
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > cases
> > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > >> > > > > > > > > > > >> simple
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > regexes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > which
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > work
> > > > > with
> > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > >> engine. The
> > > > > > > > > > > > > >> > > > > > > > > coordinator will
> > > > > > > > > > > > > >> > > > > > > > > > > >> > verify
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > regex
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > anyway
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > and
> > > > > reject
> > > > > > > > the
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > >> if the
> > > > > > > > > > > > > >> > > > > > regex is
> > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > >> > > > > > > > > > > >> valid.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Coming
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > back
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > migration
> > > > > > > > path,
> > > > > > > > > > in
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> worst
> > > > > > > > > > > > > >> > > > case,
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> joining
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > the
> > > > > group
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > > > > >> rejected.
> > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > >> > > > > > should
> > > > > > > > > > > > > >> > > > > > > > > be used
> > > > > > > > > > > > > >> > > > > > > > > > > >> > as the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> last
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> defence, I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > would
> > > > > say.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > One
> > > > > way
> > > > > > > for
> > > > > > > > > > > > customers to
> > > > > > > > > > > > > >> > > > validate
> > > > > > > > > > > > > >> > > > > > their
> > > > > > > > > > > > > >> > > > > > > > > regex
> > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > upgrading
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > their
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > prod
> > > > > would
> > > > > > > > be to
> > > > > > > > > > > > test
> > > > > > > > > > > > > >> them with
> > > > > > > > > > > > > >> > > > > > another
> > > > > > > > > > > > > >> > > > > > > > > group.
> > > > > > > > > > > > > >> > > > > > > > > > > >> For
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instance,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > could
> > > > > be
> > > > > > > > done
> > > > > > > > > > in a
> > > > > > > > > > > > > >> pre-prod
> > > > > > > > > > > > > >> > > > > > environment.
> > > > > > > > > > > > > >> > > > > > > > > > > >> Another
> > > > > > > > > > > > > >> > > > > > > > > > > >> > way
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> would be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > extend the
> > > > > > > > > > > > > >> consumer-group tool
> > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > >> > > > > > > > > provide a
> > > > > > > > > > > > > >> > > > > > > > > > > >> regex
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validation
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > mechanism.
> > > > > > > > Would
> > > > > > > > > > > > this be
> > > > > > > > > > > > > >> > > > enough in
> > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > >> > > > > > > > > > > >> opinion?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 5.
> > > > > > > > Related to
> > > > > > > > > > > the
> > > > > > > > > > > > last
> > > > > > > > > > > > > >> > > > question,
> > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> Java
> > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> allow
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> users
> > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > stick
> > > > > > > > with the
> > > > > > > > > > > > > >> current regex
> > > > > > > > > > > > > >> > > > > > engine for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > compatibility
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > reasons?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > For
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > example,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > it
> > > > > may
> > > > > > > be
> > > > > > > > > > handy
> > > > > > > > > > > to
> > > > > > > > > > > > > >> keep using
> > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > >> > > > > > > > > based
> > > > > > > > > > > > > >> > > > > > > > > > > >> > regex at
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > keep
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > migrations
> > > > > > > > > > > simple
> > > > > > > > > > > > and
> > > > > > > > > > > > > >> then
> > > > > > > > > > > > > >> > > > > > migrate to
> > > > > > > > > > > > > >> > > > > > > > > server
> > > > > > > > > > > > > >> > > > > > > > > > > >> > based
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regexes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > as a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> second
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > step.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I
> > > > > > > understand
> > > > > > > > > > your
> > > > > > > > > > > > point
> > > > > > > > > > > > > >> but I
> > > > > > > > > > > > > >> > > > am
> > > > > > > > > > > > > >> > > > > > > > > concerned that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > would
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > allow
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > users
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > > actually
> > > > > > > > stay
> > > > > > > > > > > in
> > > > > > > > > > > > > >> this mode.
> > > > > > > > > > > > > >> > > > That
> > > > > > > > > > > > > >> > > > > > > > > would go
> > > > > > > > > > > > > >> > > > > > > > > > > >> > against our
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> goal
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > simplifying
> > > > > > > > the
> > > > > > > > > > > > client
> > > > > > > > > > > > > >> because
> > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > continue
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > monitoring
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > the
> > > > > > > > metadata on
> > > > > > > > > > > the
> > > > > > > > > > > > > >> client
> > > > > > > > > > > > > >> > > > side. I
> > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > >> > > > > > > > > rather
> > > > > > > > > > > > > >> > > > > > > > > > > >> > not do
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 6.
> > > > > When
> > > > > > > > we say
> > > > > > > > > > > > that
> > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > >> > > > > > > > > coordinator
> > > > > > > > > > > > > >> > > > > > > > > > > >> will
> > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> responsible
> > > for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > storing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > the
> > > > > > > > > > > configurations
> > > > > > > > > > > > > >> and that
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> configurations
> > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > deleted
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > group is
> > > > > > > > > > > deleted.
> > > > > > > > > > > > > >> Will a
> > > > > > > > > > > > > >> > > > > > transition to
> > > > > > > > > > > > > >> > > > > > > > > DEAD
> > > > > > > > > > > > > >> > > > > > > > > > > >> > trigger
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > deletion
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > configurations?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > That's
> > > > > > > > right.
> > > > > > > > > > The
> > > > > > > > > > > > > >> > > > configurations
> > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > >> > > > > > > > > > > >> deleted
> > > > > > > > > > > > > >> > > > > > > > > > > >> > when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > deleted.
> > > > > > > > They go
> > > > > > > > > > > > > >> together.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 7.
> > > > > Will
> > > > > > > > the
> > > > > > > > > > > > choice to
> > > > > > > > > > > > > >> store
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > configs in
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> coordinator
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> make
> > > it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > harder
> > > > > > > to
> > > > > > > > list
> > > > > > > > > > > all
> > > > > > > > > > > > > >> cluster
> > > > > > > > > > > > > >> > > > > > configs and
> > > > > > > > > > > > > >> > > > > > > > > their
> > > > > > > > > > > > > >> > > > > > > > > > > >> > values?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I
> > > > > don't
> > > > > > > > think
> > > > > > > > > > so.
> > > > > > > > > > > > The
> > > > > > > > > > > > > >> group
> > > > > > > > > > > > > >> > > > > > > > > configurations are
> > > > > > > > > > > > > >> > > > > > > > > > > >> > overrides
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > cluster
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > configs.
> > > > > > > If
> > > > > > > > you
> > > > > > > > > > > > want to
> > > > > > > > > > > > > >> know
> > > > > > > > > > > > > >> > > > all
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > overrides
> > > > > > > > > > > > > >> > > > > > > > > > > >> > though,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > you
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > would
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > have
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > ask all
> > > > > > > > the
> > > > > > > > > > > group
> > > > > > > > > > > > > >> > > > coordinators.
> > > > > > > > > > > > > >> > > > > > You
> > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > >> > > > > > > > > > > >> > rely on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > metadata
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > log
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > for
> > > > > > > > instance.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 8.
> > > > > How
> > > > > > > > would
> > > > > > > > > > > > someone
> > > > > > > > > > > > > >> > > > configure a
> > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > >> > > > > > > > > before
> > > > > > > > > > > > > >> > > > > > > > > > > >> > starting
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> consumers?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Have
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > we
> > > > > > > > considered
> > > > > > > > > > > > > >> allowing the
> > > > > > > > > > > > > >> > > > > > explicit
> > > > > > > > > > > > > >> > > > > > > > > creation
> > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > groups?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > Alternatively,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > configs
> > > > > > > > could
> > > > > > > > > > be
> > > > > > > > > > > > > >> decoupled
> > > > > > > > > > > > > >> > > > from
> > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > lifecycle.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Yes.
> > > > > The
> > > > > > > > group
> > > > > > > > > > > will
> > > > > > > > > > > > be
> > > > > > > > > > > > > >> > > > > > automatically
> > > > > > > > > > > > > >> > > > > > > > > created in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > However,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > the
> > > > > > > configs
> > > > > > > > will
> > > > > > > > > > > be
> > > > > > > > > > > > > >> lost after
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > retention
> > > > > > > > > > > > > >> > > > > > > > > > > >> > period of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > passes.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 9.
> > > > > Will
> > > > > > > > the
> > > > > > > > > > > > > >> > > > Consumer.subscribe
> > > > > > > > > > > > > >> > > > > > method
> > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> client
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > still
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > take
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > >> `java.util.regex.Pattern` of
> > > > > > > > > > > > > >> > > > do
> > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > introduce
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > overload?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > That's a
> > > > > > > > very
> > > > > > > > > > > group
> > > > > > > > > > > > > >> question. I
> > > > > > > > > > > > > >> > > > > > forgot
> > > > > > > > > > > > > >> > > > > > > > > about
> > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > one.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > As
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> `java.util.regex.Pattern` is
> > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > >> > > > > > fully
> > > > > > > > > > > > > >> > > > > > > > > > > >> compatible
> > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > engine
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > we
> > > > > plan to
> > > > > > > > use,
> > > > > > > > > > it
> > > > > > > > > > > > > >> might be
> > > > > > > > > > > > > >> > > > better
> > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > >> > > > > > > > > deprecate
> > > > > > > > > > > > > >> > > > > > > > > > > >> > it and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> use an
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> overload
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > which
> > > > > > > takes
> > > > > > > > a
> > > > > > > > > > > > string.
> > > > > > > > > > > > > >> We would
> > > > > > > > > > > > > >> > > > > > rely on
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> server
> > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> validation.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > During the
> > > > > > > > > > > > migration, I
> > > > > > > > > > > > > >> think
> > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > >> > > > > > > > > > > >> still
> > > > > > > > > > > > > >> > > > > > > > > > > >> > try to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > toString
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > regex
> > > > > and
> > > > > > > > use
> > > > > > > > > > it.
> > > > > > > > > > > > That
> > > > > > > > > > > > > >> should
> > > > > > > > > > > > > >> > > > > > work, I
> > > > > > > > > > > > > >> > > > > > > > > think, in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> majority
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > cases.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 10.
> > > > > I
> > > > > > > > agree
> > > > > > > > > > with
> > > > > > > > > > > > > >> Justine
> > > > > > > > > > > > > >> > > > that we
> > > > > > > > > > > > > >> > > > > > > > > should be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > clearer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > about
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > reason
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > switch
> > > > > > > to
> > > > > > > > > > > > > >> > > > IBP/metadata.version
> > > > > > > > > > > > > >> > > > > > from the
> > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > >> > > > > > > > > > > >> > flag.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Maybe
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > mean
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > can
> > > > > > > > switch the
> > > > > > > > > > > > > >> default for
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > feature
> > > > > > > > > > > > > >> > > > > > > > > flag
> > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > true
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> based
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > metadata.version
> > > > > > > > > > > > once
> > > > > > > > > > > > > >> we
> > > > > > > > > > > > > >> > > > want to
> > > > > > > > > > > > > >> > > > > > make
> > > > > > > > > > > > > >> > > > > > > > > it the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > default.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > My
> > > > > plan
> > > > > > > was
> > > > > > > > to
> > > > > > > > > > use
> > > > > > > > > > > > that
> > > > > > > > > > > > > >> feature
> > > > > > > > > > > > > >> > > > > > flag
> > > > > > > > > > > > > >> > > > > > > > > mainly
> > > > > > > > > > > > > >> > > > > > > > > > > >> > during the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> development
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > phase. I
> > > > > > > > should
> > > > > > > > > > > not
> > > > > > > > > > > > have
> > > > > > > > > > > > > >> > > > mentioned
> > > > > > > > > > > > > >> > > > > > it, I
> > > > > > > > > > > > > >> > > > > > > > > think,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > because
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > could
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > use
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > an
> > > > > > > internal
> > > > > > > > > > config
> > > > > > > > > > > > for
> > > > > > > > > > > > > >> it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 11.
> > > > > Some
> > > > > > > > of
> > > > > > > > > > the
> > > > > > > > > > > > > >> protocol APIs
> > > > > > > > > > > > > >> > > > > > don't
> > > > > > > > > > > > > >> > > > > > > > > mention
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > required
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ACLs, it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> would
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > good to
> > > > > > > > add
> > > > > > > > > > that
> > > > > > > > > > > > for
> > > > > > > > > > > > > >> > > > consistency.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Noted.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 12.
> > > > > It
> > > > > > > is
> > > > > > > > a
> > > > > > > > > > bit
> > > > > > > > > > > > odd
> > > > > > > > > > > > > >> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> ConsumerGroupHeartbeat
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > requires
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > "Read
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Group"
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > even
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > though
> > > > > > > it
> > > > > > > > > > seems
> > > > > > > > > > > > to do
> > > > > > > > > > > > > >> more
> > > > > > > > > > > > > >> > > > than
> > > > > > > > > > > > > >> > > > > > > > > reading.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I
> > > > > agree.
> > > > > > > > This is
> > > > > > > > > > > > how the
> > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > >> > > > > > > > > protocol works
> > > > > > > > > > > > > >> > > > > > > > > > > >> > though.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > We
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > only
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > require
> > > > > > > > "Read
> > > > > > > > > > > > Group" to
> > > > > > > > > > > > > >> join a
> > > > > > > > > > > > > >> > > > > > group. We
> > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > >> > > > > > > > > > > >> > consider
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > changing
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > this
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > but I
> > > > > am
> > > > > > > not
> > > > > > > > > > sure
> > > > > > > > > > > > that
> > > > > > > > > > > > > >> it is
> > > > > > > > > > > > > >> > > > worth
> > > > > > > > > > > > > >> > > > > > it.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 13.
> > > > > How
> > > > > > > is
> > > > > > > > > > topic
> > > > > > > > > > > > > >> recreation
> > > > > > > > > > > > > >> > > > > > handled by
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > consumer
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> group
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > protocol?
> > > > > > > > It
> > > > > > > > > > > > would be
> > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > >> > > > > > have a
> > > > > > > > > > > > > >> > > > > > > > > section
> > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > >> > > > > > > > > > > >> > this.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Noted.
> > > > > > > From
> > > > > > > > a
> > > > > > > > > > > > protocol
> > > > > > > > > > > > > >> > > > > > perspective, the
> > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > will
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> have a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > topic
> > > > > id
> > > > > > > so
> > > > > > > > it
> > > > > > > > > > > will
> > > > > > > > > > > > > >> treat it
> > > > > > > > > > > > > >> > > > like a
> > > > > > > > > > > > > >> > > > > > > > > topic with
> > > > > > > > > > > > > >> > > > > > > > > > > >> a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > different
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > name.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > The
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > only
> > > > > issue
> > > > > > > > is
> > > > > > > > > > that
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> > > > fetch/commit
> > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > >> > > > > > > > > > > >> APIs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > do not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > support
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > IDs
> > > > > so the
> > > > > > > > > > > consumer
> > > > > > > > > > > > > >> would
> > > > > > > > > > > > > >> > > > reuse the
> > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > >> > > > > > > > > > > >> based
> > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > same.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > that
> > > > > we
> > > > > > > > should
> > > > > > > > > > > > update
> > > > > > > > > > > > > >> those
> > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > >> > > > > > as well
> > > > > > > > > > > > > >> > > > > > > > > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> order
> > > > > > > > > > > > > >> > > > > > > > > > > >> > to be
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consistent
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > end
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > end.
> > > > > > > That
> > > > > > > > > > would
> > > > > > > > > > > > > >> strengthen
> > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > >> > > > > > > > > semantics of
> > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consumer.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 14.
> > > > > The
> > > > > > > > KIP
> > > > > > > > > > > > mentions
> > > > > > > > > > > > > >> we will
> > > > > > > > > > > > > >> > > > > > write the
> > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > coordinator
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Java.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Even
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > though
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > this is
> > > > > > > an
> > > > > > > > > > > > > >> implementation
> > > > > > > > > > > > > >> > > > > > detail, do
> > > > > > > > > > > > > >> > > > > > > > > we plan
> > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > have a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > gradle
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> module
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > for
> > > > > it?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Yes.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 15.
> > > > > Do
> > > > > > > we
> > > > > > > > > > have a
> > > > > > > > > > > > > >> scalability
> > > > > > > > > > > > > >> > > > > > goal when
> > > > > > > > > > > > > >> > > > > > > > > it
> > > > > > > > > > > > > >> > > > > > > > > > > >> comes
> > > > > > > > > > > > > >> > > > > > > > > > > >> > to how
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> many
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> members
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > new
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > group
> > > > > > > > protocol
> > > > > > > > > > > can
> > > > > > > > > > > > > >> support?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > We
> > > > > don't
> > > > > > > > have
> > > > > > > > > > > > numbers
> > > > > > > > > > > > > >> at the
> > > > > > > > > > > > > >> > > > > > moment. The
> > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > support
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > 1000s
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > of
> > > > > members
> > > > > > > > per
> > > > > > > > > > > > group.
> > > > > > > > > > > > > >> We will
> > > > > > > > > > > > > >> > > > > > measure
> > > > > > > > > > > > > >> > > > > > > > > this when
> > > > > > > > > > > > > >> > > > > > > > > > > >> > we have
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > first
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > implementation.
> > > > > > > > > > > Note
> > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > >> > > > might
> > > > > > > > > > > > > >> > > > > > have
> > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > bottlenecks
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > down
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > road
> > > > > (e.g.
> > > > > > > > > > offset
> > > > > > > > > > > > > >> commits).
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > 16.
> > > > > Did
> > > > > > > we
> > > > > > > > > > > > consider
> > > > > > > > > > > > > >> having
> > > > > > > > > > > > > >> > > > > > > > > SubscribedTopidIds
> > > > > > > > > > > > > >> > > > > > > > > > > >> > instead
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > SubscribedTopicNames
> > > > > > > > > > > > > >> in
> > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > ConsumerGroupHeartbeatRequest?
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > idea
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> that
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > since we
> > > > > > > > have
> > > > > > > > > > to
> > > > > > > > > > > > > >> resolve the
> > > > > > > > > > > > > >> > > > > > regex on
> > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > server, we
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> can do
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > the
> > > > > > > topic
> > > > > > > > > > name?
> > > > > > > > > > > > The
> > > > > > > > > > > > > >> > > > difference
> > > > > > > > > > > > > >> > > > > > is that
> > > > > > > > > > > > > >> > > > > > > > > > > >> sending
> > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regex is
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > more
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > efficient
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > whereas
> > > > > > > > > > sending
> > > > > > > > > > > > the
> > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > >> > > > > > is less
> > > > > > > > > > > > > >> > > > > > > > > > > >> > efficient.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> Furthermore,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> delete
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > recreation is
> > > > > > > > > > > > easier
> > > > > > > > > > > > > >> to
> > > > > > > > > > > > > >> > > > handle
> > > > > > > > > > > > > >> > > > > > if we
> > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > ids.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > The
> > > > > idea
> > > > > > > > was to
> > > > > > > > > > > > > >> consolidate the
> > > > > > > > > > > > > >> > > > > > metadata
> > > > > > > > > > > > > >> > > > > > > > > lookup
> > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> server
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > for
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > paths
> > > > > but
> > > > > > > I
> > > > > > > > do
> > > > > > > > > > > agree
> > > > > > > > > > > > > >> with your
> > > > > > > > > > > > > >> > > > > > point. As
> > > > > > > > > > > > > >> > > > > > > > > a
> > > > > > > > > > > > > >> > > > > > > > > > > >> second
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > though,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > ids
> > > > > may be
> > > > > > > > > > better
> > > > > > > > > > > > here
> > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > >> > > > > > delete and
> > > > > > > > > > > > > >> > > > > > > > > > > >> > recreation
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Also, I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > suppose
> > > > > > > > that we
> > > > > > > > > > > may
> > > > > > > > > > > > > >> allow
> > > > > > > > > > > > > >> > > > users to
> > > > > > > > > > > > > >> > > > > > > > > subscribe
> > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > >> > > > > > > > > > > >> > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> ids
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > future
> > > > > > > > because
> > > > > > > > > > > that
> > > > > > > > > > > > is
> > > > > > > > > > > > > >> the only
> > > > > > > > > > > > > >> > > > > > way to be
> > > > > > > > > > > > > >> > > > > > > > > > > >> really
> > > > > > > > > > > > > >> > > > > > > > > > > >> > robust
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > re-creation.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > Best,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > David
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > On
> > > > > Tue,
> > > > > > > Jul
> > > > > > > > 12,
> > > > > > > > > > > > 2022 at
> > > > > > > > > > > > > >> 1:38 PM
> > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > >> > > > > > > > > Jacot <
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > djacot@confluent.io>
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> wrote:
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > Hi
> > > > > > > > Justine,
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > Thanks
> > > > > > > for
> > > > > > > > > > your
> > > > > > > > > > > > > >> comments.
> > > > > > > > > > > > > >> > > > Please
> > > > > > > > > > > > > >> > > > > > find
> > > > > > > > > > > > > >> > > > > > > > > my
> > > > > > > > > > > > > >> > > > > > > > > > > >> answers
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > below.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > -
> > > > > Yes,
> > > > > > > > the new
> > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > >> > > > relies on
> > > > > > > > > > > > > >> > > > > > > > > topic IDs
> > > > > > > > > > > > > >> > > > > > > > > > > >> > with the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > exception
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > of the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > topic
> > > > > > > > names
> > > > > > > > > > > based
> > > > > > > > > > > > in
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > ConsumerGroupHeartbeatRequest.
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > am
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > sure
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > if
> > > > > using
> > > > > > > > topic
> > > > > > > > > > > > names
> > > > > > > > > > > > > >> is the
> > > > > > > > > > > > > >> > > > > > right call
> > > > > > > > > > > > > >> > > > > > > > > here.
> > > > > > > > > > > > > >> > > > > > > > > > > >> I
> > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > about
> it
> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > a
> > > > > little
> > > > > > > > more.
> > > > > > > > > > > > > >> Obviously, the
> > > > > > > > > > > > > >> > > > > > KIP does
> > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > >> > > > > > > > > > > >> > change the
> > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > >
> >
> >
> > --
> > -- Guozhang
>
--
-- Guozhang
Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol
Posted by David Jacot <dj...@confluent.io.INVALID>.
Hi Guozhang,
Thanks. I have updated those assignor interfaces to not include topic
ids. I agree with you that it makes more sense to not expose them at
the moment. This also keeps the Consumer API consistent as we don't
expose them anywhere else.
Best,
David
On Tue, Oct 4, 2022 at 6:30 PM Guozhang Wang <wa...@gmail.com> wrote:
>
> Hi David,
>
> I made another pass on the doc, and assuming we will change the
> "TopicIdPartition" fields to "TopicPartition" for the consumer-side APIs, I
> think this proposal looks great for me.
>
> Guozhang
>
> On Mon, Oct 3, 2022 at 9:36 AM Guozhang Wang <wa...@gmail.com> wrote:
>
> > 3. Ah thanks, that's my mis-read then. I think keeping the current class
> > and the params of `onAssignment` is fine then.
> >
> > On Mon, Oct 3, 2022 at 4:32 AM David Jacot <dj...@confluent.io.invalid>
> > wrote:
> >
> >> 3. ConsumerGroupMetadata is already defined in the consumer [1]. I
> >> realized that we pass it in the current assignor so I thought that we
> >> could reuse it. The alternative would be to add the same information
> >> in a class which includes the error and the member assignment as well.
> >>
> >> [1]
> >> https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerGroupMetadata.java
> >>
> >> Best,
> >> David
> >>
> >> On Mon, Oct 3, 2022 at 6:15 AM Guozhang Wang <wa...@gmail.com> wrote:
> >> >
> >> > 1. Yeah that makes sense. I originally concerned if there's any
> >> consistency
> >> > issues if the HB request with encoded topic IDs and the GPA/GIA RPCs
> >> with
> >> > encoded topic IDs use mappings from two different brokers, and one of
> >> them
> >> > is stale, but after thinking twice about it I think as long as we handle
> >> > e.g. `unknown topic ids` from HB responses then we should be fine.
> >> >
> >> > 3. Ack, in that case I think we also need to define
> >> "ConsumerGroupMetadata"
> >> > which is passed in `onAssignment`. We have a schema defined in the
> >> > persistent storage for different purposes, but not in the consumer APIs
> >> > yet. I think that would just include a single "Map<String, Metadata>"?
> >> >
> >> > On Fri, Sep 30, 2022 at 5:51 AM David Jacot <djacot@confluent.io.invalid
> >> >
> >> > wrote:
> >> >
> >> > > Hey Guozhang,
> >> > >
> >> > > Thanks for your comments.
> >> > >
> >> > > 1. That's a very good question. I think that it is better to separate
> >> > > the two concerns here. In my opinion, the GPA should only be used to
> >> > > feed the assignor. I see the GPA as a way to call a remote function so
> >> > > the GPA provides all its inputs to ensure that the assignor does not
> >> > > depend on any local state. Knowing this, I would not use the GPA to
> >> > > update the local state either. That is not its purpose. The consumer
> >> > > is responsible to update its local metadata cache either when it gets
> >> > > an unknown topic in the user-provided subscription or in the
> >> > > coordinator-provided assignment. The consumer has to do this anyway to
> >> > > get all topic metadata (e.g. leaders).
> >> > >
> >> > > 2. Interesting thought. Handling the translation is definitely
> >> > > possible. I wonder if there are cases where having the topic ids is
> >> > > beneficial. I am thinking about when a topic is deleted and recreated
> >> > > right away. In this case, the current member assignment would still
> >> > > have the old topic id and we would not be able to resolve it anymore.
> >> > > I suppose that we should directly remove it from the current
> >> > > assignment on the server side in this case and only add the new one.
> >> > > As I first thought, that seems OK. The major downside is that the user
> >> > > of the assignor would not know that the topic is different from what
> >> > > it may have cached locally.
> >> > >
> >> > > We could say that we don't expose topic ids externally for now in
> >> > > order to stay consistent with the other interfaces but it would be
> >> > > annoying to add it back if we want to expose topic ids in the future.
> >> > > I personally believe that we will have to expose them at some point.
> >> > > For instance, in the rebalance listener, it would be useful to know
> >> > > that a topic has been deleted and recreated. The only way to do this
> >> > > is to know the topic id. Let me think a little more about this and
> >> > > come back to you.
> >> > >
> >> > > 3. Sorry for this one. I was a typo. PartitionAssingor#assign should
> >> > > return GroupAssignment.
> >> > >
> >> > > Best,
> >> > > David
> >> > >
> >> > > On Thu, Sep 29, 2022 at 6:06 PM Guozhang Wang <wa...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > Thanks David, I read through the updated wiki, and I have a couple
> >> > > > follow-up questions:
> >> > > >
> >> > > > 1. It seems we would have two potential sources to update the
> >> consumer's
> >> > > > internally maintained topic name -> topic ID mapping, which is used
> >> for
> >> > > > encoding subscribed topic Ids in the HB request, and offset commit
> >> / list
> >> > > > offset requests, etc. One is from metadata responses, and another
> >> one is
> >> > > > from the GPA (group prepare assignment). On the other hand, the
> >> assignor
> >> > > > APIs directly take parsed AssignmentMemberSpec and
> >> > > AssignmentTopicMetadata
> >> > > > from GPA responses directly instead of trying to consult the
> >> consumer's
> >> > > own
> >> > > > cached mapping. So just to clarify, consumers would also update its
> >> > > > internal mapping from the GPA responses as well, right?
> >> > > >
> >> > > > 2. I'm wondering if it's necessary to expose the topic IDs to the
> >> > > assignor
> >> > > > developers. What they need to do at the moment is to take the
> >> mapping
> >> > > from
> >> > > > AssignmentTopicMetadata, and translate their decided assignment into
> >> > > topic
> >> > > > IDs when returning the MemberAssignment. We can instead just let
> >> consumer
> >> > > > to do such translation for them internally and still let the
> >> assignor
> >> > > > developers to only deal with the topic names. Of course that means
> >> the
> >> > > > client side assignor APIs would be a bit further diverged from the
> >> > > > broker-side assignor such that:
> >> > > >
> >> > > > a) Topic metadata would be encoded just as String -> num.partitions;
> >> > > > b) AssignmentMemberSpec contains subscribedTopicNames and
> >> topicPartitions
> >> > > > as target assignments;
> >> > > > c) Returned MemberAssignment contains list of topicPartitions
> >> instead of
> >> > > > topicIdPartitions.
> >> > > >
> >> > > > Internally, consumers would convert the IDs to names when calling
> >> the
> >> > > > assign() function, and convert them back when sending the request;
> >> and
> >> > > > similarly it would convert the IDs to names when calling
> >> onAssignment()
> >> > > as
> >> > > > well.
> >> > > >
> >> > > > In short, I'm wondering if we should bite the bullet for the users,
> >> > > eating
> >> > > > more complexities internally than pushing that to users.
> >> > > >
> >> > > > 3. This is a nit one: the return type of `Assignment
> >> > > assign(AssignmentSpec
> >> > > > assignmentSpec);` seems not defined in the doc. I think it would
> >> just
> >> > > > include two fields, as:
> >> > > >
> >> > > > class Assignment {
> >> > > > GroupAssignment groupAssignment;
> >> > > >
> >> > > > ConsumerGroupMetadata consumerGroupMetadata;
> >> > > > }
> >> > > >
> >> > > > Is that right?
> >> > > >
> >> > > > Guozhang
> >> > > >
> >> > > > On Thu, Sep 29, 2022 at 1:26 AM David Jacot
> >> <djacot@confluent.io.invalid
> >> > > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Guozhang,
> >> > > > >
> >> > > > > Thanks for your input.
> >> > > > >
> >> > > > > > 1) in HB request, when clients send the subscription, they are
> >> still
> >> > > > > > sending them in terms of topic names. That is, the consumer
> >> would
> >> > > not try
> >> > > > > > to keep all the mappings from topic names to topic ids for the
> >> > > subscribed
> >> > > > > > topics.
> >> > > > >
> >> > > > > This is a mistake of mine. The heartbeat request should use Topic
> >> Ids.
> >> > > > > I forgot to change it. The idea is that the client will get the
> >> > > > > metadata first to translate the topic names to topic ids and thus
> >> also
> >> > > > > have the mapping in its local metadata cache.
> >> > > > >
> >> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> >> > > already.
> >> > > > > And
> >> > > > > > fetch requests can directly rely on that since they just need
> >> topic
> >> > > IDs
> >> > > > > and
> >> > > > > > do not care what actual topic names are.
> >> > > > >
> >> > > > > That's right.
> >> > > > >
> >> > > > > > 3) in prepare-assignment responses, we return subscriptions as
> >> topic
> >> > > IDs
> >> > > > > as
> >> > > > > > well.
> >> > > > >
> >> > > > > That's right.
> >> > > > >
> >> > > > > > In a word, the consumer does not try to maintain the mapping of
> >> topic
> >> > > > > names
> >> > > > > > to IDs at all, and even if they do, they will not expose it in
> >> e.g.
> >> > > > > > assignor APIs. But for client-side assignors like Streams, if
> >> they
> >> > > do not
> >> > > > > > maintain the mapping from topic IDs to topic names, certain
> >> logic
> >> > > such as
> >> > > > > > co-partitioning indeed cannot be executed.
> >> > > > >
> >> > > > > As I said, the consumer will have the mapping in its local
> >> metadata
> >> > > > > cache. It needs to get the partition metadata anyway and the
> >> topic id
> >> > > > > is part of it. However, I do agree that this mapping is not
> >> exposed.
> >> > > > >
> >> > > > > > So I think we'd need to consider 1) use topic names in
> >> subscriptions
> >> > > in
> >> > > > > the
> >> > > > > > prepare-assignment responses, 2) use topic Names in client-side
> >> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
> >> > > > > > PartitionAssignor#TopicMetadata#topicName.
> >> > > > >
> >> > > > > 1) makes sense to me. I think that we could actually provide both
> >> > > > > names and ids. Regarding 2), I think that I am slightly in favor
> >> of
> >> > > > > using both names and ids as well. Let me try to update the
> >> protocol
> >> > > > > and the interface towards this.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > David
> >> > > > >
> >> > > > > On Wed, Sep 28, 2022 at 9:35 PM Guozhang Wang <wangguoz@gmail.com
> >> >
> >> > > wrote:
> >> > > > > >
> >> > > > > > Hello Jun/David,
> >> > > > > >
> >> > > > > > Just to chime in here for Jun's comment #50, today Streams does
> >> not
> >> > > have
> >> > > > > a
> >> > > > > > way to map topic names to topic IDs, and in its topology
> >> description
> >> > > > > topics
> >> > > > > > are encoded with names.
> >> > > > > >
> >> > > > > > What I originally thought was that, on the consumer side, we'd
> >> need
> >> > > to
> >> > > > > > maintain this mapping from topic names to topic IDs anyways, and
> >> > > Streams
> >> > > > > > should leverage on the consumer's functionality to get the
> >> > > corresponding
> >> > > > > > topic names for the given topic IDs. But when I read through
> >> the RPC
> >> > > > > again
> >> > > > > > I realized it may not be the case, since:
> >> > > > > >
> >> > > > > > 1) in HB request, when clients send the subscription, they are
> >> still
> >> > > > > > sending them in terms of topic names. That is, the consumer
> >> would
> >> > > not try
> >> > > > > > to keep all the mappings from topic names to topic ids for the
> >> > > subscribed
> >> > > > > > topics.
> >> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> >> > > already.
> >> > > > > And
> >> > > > > > fetch requests can directly rely on that since they just need
> >> topic
> >> > > IDs
> >> > > > > and
> >> > > > > > do not care what actual topic names are.
> >> > > > > > 3) in prepare-assignment responses, we return subscriptions as
> >> topic
> >> > > IDs
> >> > > > > as
> >> > > > > > well.
> >> > > > > >
> >> > > > > > In a word, the consumer does not try to maintain the mapping of
> >> topic
> >> > > > > names
> >> > > > > > to IDs at all, and even if they do, they will not expose it in
> >> e.g.
> >> > > > > > assignor APIs. But for client-side assignors like Streams, if
> >> they
> >> > > do not
> >> > > > > > maintain the mapping from topic IDs to topic names, certain
> >> logic
> >> > > such as
> >> > > > > > co-partitioning indeed cannot be executed.
> >> > > > > >
> >> > > > > > So I think we'd need to consider 1) use topic names in
> >> subscriptions
> >> > > in
> >> > > > > the
> >> > > > > > prepare-assignment responses, 2) use topic Names in client-side
> >> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
> >> > > > > > PartitionAssignor#TopicMetadata#topicName.
> >> > > > > >
> >> > > > > > On Wed, Sep 28, 2022 at 1:26 AM David Jacot
> >> > > <djacot@confluent.io.invalid
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Jun,
> >> > > > > > >
> >> > > > > > > Thanks for your comments.
> >> > > > > > >
> >> > > > > > > 48. Okay. There is a typo in the text:
> >> > > > > > > > Note that the persisted target assignment of B already
> >> includes
> >> > > foo-2
> >> > > > > > > here
> >> > > > > > > but B does not get it yet because foo-2 is not free.
> >> > > > > > >
> >> > > > > > > It should be:
> >> > > > > > > > Note that the persisted member assignment of B already
> >> includes
> >> > > foo-2
> >> > > > > > > here
> >> > > > > > > but B does not get it yet because foo-2 is not free.
> >> > > > > > >
> >> > > > > > > The examples are showing the persisted state in the
> >> > > __consumer_offsets
> >> > > > > > > topics. When a member transitions to the next epoch, we
> >> directly
> >> > > > > > > persist the whole target assignment in order to reduce the
> >> number
> >> > > of
> >> > > > > > > writes and we manage dependencies in memory. So "B - epoch=2,
> >> > > > > > > partitions=[foo-2]" is actually correct from a persisted
> >> record
> >> > > > > > > perspective. However, the in-memory state would be different
> >> > > because
> >> > > > > > > foo-2 is not yet available as you pointed out. The in-memory
> >> state
> >> > > > > > > would like as follow:
> >> > > > > > >
> >> > > > > > > Group (epoch=2)
> >> > > > > > > A
> >> > > > > > > B
> >> > > > > > > Target Assignment (epoch=2)
> >> > > > > > > A - partitions=[foo-0, foo-1]
> >> > > > > > > B - partitions=[foo-2]
> >> > > > > > > Member Assignment
> >> > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2], pending
> >> > > partitions=[]
> >> > > > > > > B - epoch=2, partitions=[], pending partitions=[foo-2]
> >> > > > > > >
> >> > > > > > > I suppose that the confusion comes from this. I will go and
> >> convert
> >> > > > > > > all the examples to their in-memory representation in order to
> >> > > reduce
> >> > > > > > > the confusion.
> >> > > > > > >
> >> > > > > > > 50. Okay. I have to admit that I am not so familiar with the
> >> > > internals
> >> > > > > > > of KStreams. If I understand correctly, you're saying that
> >> KStreams
> >> > > > > > > may not have topic ids in its topology so topic names are
> >> missing
> >> > > here
> >> > > > > > > to match the topics. I was hoping that KStreams would have
> >> topic
> >> > > ids
> >> > > > > > > in the topology as well. If that is not the case, we can also
> >> add
> >> > > > > > > topic names on our side. I will check this.
> >> > > > > > >
> >> > > > > > > 53. It seems that I was not clear. Sorry for that. I did not
> >> mean
> >> > > to
> >> > > > > > > say that MemberAssignment.metadata is not used by the group
> >> > > > > > > coordinator. Let me re-explain the flow.
> >> > > > > > >
> >> > > > > > > PartitionAssignor.assign() returns an Assignment. Assignment
> >> > > contains
> >> > > > > > > MemberAssignments where each member has
> >> MemberAssignment.metadata.
> >> > > The
> >> > > > > > > Assignment is installed by the client with the
> >> > > > > > > ConsumerGroupInstallAssignment API and this overrides the
> >> current
> >> > > > > > > target assignment in the coordinator. Then the group
> >> coordinator
> >> > > uses
> >> > > > > > > it to reconcile members. Therefore the target assignment will
> >> > > > > > > eventually become the current member assignment and it will be
> >> > > > > > > delivered to the member, including the original
> >> > > > > > > MemberAssignment.metadata defines by the assignor.
> >> > > > > > >
> >> > > > > > > I hope that this helps to clarify how
> >> MemberAssignment.metadata is
> >> > > > > > > used afterwards.
> >> > > > > > >
> >> > > > > > > Best,
> >> > > > > > > David
> >> > > > > > >
> >> > > > > > > On Tue, Sep 27, 2022 at 10:31 PM Jun Rao
> >> <jun@confluent.io.invalid
> >> > > >
> >> > > > > wrote:
> >> > > > > > > > Hi, David,
> >> > > > > > > >
> >> > > > > > > > Thanks for the reply. A few more follow up questions.
> >> > > > > > > >
> >> > > > > > > > 48. Hmm, so member assignment is what the client sees.
> >> Then, I
> >> > > am not
> >> > > > > > > sure
> >> > > > > > > > about the following. The text says B cannot get foo-2 yet.
> >> But in
> >> > > > > Member
> >> > > > > > > > Assignment, B has foo-2.
> >> > > > > > > >
> >> > > > > > > > At this point B can transition to epoch 2 but cannot get
> >> foo-2
> >> > > until
> >> > > > > A
> >> > > > > > > > revokes it.
> >> > > > > > > >
> >> > > > > > > > Note that the persisted target assignment of B already
> >> includes
> >> > > foo-2
> >> > > > > > > here
> >> > > > > > > > but B does not get it yet because foo-2 is not free.
> >> > > > > > > >
> >> > > > > > > > Group (epoch=2)
> >> > > > > > > > A
> >> > > > > > > > B
> >> > > > > > > > Target Assignment (epoch=2)
> >> > > > > > > > A - partitions=[foo-0, foo-1]
> >> > > > > > > > B - partitions=[foo-2]
> >> > > > > > > > Member Assignment
> >> > > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2]
> >> > > > > > > > B - epoch=2, partitions=[foo-2]
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > 50. Hmm, I thought KStreams doesn't use range assignor all
> >> the
> >> > > time.
> >> > > > > It
> >> > > > > > > > does the assignment based on the topology. For example, if
> >> you
> >> > > are
> >> > > > > > > joining
> >> > > > > > > > two topics A and B and deriving the join result in a 3rd
> >> topic C,
> >> > > > > which
> >> > > > > > > > will be aggregated. You want to use range to co-locate the
> >> > > partitions
> >> > > > > > > from
> >> > > > > > > > topic A and B. However, for topic C, the partition could
> >> > > potentially
> >> > > > > be
> >> > > > > > > > assigned in round robin. Without the topic name, KStreams
> >> won't
> >> > > know
> >> > > > > > > which
> >> > > > > > > > topics need to be co-located.
> >> > > > > > > >
> >> > > > > > > > 53. If MemberAssignment.metadata is not used by the group
> >> > > > > coordinator,
> >> > > > > > > > where is it being used?
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > >
> >> > > > > > > > Jun
> >> > > > > > > >
> >> > > > > > > > On Tue, Sep 27, 2022 at 1:20 PM Jason Gustafson
> >> > > > > > > <ja...@confluent.io.invalid>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi David,
> >> > > > > > > > >
> >> > > > > > > > > Thanks for the response. Replied below:
> >> > > > > > > > >
> >> > > > > > > > > > I don't have a strong opinion on this one. However, it
> >> seems
> >> > > that
> >> > > > > > > this
> >> > > > > > > > > could be a pandora box if our subset turns out to not be
> >> > > sufficient
> >> > > > > > > > > for some existing users. We will have to add more to it
> >> and the
> >> > > > > > > > > compatibility story between versions could be a real
> >> > > nightmare. One
> >> > > > > > > > > thing that we could do for sure is disabling all unicode
> >> > > groups in
> >> > > > > the
> >> > > > > > > > > library. That seems possible and would already reduce the
> >> > > scope.
> >> > > > > > > > >
> >> > > > > > > > > Changes to the regex syntax seems not a major problem to
> >> me. We
> >> > > > > would
> >> > > > > > > bump
> >> > > > > > > > > the protocol version as we usually do. Actually that begs
> >> the
> >> > > same
> >> > > > > > > question
> >> > > > > > > > > about RE2. How stable is the syntax and does it differ
> >> > > depending
> >> > > > > on the
> >> > > > > > > > > version of the library? In general, I do agree that we'd
> >> be
> >> > > better
> >> > > > > off
> >> > > > > > > not
> >> > > > > > > > > having to manage the syntax ourselves if we can convince
> >> > > ourselves
> >> > > > > that
> >> > > > > > > > > compatibility won't be a problem and the client won't
> >> > > eventually
> >> > > > > need
> >> > > > > > > to
> >> > > > > > > > > have access to it.
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > Jason
> >> > > > > > > > >
> >> > > > > > > > > On Tue, Sep 27, 2022 at 6:21 AM David Jacot
> >> > > > > > > <dj...@confluent.io.invalid>
> >> > > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Hi Luke,
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks for your comments.
> >> > > > > > > > > >
> >> > > > > > > > > > Your suggestion makes sense. Jun's made the same one.
> >> We are
> >> > > also
> >> > > > > > > > > > discussing whether we should directly return the group
> >> state
> >> > > in
> >> > > > > the
> >> > > > > > > > > > response and drop the prepare assignment api. It seems
> >> that
> >> > > the
> >> > > > > > > > > > current consensus is to keep it though. If that stays
> >> like
> >> > > this,
> >> > > > > I
> >> > > > > > > > > > will apply your suggestion.
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > David
> >> > > > > > > > > >
> >> > > > > > > > > > On Tue, Sep 27, 2022 at 10:40 AM Luke Chen <
> >> > > showuon@gmail.com>
> >> > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > Hi David,
> >> > > > > > > > > > >
> >> > > > > > > > > > > Thanks for your patience for answering all these
> >> questions!
> >> > > > > > > > > > >
> >> > > > > > > > > > > One more question from me:
> >> > > > > > > > > > > 1. When the selected consumer is going to be in
> >> client-side
> >> > > > > > > assigning,
> >> > > > > > > > > we
> >> > > > > > > > > > > will return COMPUTE_ASSIGNMENT in heartbeat response.
> >> I'm
> >> > > > > > > wondering if
> >> > > > > > > > > we
> >> > > > > > > > > > > could use a flag to indicate the assignment is
> >> required.
> >> > > The
> >> > > > > > > rationale
> >> > > > > > > > > is
> >> > > > > > > > > > > when returning error, we lose a chance to catch up
> >> with the
> >> > > > > latest
> >> > > > > > > > > target
> >> > > > > > > > > > > assignment or revoke partitions. Since we expected the
> >> > > record
> >> > > > > > > fetching
> >> > > > > > > > > is
> >> > > > > > > > > > > not blocked by the rebalance/assignment computing, we
> >> might
> >> > > > > want to
> >> > > > > > > > > allow
> >> > > > > > > > > > > the members to update the member assignment in each
> >> > > heartbeat.
> >> > > > > > > > > > >
> >> > > > > > > > > > > Luke
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Tue, Sep 27, 2022 at 9:43 AM Jason Gustafson
> >> > > > > > > > > > <ja...@confluent.io.invalid>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Hey David,
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Thanks for the response.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > > Do you think that we could have a case where two
> >> > > members
> >> > > > > with
> >> > > > > > > the
> >> > > > > > > > > > same
> >> > > > > > > > > > > > static id are active?
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I was just thinking about the handoff when we spin
> >> up a
> >> > > new
> >> > > > > node
> >> > > > > > > > > > before the
> >> > > > > > > > > > > > old one completes shutdown. It's probably ok to
> >> fence
> >> > > the old
> >> > > > > > > member
> >> > > > > > > > > > > > abruptly. I do recall we had a few issues with this
> >> > > handoff
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > > > past. It
> >> > > > > > > > > > > > was due to the fact that the replaced node might
> >> have
> >> > > been
> >> > > > > the
> >> > > > > > > > > leader.
> >> > > > > > > > > > > > Since we've removed the leader notion, perhaps this
> >> kind
> >> > > of
> >> > > > > issue
> >> > > > > > > > > won't
> >> > > > > > > > > > > > resurface.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > In regard to the RE2 library. I understand the
> >> intent to
> >> > > pass
> >> > > > > > > this
> >> > > > > > > > > > through
> >> > > > > > > > > > > > as a string without any validation so that we don't
> >> need
> >> > > the
> >> > > > > > > > > > dependence on
> >> > > > > > > > > > > > the client. I do wonder if that will be restrictive
> >> in
> >> > > > > practice.
> >> > > > > > > For
> >> > > > > > > > > > > > example, if a regex changes on the client side,
> >> would we
> >> > > > > need to
> >> > > > > > > > > > validate
> >> > > > > > > > > > > > against the current assignment to see which
> >> partitions to
> >> > > > > > > continue
> >> > > > > > > > > > > > fetching? Or would we instead invalidate the
> >> complete
> >> > > current
> >> > > > > > > > > > assignment?
> >> > > > > > > > > > > > Perhaps this is a rare case and we can do the
> >> latter.
> >> > > There
> >> > > > > > > could be
> >> > > > > > > > > > more
> >> > > > > > > > > > > > situations though.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > As an alternative, I wonder if we could define a
> >> > > minimalistic
> >> > > > > > > regex
> >> > > > > > > > > > syntax
> >> > > > > > > > > > > > (a subset of RE2 say) that could map more easily to
> >> regex
> >> > > > > support
> >> > > > > > > > > that
> >> > > > > > > > > > > > already exists in clients? For example, in this
> >> > > scenario, we
> >> > > > > are
> >> > > > > > > > > > matching
> >> > > > > > > > > > > > topics, so we only need to be able to represent
> >> topic
> >> > > > > characters
> >> > > > > > > and
> >> > > > > > > > > > not
> >> > > > > > > > > > > > the whole smorgasbord of whitespace and unicode
> >> options.
> >> > > > > Would it
> >> > > > > > > > > make
> >> > > > > > > > > > > > sense to have a reduced syntax which includes only
> >> > > standard
> >> > > > > topic
> >> > > > > > > > > > > > characters, wildcards, character classes, and
> >> > > repetitions?
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Best,
> >> > > > > > > > > > > > Jason
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > On Thu, Sep 22, 2022 at 9:02 AM Guozhang Wang <
> >> > > > > > > wangguoz@gmail.com>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > > Hi David, thanks for all the detailed
> >> explanations. I
> >> > > think
> >> > > > > > > they
> >> > > > > > > > > all
> >> > > > > > > > > > make
> >> > > > > > > > > > > > > sense. Just want to have a couple follow-ups here:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > > I don't really see the benefits here because
> >> server
> >> > > side
> >> > > > > > > > > assignors
> >> > > > > > > > > > > > > don't have metadata at all. They only assign
> >> > > > > topic-partitions.
> >> > > > > > > They
> >> > > > > > > > > > > > > are not supposed to generate metadata nor to
> >> receive
> >> > > > > metadata
> >> > > > > > > from
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > members.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Just to clarify I was asking about the `version`
> >> of the
> >> > > > > > > assignor
> >> > > > > > > > > > (i.e. up
> >> > > > > > > > > > > > > to what version that the client would support),
> >> and I
> >> > > do
> >> > > > > agree
> >> > > > > > > we
> >> > > > > > > > > > would
> >> > > > > > > > > > > > not
> >> > > > > > > > > > > > > need metadata. What I have in mind is that, for
> >> some
> >> > > > > specific
> >> > > > > > > > > > built-in
> >> > > > > > > > > > > > > broker-assignors, e.g. rack-aware assignors, if
> >> it's
> >> > > > > possible
> >> > > > > > > that
> >> > > > > > > > > > in a
> >> > > > > > > > > > > > > newer version we would have a hierarchical rack ID
> >> > > string
> >> > > > > > > format,
> >> > > > > > > > > > like
> >> > > > > > > > > > > > > "tier1-tier2" etc, but if some client has not
> >> upgraded
> >> > > > > their
> >> > > > > > > rack
> >> > > > > > > > > ID
> >> > > > > > > > > > > > > would still be in old format. In this case, the
> >> broker
> >> > > then
> >> > > > > > > needs
> >> > > > > > > > > to
> >> > > > > > > > > > > > choose
> >> > > > > > > > > > > > > the old versioned assignor. I'm probably making
> >> > > something
> >> > > > > up
> >> > > > > > > here
> >> > > > > > > > > for
> >> > > > > > > > > > > > rack
> >> > > > > > > > > > > > > aware assignors, but I'm wondering if in general
> >> such
> >> > > an
> >> > > > > > > > > > "auto-downgrade"
> >> > > > > > > > > > > > > behavior would be needed still for broker-side
> >> > > assignor,
> >> > > > > and
> >> > > > > > > if yes
> >> > > > > > > > > > would
> >> > > > > > > > > > > > > "version" still be useful.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > > Yeah, that's right. Within a rebalance,
> >> > > `onAssignment` is
> >> > > > > > > called
> >> > > > > > > > > > once
> >> > > > > > > > > > > > > when the member transitions to a new epoch. This
> >> one
> >> > > > > contains
> >> > > > > > > the
> >> > > > > > > > > > full
> >> > > > > > > > > > > > > metadata provided by the client side assignor.
> >> Then,
> >> > > > > > > `onAssignment`
> >> > > > > > > > > > > > > can be called max N times where N is the number of
> >> > > > > partitions
> >> > > > > > > > > pending
> >> > > > > > > > > > > > > revocation by other members. Let me try to clarify
> >> > > this in
> >> > > > > the
> >> > > > > > > KIP.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Okay, my understanding is that the calling
> >> ordering of
> >> > > > > these
> >> > > > > > > > > > callbacks
> >> > > > > > > > > > > > > would be like the following:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > ----------------------------------------
> >> > > > > > > > > > > > > onPartitionsRevoked(); // just once, since we
> >> do not
> >> > > > > really
> >> > > > > > > need
> >> > > > > > > > > > > > > to revoke incrementally.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > onAssignment(); // the first call, with epoch
> >> > > > > incremented
> >> > > > > > > > > > > > > onPartitionsAssigned(); // paired with the
> >> > > onAssignment
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > onAssignment(); // the first
> >> onAssignment
> >> > > > > would
> >> > > > > > > bump
> >> > > > > > > > > up
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > epoch, and the metadata reflected.
> >> > > > > > > > > > > > > onPartitionsAssigned(); // each time we get an
> >> > > additional
> >> > > > > > > > > > assignment,
> >> > > > > > > > > > > > we
> >> > > > > > > > > > > > > call onAssignment and then paired with an
> >> > > > > onPartitionsAssigned
> >> > > > > > > > > > > > > ...
> >> > > > > > > > > > > > > onAssignment();
> >> > > > > > > > > > > > > onPartitionsAssigned(); // on each of the
> >> > > onAssignment
> >> > > > > > > calls, the
> >> > > > > > > > > > > > encoded
> >> > > > > > > > > > > > > metadata would not change, only the incrementally
> >> added
> >> > > > > > > partitions
> >> > > > > > > > > be
> >> > > > > > > > > > > > > reflected
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Is that the case?
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > I'm wondering if we would still call onAssignment
> >> just
> >> > > > > once,
> >> > > > > > > that
> >> > > > > > > > > > encodes
> >> > > > > > > > > > > > > all the assignment for this rebalance, including
> >> all
> >> > > the
> >> > > > > > > partitions
> >> > > > > > > > > > that
> >> > > > > > > > > > > > > should be assigned to the member but not yet
> >> assigned
> >> > > since
> >> > > > > > > they
> >> > > > > > > > > > have not
> >> > > > > > > > > > > > > been revoked by others. In that case the call
> >> ordering
> >> > > > > would
> >> > > > > > > be:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > ----------------------------------------
> >> > > > > > > > > > > > > onPartitionsRevoked(); // just once
> >> > > > > > > > > > > > > onAssignment(); // just once, with epoch
> >> > > incremented,
> >> > > > > and
> >> > > > > > > > > metadata
> >> > > > > > > > > > > > > encoded changed, the "assignment" field also
> >> reflect
> >> > > the
> >> > > > > final
> >> > > > > > > > > target
> >> > > > > > > > > > > > > assignment
> >> > > > > > > > > > > > > onPartitionsAssigned(); // multiple times, which
> >> > > > > represent
> >> > > > > > > > > > > > incrementally
> >> > > > > > > > > > > > > added partitions
> >> > > > > > > > > > > > > ...
> >> > > > > > > > > > > > > onPartitionsAssigned();
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > The motivation from this is that, most users
> >> would only
> >> > > > > > > implement
> >> > > > > > > > > the
> >> > > > > > > > > > > > > rebalance callback listeners and hence we'd
> >> definitely
> >> > > > > need to
> >> > > > > > > make
> >> > > > > > > > > > sure
> >> > > > > > > > > > > > > the semantics of that does not change much, and
> >> the
> >> > > time
> >> > > > > > > > > > > > > `onPartitionsAssigned` indicate the time when the
> >> > > > > partitions
> >> > > > > > > are
> >> > > > > > > > > > actually
> >> > > > > > > > > > > > > assigned to it; while for assignors, the
> >> > > `onAssignment` is
> >> > > > > > > used to
> >> > > > > > > > > > > > indicate
> >> > > > > > > > > > > > > what decision is made regarding for this member,
> >> i.e.
> >> > > when
> >> > > > > the
> >> > > > > > > > > > partitions
> >> > > > > > > > > > > > > are decided to be given to it, but not necessarily
> >> > > meant
> >> > > > > that
> >> > > > > > > it
> >> > > > > > > > > has
> >> > > > > > > > > > been
> >> > > > > > > > > > > > > given, since that time should be determined by the
> >> > > time of
> >> > > > > > > > > > > > > `onPartitionsAssigned`. The benefits though,
> >> would be
> >> > > that
> >> > > > > > > assignor
> >> > > > > > > > > > > > > implementers would not need to reason which
> >> > > `onAssignment`
> >> > > > > > > would be
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > last one for this epoch.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Guozhang
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Guozhang
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > On Thu, Sep 22, 2022 at 2:20 AM David Jacot
> >> > > > > > > > > > <djacot@confluent.io.invalid
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > > Hi Guozhang,
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 1) The client-side
> >> "PartitionAssignor#Assignment"
> >> > > has
> >> > > > > an
> >> > > > > > > Error
> >> > > > > > > > > > byte
> >> > > > > > > > > > > > > > field,
> >> > > > > > > > > > > > > > > while the broker-side
> >> > > "PartitionAssignor#Assignment"
> >> > > > > does
> >> > > > > > > not.
> >> > > > > > > > > > And
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > rationale seems to be assuming that we should
> >> > > always be
> >> > > > > > > able to
> >> > > > > > > > > > do
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > assignment at the broker-side assignor without
> >> > > errors.
> >> > > > > > > > > > Personally I
> >> > > > > > > > > > > > > think
> >> > > > > > > > > > > > > > > it's still potentially beneficial to add the
> >> Error
> >> > > > > field
> >> > > > > > > even
> >> > > > > > > > > for
> >> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge
> >> cases
> >> > > where
> >> > > > > some
> >> > > > > > > > > > subscribed
> >> > > > > > > > > > > > > > > topics are not recognized with the current
> >> broker's
> >> > > > > > > metadata.
> >> > > > > > > > > > What do
> >> > > > > > > > > > > > > you
> >> > > > > > > > > > > > > > > think?
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > Yeah, that seems reasonable. However, I wonder
> >> if it
> >> > > > > would be
> >> > > > > > > > > > better
> >> > > > > > > > > > > > > > to use an exception on the server side. This is
> >> what
> >> > > we
> >> > > > > > > usually
> >> > > > > > > > > do
> >> > > > > > > > > > for
> >> > > > > > > > > > > > > > server side plugins. On the client side, we use
> >> a
> >> > > field
> >> > > > > > > because
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > errors are not defined in advance.
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > Your comment also makes me think about what we
> >> > > should do
> >> > > > > > > when the
> >> > > > > > > > > > > > > > server side assignor fails. I suppose that we
> >> have to
> >> > > > > keep
> >> > > > > > > the
> >> > > > > > > > > > current
> >> > > > > > > > > > > > > > assignment until a new event occurs. For
> >> instance, in
> >> > > > > your
> >> > > > > > > > > example,
> >> > > > > > > > > > > > > > the coordinator would have to trigger a
> >> rebalance
> >> > > when
> >> > > > > > > > > unrecognized
> >> > > > > > > > > > > > > > topics are available. This would be part of the
> >> > > metadata
> >> > > > > > > > > > monitoring.
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> >> > > additional
> >> > > > > > > fields
> >> > > > > > > > > > > > > > > reason/version/metadata compared with the
> >> > > broker-side
> >> > > > > > > > > > GroupMember. I
> >> > > > > > > > > > > > > > agree
> >> > > > > > > > > > > > > > > that broker-side assignor would not need
> >> > > > > reason/metadata
> >> > > > > > > since
> >> > > > > > > > > > they
> >> > > > > > > > > > > > are
> >> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
> >> what
> >> > > about
> >> > > > > > > version?
> >> > > > > > > > > > E.g.
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > > it
> >> > > > > > > > > > > > > > > possible that we evolve our broker-side
> >> built-in
> >> > > > > assignor
> >> > > > > > > but
> >> > > > > > > > > > the old
> >> > > > > > > > > > > > > > > versioned clients would not be able to work
> >> with
> >> > > the
> >> > > > > new
> >> > > > > > > > > > version, in
> >> > > > > > > > > > > > > > which
> >> > > > > > > > > > > > > > > case we need to let the broker being aware of
> >> this
> >> > > and
> >> > > > > > > upgrade
> >> > > > > > > > > > its
> >> > > > > > > > > > > > > > behavior
> >> > > > > > > > > > > > > > > to cooperate with the clients?
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > I don't really see the benefits here because
> >> server
> >> > > side
> >> > > > > > > > > assignors
> >> > > > > > > > > > > > > > don't have metadata at all. They only assign
> >> > > > > > > topic-partitions.
> >> > > > > > > > > They
> >> > > > > > > > > > > > > > are not supposed to generate metadata nor to
> >> receive
> >> > > > > metadata
> >> > > > > > > > > from
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > members.
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
> >> client-side
> >> > > > > > > "GroupMember",
> >> > > > > > > > > > > > instead
> >> > > > > > > > > > > > > > of
> >> > > > > > > > > > > > > > > including these three fields, what about just
> >> > > adding
> >> > > > > the
> >> > > > > > > > > > "Metadata"
> >> > > > > > > > > > > > > field
> >> > > > > > > > > > > > > > > class which has these three fields? Also,
> >> there
> >> > > are two
> >> > > > > > > > > > "Metadata"
> >> > > > > > > > > > > > > > > currently in the APIs, the first is a class
> >> that
> >> > > > > encodes
> >> > > > > > > > > > > > > > > reason/version/metadata, and the second is
> >> just the
> >> > > > > encoded
> >> > > > > > > > > > metadata
> >> > > > > > > > > > > > > > bytes.
> >> > > > > > > > > > > > > > > I'm wondering what about just naming the
> >> first as
> >> > > > > > > > > memberMetadata,
> >> > > > > > > > > > > > which
> >> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
> >> instead
> >> > > > > naming the
> >> > > > > > > > > > second
> >> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > That's a good point. Let me try to rationalize
> >> this
> >> > > > > interface
> >> > > > > > > > > > based on
> >> > > > > > > > > > > > > > your suggestions.
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > Best,
> >> > > > > > > > > > > > > > David
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 9:21 PM Guozhang Wang <
> >> > > > > > > > > wangguoz@gmail.com>
> >> > > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > Hello David,
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > Just had a few more nit questions about the
> >> public
> >> > > > > APIs:
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 1) The client-side
> >> "PartitionAssignor#Assignment"
> >> > > has
> >> > > > > an
> >> > > > > > > Error
> >> > > > > > > > > > byte
> >> > > > > > > > > > > > > > field,
> >> > > > > > > > > > > > > > > while the broker-side
> >> > > "PartitionAssignor#Assignment"
> >> > > > > does
> >> > > > > > > not.
> >> > > > > > > > > > And
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > rationale seems to be assuming that we should
> >> > > always be
> >> > > > > > > able to
> >> > > > > > > > > > do
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > assignment at the broker-side assignor without
> >> > > errors.
> >> > > > > > > > > > Personally I
> >> > > > > > > > > > > > > think
> >> > > > > > > > > > > > > > > it's still potentially beneficial to add the
> >> Error
> >> > > > > field
> >> > > > > > > even
> >> > > > > > > > > for
> >> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge
> >> cases
> >> > > where
> >> > > > > some
> >> > > > > > > > > > subscribed
> >> > > > > > > > > > > > > > > topics are not recognized with the current
> >> broker's
> >> > > > > > > metadata.
> >> > > > > > > > > > What do
> >> > > > > > > > > > > > > you
> >> > > > > > > > > > > > > > > think?
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> >> > > additional
> >> > > > > > > fields
> >> > > > > > > > > > > > > > > reason/version/metadata compared with the
> >> > > broker-side
> >> > > > > > > > > > GroupMember. I
> >> > > > > > > > > > > > > > agree
> >> > > > > > > > > > > > > > > that broker-side assignor would not need
> >> > > > > reason/metadata
> >> > > > > > > since
> >> > > > > > > > > > they
> >> > > > > > > > > > > > are
> >> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
> >> what
> >> > > about
> >> > > > > > > version?
> >> > > > > > > > > > E.g.
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > > it
> >> > > > > > > > > > > > > > > possible that we evolve our broker-side
> >> built-in
> >> > > > > assignor
> >> > > > > > > but
> >> > > > > > > > > > the old
> >> > > > > > > > > > > > > > > versioned clients would not be able to work
> >> with
> >> > > the
> >> > > > > new
> >> > > > > > > > > > version, in
> >> > > > > > > > > > > > > > which
> >> > > > > > > > > > > > > > > case we need to let the broker being aware of
> >> this
> >> > > and
> >> > > > > > > upgrade
> >> > > > > > > > > > its
> >> > > > > > > > > > > > > > behavior
> >> > > > > > > > > > > > > > > to cooperate with the clients?
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
> >> client-side
> >> > > > > > > "GroupMember",
> >> > > > > > > > > > > > instead
> >> > > > > > > > > > > > > > of
> >> > > > > > > > > > > > > > > including these three fields, what about just
> >> > > adding
> >> > > > > the
> >> > > > > > > > > > "Metadata"
> >> > > > > > > > > > > > > field
> >> > > > > > > > > > > > > > > class which has these three fields? Also,
> >> there
> >> > > are two
> >> > > > > > > > > > "Metadata"
> >> > > > > > > > > > > > > > > currently in the APIs, the first is a class
> >> that
> >> > > > > encodes
> >> > > > > > > > > > > > > > > reason/version/metadata, and the second is
> >> just the
> >> > > > > encoded
> >> > > > > > > > > > metadata
> >> > > > > > > > > > > > > > bytes.
> >> > > > > > > > > > > > > > > I'm wondering what about just naming the
> >> first as
> >> > > > > > > > > memberMetadata,
> >> > > > > > > > > > > > which
> >> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
> >> instead
> >> > > > > naming the
> >> > > > > > > > > > second
> >> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > Guozhang
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 12:08 PM Guozhang
> >> Wang <
> >> > > > > > > > > > wangguoz@gmail.com>
> >> > > > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > Hello David,
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > Thanks for bringing this question up. I
> >> think the
> >> > > > > main
> >> > > > > > > > > > benefits as
> >> > > > > > > > > > > > > you
> >> > > > > > > > > > > > > > > > listed is 2) above if it stays; just to
> >> clarify,
> >> > > we
> >> > > > > would
> >> > > > > > > > > only
> >> > > > > > > > > > be
> >> > > > > > > > > > > > > able
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > > save one round trip if the rebalance is
> >> still
> >> > > > > triggered
> >> > > > > > > by
> >> > > > > > > > > the
> >> > > > > > > > > > > > > broker;
> >> > > > > > > > > > > > > > if
> >> > > > > > > > > > > > > > > > the rebalance is triggered by the client
> >> then the
> >> > > > > > > > > > num.round.trips
> >> > > > > > > > > > > > are
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > > same:
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > 1) With GroupPrepareAssignment:
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> >> > > suppose
> >> > > > > it has
> >> > > > > > > > > > already
> >> > > > > > > > > > > > > sent
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
> >> first
> >> > > since
> >> > > > > > > only
> >> > > > > > > > > one
> >> > > > > > > > > > > > > request
> >> > > > > > > > > > > > > > /
> >> > > > > > > > > > > > > > > > response can be inflight with the
> >> coordinator's
> >> > > > > socket.
> >> > > > > > > > > > > > > > > > T1: client receives the HB response, and
> >> then
> >> > > sends
> >> > > > > the
> >> > > > > > > > > > > > > > > > GroupPrepareAssignment request.
> >> > > > > > > > > > > > > > > > T2: the GroupPrepareAssignment response is
> >> > > returned.
> >> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
> >> sends a
> >> > > > > > > > > > > > > > GroupInstallAssignment
> >> > > > > > > > > > > > > > > > request.
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > In total, two round trips.
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > 2) Without GroupPrepareAssignment:
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> >> > > suppose
> >> > > > > it has
> >> > > > > > > > > > already
> >> > > > > > > > > > > > > sent
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
> >> first
> >> > > since
> >> > > > > > > only
> >> > > > > > > > > one
> >> > > > > > > > > > > > > request
> >> > > > > > > > > > > > > > /
> >> > > > > > > > > > > > > > > > response can be inflight with the
> >> coordinator's
> >> > > > > socket.
> >> > > > > > > > > > > > > > > > T1: client receives the HB response, and
> >> then
> >> > > sends
> >> > > > > the
> >> > > > > > > new
> >> > > > > > > > > HB
> >> > > > > > > > > > > > > request
> >> > > > > > > > > > > > > > > > with the flag indicating a new rebalance
> >> needed..
> >> > > > > > > > > > > > > > > > T2: the HB response with the optional member
> >> > > metadata
> >> > > > > > > map is
> >> > > > > > > > > > > > > returned.
> >> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
> >> sends a
> >> > > > > > > > > > > > > > GroupInstallAssignment
> >> > > > > > > > > > > > > > > > request.
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > In total, two round trips as well.
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > -----------------------------
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > So to complete the full picture here, we'd
> >> need
> >> > > to
> >> > > > > modify
> >> > > > > > > > > both
> >> > > > > > > > > > HB
> >> > > > > > > > > > > > > > request
> >> > > > > > > > > > > > > > > > and response so that the client can also
> >> > > indicate a
> >> > > > > new
> >> > > > > > > > > > rebalance
> >> > > > > > > > > > > > via
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > > HB request as well, right?
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > Assuming all above is true, I think it's
> >> okay to
> >> > > > > merge
> >> > > > > > > the
> >> > > > > > > > > > > > > > > > GroupPrepareAssignment into HB given that
> >> we can
> >> > > > > make the
> >> > > > > > > > > > > > additional
> >> > > > > > > > > > > > > > fields
> >> > > > > > > > > > > > > > > > encoding the full member (subscription)
> >> metadata
> >> > > and
> >> > > > > > > topic
> >> > > > > > > > > > metadata
> >> > > > > > > > > > > > > as
> >> > > > > > > > > > > > > > > > optional fields.
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > Guozhang
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > > On Mon, Sep 12, 2022 at 5:22 AM David Jacot
> >> > > > > > > > > > > > > > <dj...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> Hi all,
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> During an offline conversation, someone
> >> asked
> >> > > why we
> >> > > > > > > need
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> ConsumerGroupPrepareAssignment API and
> >> suggested
> >> > > > > that we
> >> > > > > > > > > could
> >> > > > > > > > > > > > > instead
> >> > > > > > > > > > > > > > > >> provide the group state in the heartbeat
> >> > > response.
> >> > > > > This
> >> > > > > > > has
> >> > > > > > > > > a
> >> > > > > > > > > > few
> >> > > > > > > > > > > > > > > >> advantages: 1) it does not require using a
> >> > > special
> >> > > > > error
> >> > > > > > > > > code
> >> > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> signal that a new assignment is required
> >> as the
> >> > > > > signal
> >> > > > > > > would
> >> > > > > > > > > > be
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> provided group state; 2) it removes one
> >> round
> >> > > trip
> >> > > > > when
> >> > > > > > > a
> >> > > > > > > > > > client
> >> > > > > > > > > > > > > side
> >> > > > > > > > > > > > > > > >> assignor is used. The downside is that it
> >> makes
> >> > > the
> >> > > > > > > > > heartbeat
> >> > > > > > > > > > > > > > > >> response's definition quite large. I recall
> >> > > that I
> >> > > > > went
> >> > > > > > > with
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> current approach due to this.
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> Providing the group state in the heartbeat
> >> > > response
> >> > > > > is
> >> > > > > > > > > > appealing.
> >> > > > > > > > > > > > > What
> >> > > > > > > > > > > > > > > >> do you guys think?
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> Best,
> >> > > > > > > > > > > > > > > >> David
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> On Mon, Sep 12, 2022 at 2:17 PM David
> >> Jacot <
> >> > > > > > > > > > djacot@confluent.io>
> >> > > > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > Hi Guozhang,
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > 1. I have added a reference to the
> >> relevant
> >> > > > > chapter
> >> > > > > > > > > instead
> >> > > > > > > > > > of
> >> > > > > > > > > > > > > > > >> > repeating the whole thing. Does that
> >> work for
> >> > > you?
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > 2. The "Rebalance Triggers" section you
> >> are
> >> > > > > referring
> >> > > > > > > to
> >> > > > > > > > > is
> >> > > > > > > > > > > > about
> >> > > > > > > > > > > > > > when
> >> > > > > > > > > > > > > > > >> > a rebalance should be triggered for the
> >> > > > > non-upgraded
> >> > > > > > > > > members
> >> > > > > > > > > > > > using
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > old protocol. The section mentions that a
> >> > > > > rebalance
> >> > > > > > > must
> >> > > > > > > > > be
> >> > > > > > > > > > > > > > triggered
> >> > > > > > > > > > > > > > > >> > when a new assignment is installed. This
> >> > > implies
> >> > > > > that
> >> > > > > > > the
> >> > > > > > > > > > group
> >> > > > > > > > > > > > > > epoch
> >> > > > > > > > > > > > > > > >> > was updated either by a native member or
> >> a
> >> > > > > > > non-upgraded
> >> > > > > > > > > > member.
> >> > > > > > > > > > > > > For
> >> > > > > > > > > > > > > > > >> > the latter, the JoinGroup request would
> >> be the
> >> > > > > > > trigger. I
> >> > > > > > > > > > have
> >> > > > > > > > > > > > > > added a
> >> > > > > > > > > > > > > > > >> > reference to the relevant chapter in the
> >> > > > > "JoinGroup
> >> > > > > > > > > > Handling"
> >> > > > > > > > > > > > > > section
> >> > > > > > > > > > > > > > > >> > as well. Does that make sense?
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > Thanks,
> >> > > > > > > > > > > > > > > >> > David
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > On Fri, Sep 9, 2022 at 10:35 PM Guozhang
> >> Wang
> >> > > <
> >> > > > > > > > > > > > wangguoz@gmail.com
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> wrote:
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > Hello David,
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > Alright I think that's sufficient.
> >> Just to
> >> > > make
> >> > > > > that
> >> > > > > > > > > > clear in
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > doc,
> >> > > > > > > > > > > > > > > >> > > could we update:
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > 1) the heartbeat request handling
> >> section,
> >> > > > > stating
> >> > > > > > > when
> >> > > > > > > > > > > > > > coordinator
> >> > > > > > > > > > > > > > > >> will
> >> > > > > > > > > > > > > > > >> > > trigger rebalance based on the HB's
> >> member
> >> > > > > metadata
> >> > > > > > > /
> >> > > > > > > > > > reason?
> >> > > > > > > > > > > > > > > >> > > 2) the "Rebalance Triggers" section to
> >> > > include
> >> > > > > what
> >> > > > > > > we
> >> > > > > > > > > > > > described
> >> > > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> "Group
> >> > > > > > > > > > > > > > > >> > > Epoch - Trigger a rebalance" section as
> >> > > well?
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > Guozhang
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > On Fri, Sep 9, 2022 at 1:28 AM David
> >> Jacot
> >> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > >> > > wrote:
> >> > > > > > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > Hi Guozhang,
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > I thought that the assignor will
> >> always be
> >> > > > > > > consulted
> >> > > > > > > > > > when
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > next
> >> > > > > > > > > > > > > > > >> > > > heartbeat request is constructed. In
> >> other
> >> > > > > words,
> >> > > > > > > > > > > > > > > >> > > > `PartitionAssignor#metadata` will be
> >> > > called
> >> > > > > for
> >> > > > > > > every
> >> > > > > > > > > > > > > heartbeat.
> >> > > > > > > > > > > > > > > >> This
> >> > > > > > > > > > > > > > > >> > > > gives the opportunity for the
> >> assignor to
> >> > > > > enforce
> >> > > > > > > a
> >> > > > > > > > > > > > rebalance
> >> > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > > >> > > > setting the reason to a non-zero
> >> value or
> >> > > by
> >> > > > > > > changing
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > bytes. Do
> >> > > > > > > > > > > > > > > >> > > > you think that this is not
> >> sufficient?
> >> > > Are you
> >> > > > > > > > > > concerned by
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> delay?
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > Best,
> >> > > > > > > > > > > > > > > >> > > > David
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > On Fri, Sep 9, 2022 at 7:10 AM
> >> Guozhang
> >> > > Wang <
> >> > > > > > > > > > > > > > wangguoz@gmail.com>
> >> > > > > > > > > > > > > > > >> wrote:
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > Hello David,
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > One of Jun's comments make me
> >> thinking:
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > ```
> >> > > > > > > > > > > > > > > >> > > > > In this case, a new assignment is
> >> > > triggered
> >> > > > > by
> >> > > > > > > the
> >> > > > > > > > > > client
> >> > > > > > > > > > > > > side
> >> > > > > > > > > > > > > > > >> > > > > assignor. When constructing the
> >> HB, the
> >> > > > > consumer
> >> > > > > > > > > will
> >> > > > > > > > > > > > always
> >> > > > > > > > > > > > > > > >> consult
> >> > > > > > > > > > > > > > > >> > > > > the client side assignor and
> >> propagate
> >> > > the
> >> > > > > > > > > > information to
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > coordinator. In other words, we
> >> don't
> >> > > expect
> >> > > > > > > users
> >> > > > > > > > > to
> >> > > > > > > > > > call
> >> > > > > > > > > > > > > > > >> > > > > Consumer#enforceRebalance anymore.
> >> > > > > > > > > > > > > > > >> > > > > ```
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > As I looked at the current
> >> > > > > PartitionAssignor's
> >> > > > > > > > > > interface,
> >> > > > > > > > > > > > we
> >> > > > > > > > > > > > > > > >> actually do
> >> > > > > > > > > > > > > > > >> > > > > not have a way yet to instruct how
> >> to
> >> > > > > construct
> >> > > > > > > the
> >> > > > > > > > > > next
> >> > > > > > > > > > > > HB
> >> > > > > > > > > > > > > > > >> request, e.g.
> >> > > > > > > > > > > > > > > >> > > > > when the assignor wants to enforce
> >> a new
> >> > > > > > > rebalance
> >> > > > > > > > > > with a
> >> > > > > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> assignment,
> >> > > > > > > > > > > > > > > >> > > > > we'd need some customizable APIs
> >> inside
> >> > > the
> >> > > > > > > > > > > > > PartitionAssignor
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> indicate
> >> > > > > > > > > > > > > > > >> > > > > the next HB telling broker about
> >> so.
> >> > > WDYT
> >> > > > > about
> >> > > > > > > > > adding
> >> > > > > > > > > > > > such
> >> > > > > > > > > > > > > an
> >> > > > > > > > > > > > > > > >> API on the
> >> > > > > > > > > > > > > > > >> > > > > PartitionAssignor?
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > Guozhang
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > On Tue, Sep 6, 2022 at 6:09 AM
> >> David
> >> > > Jacot
> >> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > >> > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > Hi Jun,
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > I have updated the KIP to
> >> include your
> >> > > > > > > feedback. I
> >> > > > > > > > > > have
> >> > > > > > > > > > > > > also
> >> > > > > > > > > > > > > > > >> tried to
> >> > > > > > > > > > > > > > > >> > > > > > clarify the parts which were not
> >> > > cleared.
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > Best,
> >> > > > > > > > > > > > > > > >> > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > On Fri, Sep 2, 2022 at 4:18 PM
> >> David
> >> > > > > Jacot <
> >> > > > > > > > > > > > > > djacot@confluent.io
> >> > > > > > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Hi Jun,
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Thanks for your feedback. Let
> >> me
> >> > > start
> >> > > > > by
> >> > > > > > > > > > answering
> >> > > > > > > > > > > > your
> >> > > > > > > > > > > > > > > >> questions
> >> > > > > > > > > > > > > > > >> > > > > > > inline and I will update the
> >> KIP
> >> > > next
> >> > > > > week.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
> >> the
> >> > > main
> >> > > > > > > benefits
> >> > > > > > > > > > of
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > KIP
> >> > > > > > > > > > > > > > > >> seem to
> >> > > > > > > > > > > > > > > >> > > > be
> >> > > > > > > > > > > > > > > >> > > > > > fewer
> >> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and
> >> more
> >> > > > > efficient
> >> > > > > > > > > > support of
> >> > > > > > > > > > > > > > > >> wildcard. A few
> >> > > > > > > > > > > > > > > >> > > > > > > > comments below.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > I would also add that the KIP
> >> > > removes
> >> > > > > the
> >> > > > > > > global
> >> > > > > > > > > > sync
> >> > > > > > > > > > > > > > barrier
> >> > > > > > > > > > > > > > > >> in the
> >> > > > > > > > > > > > > > > >> > > > > > > protocol which is essential to
> >> > > improve
> >> > > > > group
> >> > > > > > > > > > stability
> >> > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > > scalability, and the KIP also
> >> > > > > simplifies the
> >> > > > > > > > > > client by
> >> > > > > > > > > > > > > > moving
> >> > > > > > > > > > > > > > > >> most of
> >> > > > > > > > > > > > > > > >> > > > > > > the logic to the server side.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.
> >> ConsumerGroupHeartbeatRequest
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> >> > > singleton.
> >> > > > > Do we
> >> > > > > > > plan
> >> > > > > > > > > > to
> >> > > > > > > > > > > > > > support
> >> > > > > > > > > > > > > > > >> rolling
> >> > > > > > > > > > > > > > > >> > > > > > changing
> >> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
> >> the
> >> > > > > > > consumers?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Definitely. The group
> >> coordinator
> >> > > will
> >> > > > > use
> >> > > > > > > the
> >> > > > > > > > > > > > assignor
> >> > > > > > > > > > > > > > used
> >> > > > > > > > > > > > > > > >> by a
> >> > > > > > > > > > > > > > > >> > > > > > > majority of the members. This
> >> > > allows the
> >> > > > > > > group
> >> > > > > > > > > to
> >> > > > > > > > > > move
> >> > > > > > > > > > > > > > from
> >> > > > > > > > > > > > > > > >> one
> >> > > > > > > > > > > > > > > >> > > > > > > assignor to another by a roll.
> >> This
> >> > > is
> >> > > > > > > explained
> >> > > > > > > > > > in
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> Assignor
> >> > > > > > > > > > > > > > > >> > > > > > > Selection chapter.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could
> >> you
> >> > > explain
> >> > > > > > > whether
> >> > > > > > > > > > it's
> >> > > > > > > > > > > > > > > >> required in
> >> > > > > > > > > > > > > > > >> > > > every
> >> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios
> >> when it
> >> > > > > needs to
> >> > > > > > > be
> >> > > > > > > > > > filled?
> >> > > > > > > > > > > > > For
> >> > > > > > > > > > > > > > > >> example,
> >> > > > > > > > > > > > > > > >> > > > it's
> >> > > > > > > > > > > > > > > >> > > > > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
> >> TopicPartitions
> >> > > > > needs to
> >> > > > > > > be
> >> > > > > > > > > > filled.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > The client is expected to set
> >> those
> >> > > > > fields
> >> > > > > > > in
> >> > > > > > > > > > case of
> >> > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > >> connection
> >> > > > > > > > > > > > > > > >> > > > > > > issue (e.g. timeout) or when
> >> the
> >> > > fields
> >> > > > > have
> >> > > > > > > > > > changed
> >> > > > > > > > > > > > > since
> >> > > > > > > > > > > > > > > >> the last
> >> > > > > > > > > > > > > > > >> > > > > > > HB. The server populates those
> >> > > fields as
> >> > > > > > > long as
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > member
> >> > > > > > > > > > > > > > > >> is not
> >> > > > > > > > > > > > > > > >> > > > > > > fully reconciled - the member
> >> should
> >> > > > > > > acknowledge
> >> > > > > > > > > > that
> >> > > > > > > > > > > > it
> >> > > > > > > > > > > > > > has
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > expected epoch and assignment.
> >> I
> >> > > will
> >> > > > > > > clarify
> >> > > > > > > > > > this in
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > KIP.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> >> > > protocol,
> >> > > > > the
> >> > > > > > > rack
> >> > > > > > > > > > > > > affinity
> >> > > > > > > > > > > > > > > >> between the
> >> > > > > > > > > > > > > > > >> > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
> >> considered
> >> > > > > during
> >> > > > > > > > > > fetching,
> >> > > > > > > > > > > > but
> >> > > > > > > > > > > > > > not
> >> > > > > > > > > > > > > > > >> during
> >> > > > > > > > > > > > > > > >> > > > > > assigning
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> >> > > Sometimes,
> >> > > > > once
> >> > > > > > > the
> >> > > > > > > > > > > > > assignment
> >> > > > > > > > > > > > > > is
> >> > > > > > > > > > > > > > > >> made,
> >> > > > > > > > > > > > > > > >> > > > there
> >> > > > > > > > > > > > > > > >> > > > > > is
> >> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
> >> affinity
> >> > > > > because
> >> > > > > > > no
> >> > > > > > > > > > replicas
> >> > > > > > > > > > > > > of
> >> > > > > > > > > > > > > > > >> assigned
> >> > > > > > > > > > > > > > > >> > > > > > partitions
> >> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> >> > > > > wondering
> >> > > > > > > if we
> >> > > > > > > > > > should
> >> > > > > > > > > > > > > use
> >> > > > > > > > > > > > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > > > opportunity
> >> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
> >> rack
> >> > > in
> >> > > > > > > > > > GroupMember.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > That's an interesting idea. I
> >> don't
> >> > > see
> >> > > > > any
> >> > > > > > > > > issue
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > > adding
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > rack
> >> > > > > > > > > > > > > > > >> > > > > > > to the members. I will do so.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side,
> >> often,
> >> > > it's
> >> > > > > > > useful to
> >> > > > > > > > > > know
> >> > > > > > > > > > > > how
> >> > > > > > > > > > > > > > busy
> >> > > > > > > > > > > > > > > >> a group
> >> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> >> > > event
> >> > > > > loop
> >> > > > > > > > > model,
> >> > > > > > > > > > it
> >> > > > > > > > > > > > > seems
> >> > > > > > > > > > > > > > > >> that we
> >> > > > > > > > > > > > > > > >> > > > could
> >> > > > > > > > > > > > > > > >> > > > > > add
> >> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
> >> fraction
> >> > > of
> >> > > > > the
> >> > > > > > > time
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > event
> >> > > > > > > > > > > > > > > >> loop is
> >> > > > > > > > > > > > > > > >> > > > doing
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > actual work.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > That's a great idea. I will
> >> add it.
> >> > > > > Thanks.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> >> > > > > coordinator
> >> > > > > > > > > > failover
> >> > > > > > > > > > > > > > > >> handling? For
> >> > > > > > > > > > > > > > > >> > > > > > example,
> >> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
> >> check
> >> > > if
> >> > > > > any
> >> > > > > > > group
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> wildcard
> >> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> >> > > matching
> >> > > > > topic?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Sure. When the new group
> >> coordinator
> >> > > > > takes
> >> > > > > > > over,
> >> > > > > > > > > > it
> >> > > > > > > > > > > > has
> >> > > > > > > > > > > > > > to:
> >> > > > > > > > > > > > > > > >> > > > > > > * Setup the session timeouts.
> >> > > > > > > > > > > > > > > >> > > > > > > * Trigger a new assignment if a
> >> > > client
> >> > > > > side
> >> > > > > > > > > > assignor
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > > used.
> >> > > > > > > > > > > > > > > >> We
> >> > > > > > > > > > > > > > > >> > > > don't
> >> > > > > > > > > > > > > > > >> > > > > > > store the information about the
> >> > > member
> >> > > > > > > selected
> >> > > > > > > > > > to run
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> assignment
> >> > > > > > > > > > > > > > > >> > > > > > > so we have to start a new one.
> >> > > > > > > > > > > > > > > >> > > > > > > * Update the topics metadata,
> >> > > verify the
> >> > > > > > > > > wildcard
> >> > > > > > > > > > > > > > > >> subscriptions, and
> >> > > > > > > > > > > > > > > >> > > > > > > trigger a rebalance if needed.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 34.
> >> ConsumerGroupMetadataValue,
> >> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> ConsumerGroupMemberMetadataValue:
> >> > > > > Could we
> >> > > > > > > > > > document
> >> > > > > > > > > > > > > what
> >> > > > > > > > > > > > > > > >> the epoch
> >> > > > > > > > > > > > > > > >> > > > > > field
> >> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
> >> the
> >> > > epoch
> >> > > > > in
> >> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> >> > > > > > > > > > > > > > > >> > > > > > reflect
> >> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> >> > > about the
> >> > > > > > > one in
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > ConsumerGroupPartitionMetadataValue
> >> > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Sure. I will clarify that but
> >> it is
> >> > > > > always
> >> > > > > > > the
> >> > > > > > > > > > latest
> >> > > > > > > > > > > > > > group
> >> > > > > > > > > > > > > > > >> epoch.
> >> > > > > > > > > > > > > > > >> > > > > > > When the group state is
> >> updated, the
> >> > > > > group
> >> > > > > > > epoch
> >> > > > > > > > > > is
> >> > > > > > > > > > > > > > bumped so
> >> > > > > > > > > > > > > > > >> we use
> >> > > > > > > > > > > > > > > >> > > > > > > that one for all the change
> >> records
> >> > > > > related
> >> > > > > > > to
> >> > > > > > > > > the
> >> > > > > > > > > > > > > update.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
> >> will
> >> > > ensure
> >> > > > > > > that
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > following
> >> > > > > > > > > > > > > > > >> > > > invariants
> >> > > > > > > > > > > > > > > >> > > > > > are
> >> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members
> >> exists." It's
> >> > > > > > > possible
> >> > > > > > > > > for
> >> > > > > > > > > > a
> >> > > > > > > > > > > > > member
> >> > > > > > > > > > > > > > > >> not to
> >> > > > > > > > > > > > > > > >> > > > get any
> >> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > That's right. Here I meant
> >> that the
> >> > > > > members
> >> > > > > > > > > > provided
> >> > > > > > > > > > > > by
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> assignor
> >> > > > > > > > > > > > > > > >> > > > > > > in the assignment must exist
> >> in the
> >> > > > > group.
> >> > > > > > > The
> >> > > > > > > > > > > > assignor
> >> > > > > > > > > > > > > > can
> >> > > > > > > > > > > > > > > >> not make
> >> > > > > > > > > > > > > > > >> > > > > > > up new member ids.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> >> > > with a
> >> > > > > > > member
> >> > > > > > > > > > epoch
> >> > > > > > > > > > > > > > equals to
> >> > > > > > > > > > > > > > > >> 0":
> >> > > > > > > > > > > > > > > >> > > > When
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
> >> member
> >> > > id
> >> > > > > > > would be
> >> > > > > > > > > > used?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > A member is expected to
> >> abandon all
> >> > > its
> >> > > > > > > > > > partitions and
> >> > > > > > > > > > > > > > > >> rejoins when
> >> > > > > > > > > > > > > > > >> > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > receives the
> >> FENCED_MEMBER_EPOCH
> >> > > error.
> >> > > > > In
> >> > > > > > > this
> >> > > > > > > > > > case,
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > > coordinator will have removed
> >> the
> >> > > member
> >> > > > > > > from
> >> > > > > > > > > the
> >> > > > > > > > > > > > group.
> >> > > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> member
> >> > > > > > > > > > > > > > > >> > > > > > > can rejoin the group with the
> >> same
> >> > > > > member
> >> > > > > > > id but
> >> > > > > > > > > > with
> >> > > > > > > > > > > > 0
> >> > > > > > > > > > > > > as
> >> > > > > > > > > > > > > > > >> epoch. Let
> >> > > > > > > > > > > > > > > >> > > > > > > me see if I can clarify this
> >> in the
> >> > > KIP.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users
> >> will
> >> > > have
> >> > > > > the
> >> > > > > > > > > ability
> >> > > > > > > > > > to
> >> > > > > > > > > > > > > > trigger a
> >> > > > > > > > > > > > > > > >> > > > > > reassignment
> >> > > > > > > > > > > > > > > >> > > > > > > > by either providing a
> >> non-zero
> >> > > reason
> >> > > > > or
> >> > > > > > > by
> >> > > > > > > > > > updating
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems
> >> to be
> >> > > > > > > conflicting
> >> > > > > > > > > > with
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> deprecation
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > In this case, a new assignment
> >> is
> >> > > > > triggered
> >> > > > > > > by
> >> > > > > > > > > the
> >> > > > > > > > > > > > > client
> >> > > > > > > > > > > > > > side
> >> > > > > > > > > > > > > > > >> > > > > > > assignor. When constructing
> >> the HB,
> >> > > the
> >> > > > > > > consumer
> >> > > > > > > > > > will
> >> > > > > > > > > > > > > > always
> >> > > > > > > > > > > > > > > >> consult
> >> > > > > > > > > > > > > > > >> > > > > > > the client side assignor and
> >> > > propagate
> >> > > > > the
> >> > > > > > > > > > information
> >> > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > > coordinator. In other words, we
> >> > > don't
> >> > > > > expect
> >> > > > > > > > > > users to
> >> > > > > > > > > > > > > call
> >> > > > > > > > > > > > > > > >> > > > > > > Consumer#enforceRebalance
> >> anymore.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment
> >> examples are
> >> > > > > nice.
> >> > > > > > > But
> >> > > > > > > > > the
> >> > > > > > > > > > > > > section
> >> > > > > > > > > > > > > > > >> seems to
> >> > > > > > > > > > > > > > > >> > > > have
> >> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
> >> transitions to
> >> > > > > epoch
> >> > > > > > > 2, B
> >> > > > > > > > > > > > > > immediately
> >> > > > > > > > > > > > > > > >> gets into
> >> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1,
> >> partitions=[foo-2]",
> >> > > which
> >> > > > > seems
> >> > > > > > > > > > > > incorrect.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
> >> transitions to
> >> > > > > epoch
> >> > > > > > > 3, C
> >> > > > > > > > > > seems
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > > get
> >> > > > > > > > > > > > > > > >> into
> >> > > > > > > > > > > > > > > >> > > > > > epoch=3,
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
> >> epoch
> >> > > 3, C
> >> > > > > > > still
> >> > > > > > > > > > has A -
> >> > > > > > > > > > > > > > > >> epoch=2,
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Sorry for that! I will revise
> >> them.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
> >> consumers:
> >> > > Do
> >> > > > > we
> >> > > > > > > > > support
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > upgrade
> >> > > > > > > > > > > > > > > >> from
> >> > > > > > > > > > > > > > > >> > > > any
> >> > > > > > > > > > > > > > > >> > > > > > old
> >> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > We will support upgrading from
> >> the
> >> > > > > consumer
> >> > > > > > > > > > protocol
> >> > > > > > > > > > > > > > version
> >> > > > > > > > > > > > > > > >> 3,
> >> > > > > > > > > > > > > > > >> > > > > > > introduced in KIP-792. KIP-792
> >> is
> >> > > not
> >> > > > > > > > > implemented
> >> > > > > > > > > > yet
> >> > > > > > > > > > > > so
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> earliest
> >> > > > > > > > > > > > > > > >> > > > > > > version is unknown at the
> >> moment.
> >> > > This
> >> > > > > is
> >> > > > > > > > > > explained in
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> migration
> >> > > > > > > > > > > > > > > >> > > > > > > plan chapter.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Thanks again for your feedback,
> >> > > Jun. I
> >> > > > > will
> >> > > > > > > > > > update the
> >> > > > > > > > > > > > > KIP
> >> > > > > > > > > > > > > > > >> based on
> >> > > > > > > > > > > > > > > >> > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > next week.
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > Best,
> >> > > > > > > > > > > > > > > >> > > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > On Thu, Sep 1, 2022 at 9:07 PM
> >> Jun
> >> > > Rao
> >> > > > > > > > > > > > > > > >> <ju...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > >> > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > Hi, David,
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
> >> the
> >> > > main
> >> > > > > > > benefits
> >> > > > > > > > > > of
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > KIP
> >> > > > > > > > > > > > > > > >> seem to
> >> > > > > > > > > > > > > > > >> > > > be
> >> > > > > > > > > > > > > > > >> > > > > > fewer
> >> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and
> >> more
> >> > > > > efficient
> >> > > > > > > > > > support of
> >> > > > > > > > > > > > > > > >> wildcard. A few
> >> > > > > > > > > > > > > > > >> > > > > > > > comments below.
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.
> >> ConsumerGroupHeartbeatRequest
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> >> > > singleton.
> >> > > > > Do we
> >> > > > > > > plan
> >> > > > > > > > > > to
> >> > > > > > > > > > > > > > support
> >> > > > > > > > > > > > > > > >> rolling
> >> > > > > > > > > > > > > > > >> > > > > > changing
> >> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
> >> the
> >> > > > > > > consumers?
> >> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could
> >> you
> >> > > explain
> >> > > > > > > whether
> >> > > > > > > > > > it's
> >> > > > > > > > > > > > > > > >> required in
> >> > > > > > > > > > > > > > > >> > > > every
> >> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios
> >> when it
> >> > > > > needs to
> >> > > > > > > be
> >> > > > > > > > > > filled?
> >> > > > > > > > > > > > > For
> >> > > > > > > > > > > > > > > >> example,
> >> > > > > > > > > > > > > > > >> > > > it's
> >> > > > > > > > > > > > > > > >> > > > > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
> >> TopicPartitions
> >> > > > > needs to
> >> > > > > > > be
> >> > > > > > > > > > filled.
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> >> > > protocol,
> >> > > > > the
> >> > > > > > > rack
> >> > > > > > > > > > > > > affinity
> >> > > > > > > > > > > > > > > >> between the
> >> > > > > > > > > > > > > > > >> > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
> >> considered
> >> > > > > during
> >> > > > > > > > > > fetching,
> >> > > > > > > > > > > > but
> >> > > > > > > > > > > > > > not
> >> > > > > > > > > > > > > > > >> during
> >> > > > > > > > > > > > > > > >> > > > > > assigning
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> >> > > Sometimes,
> >> > > > > once
> >> > > > > > > the
> >> > > > > > > > > > > > > assignment
> >> > > > > > > > > > > > > > is
> >> > > > > > > > > > > > > > > >> made,
> >> > > > > > > > > > > > > > > >> > > > there
> >> > > > > > > > > > > > > > > >> > > > > > is
> >> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
> >> affinity
> >> > > > > because
> >> > > > > > > no
> >> > > > > > > > > > replicas
> >> > > > > > > > > > > > > of
> >> > > > > > > > > > > > > > > >> assigned
> >> > > > > > > > > > > > > > > >> > > > > > partitions
> >> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> >> > > > > wondering
> >> > > > > > > if we
> >> > > > > > > > > > should
> >> > > > > > > > > > > > > use
> >> > > > > > > > > > > > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > > > opportunity
> >> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
> >> rack
> >> > > in
> >> > > > > > > > > > GroupMember.
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side,
> >> often,
> >> > > it's
> >> > > > > > > useful to
> >> > > > > > > > > > know
> >> > > > > > > > > > > > how
> >> > > > > > > > > > > > > > busy
> >> > > > > > > > > > > > > > > >> a group
> >> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> >> > > event
> >> > > > > loop
> >> > > > > > > > > model,
> >> > > > > > > > > > it
> >> > > > > > > > > > > > > seems
> >> > > > > > > > > > > > > > > >> that we
> >> > > > > > > > > > > > > > > >> > > > could
> >> > > > > > > > > > > > > > > >> > > > > > add
> >> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
> >> fraction
> >> > > of
> >> > > > > the
> >> > > > > > > time
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > event
> >> > > > > > > > > > > > > > > >> loop is
> >> > > > > > > > > > > > > > > >> > > > doing
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > actual work.
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> >> > > > > coordinator
> >> > > > > > > > > > failover
> >> > > > > > > > > > > > > > > >> handling? For
> >> > > > > > > > > > > > > > > >> > > > > > example,
> >> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
> >> check
> >> > > if
> >> > > > > any
> >> > > > > > > group
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> wildcard
> >> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> >> > > matching
> >> > > > > topic?
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 34.
> >> ConsumerGroupMetadataValue,
> >> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> ConsumerGroupMemberMetadataValue:
> >> > > > > Could we
> >> > > > > > > > > > document
> >> > > > > > > > > > > > > what
> >> > > > > > > > > > > > > > > >> the epoch
> >> > > > > > > > > > > > > > > >> > > > > > field
> >> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
> >> the
> >> > > epoch
> >> > > > > in
> >> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> >> > > > > > > > > > > > > > > >> > > > > > reflect
> >> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> >> > > about the
> >> > > > > > > one in
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > ConsumerGroupPartitionMetadataValue
> >> > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
> >> will
> >> > > ensure
> >> > > > > > > that
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > following
> >> > > > > > > > > > > > > > > >> > > > invariants
> >> > > > > > > > > > > > > > > >> > > > > > are
> >> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members
> >> exists." It's
> >> > > > > > > possible
> >> > > > > > > > > for
> >> > > > > > > > > > a
> >> > > > > > > > > > > > > member
> >> > > > > > > > > > > > > > > >> not to
> >> > > > > > > > > > > > > > > >> > > > get any
> >> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> >> > > with a
> >> > > > > > > member
> >> > > > > > > > > > epoch
> >> > > > > > > > > > > > > > equals to
> >> > > > > > > > > > > > > > > >> 0":
> >> > > > > > > > > > > > > > > >> > > > When
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
> >> member
> >> > > id
> >> > > > > > > would be
> >> > > > > > > > > > used?
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users
> >> will
> >> > > have
> >> > > > > the
> >> > > > > > > > > ability
> >> > > > > > > > > > to
> >> > > > > > > > > > > > > > trigger a
> >> > > > > > > > > > > > > > > >> > > > > > reassignment
> >> > > > > > > > > > > > > > > >> > > > > > > > by either providing a
> >> non-zero
> >> > > reason
> >> > > > > or
> >> > > > > > > by
> >> > > > > > > > > > updating
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems
> >> to be
> >> > > > > > > conflicting
> >> > > > > > > > > > with
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> deprecation
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment
> >> examples are
> >> > > > > nice.
> >> > > > > > > But
> >> > > > > > > > > the
> >> > > > > > > > > > > > > section
> >> > > > > > > > > > > > > > > >> seems to
> >> > > > > > > > > > > > > > > >> > > > have
> >> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
> >> transitions to
> >> > > > > epoch
> >> > > > > > > 2, B
> >> > > > > > > > > > > > > > immediately
> >> > > > > > > > > > > > > > > >> gets into
> >> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1,
> >> partitions=[foo-2]",
> >> > > which
> >> > > > > seems
> >> > > > > > > > > > > > incorrect.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
> >> transitions to
> >> > > > > epoch
> >> > > > > > > 3, C
> >> > > > > > > > > > seems
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > > get
> >> > > > > > > > > > > > > > > >> into
> >> > > > > > > > > > > > > > > >> > > > > > epoch=3,
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> >> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
> >> epoch
> >> > > 3, C
> >> > > > > > > still
> >> > > > > > > > > > has A -
> >> > > > > > > > > > > > > > > >> epoch=2,
> >> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
> >> consumers:
> >> > > Do
> >> > > > > we
> >> > > > > > > > > support
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > upgrade
> >> > > > > > > > > > > > > > > >> from
> >> > > > > > > > > > > > > > > >> > > > any
> >> > > > > > > > > > > > > > > >> > > > > > old
> >> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > Thanks,
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > Jun
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > On Mon, Aug 29, 2022 at 9:20
> >> AM
> >> > > David
> >> > > > > > > Jacot
> >> > > > > > > > > > > > > > > >> > > > > > <dj...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > >> > > > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > Hi all,
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > The KIP states that we will
> >> > > > > > > re-implement the
> >> > > > > > > > > > > > > > coordinator
> >> > > > > > > > > > > > > > > >> in
> >> > > > > > > > > > > > > > > >> > > > Java. I
> >> > > > > > > > > > > > > > > >> > > > > > > > > discussed this offline
> >> with a
> >> > > few
> >> > > > > folks
> >> > > > > > > and
> >> > > > > > > > > > folks
> >> > > > > > > > > > > > > are
> >> > > > > > > > > > > > > > > >> concerned
> >> > > > > > > > > > > > > > > >> > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > we could introduce many
> >> > > regressions
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > > old
> >> > > > > > > > > > > > > > protocol
> >> > > > > > > > > > > > > > > >> if we do
> >> > > > > > > > > > > > > > > >> > > > so.
> >> > > > > > > > > > > > > > > >> > > > > > > > > Therefore, I am going to
> >> remove
> >> > > this
> >> > > > > > > > > statement
> >> > > > > > > > > > > > from
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> KIP. It
> >> > > > > > > > > > > > > > > >> > > > is an
> >> > > > > > > > > > > > > > > >> > > > > > > > > implementation detail
> >> after all
> >> > > so
> >> > > > > it
> >> > > > > > > does
> >> > > > > > > > > not
> >> > > > > > > > > > > > have
> >> > > > > > > > > > > > > > to be
> >> > > > > > > > > > > > > > > >> > > > decided at
> >> > > > > > > > > > > > > > > >> > > > > > > > > this stage. We will likely
> >> > > start by
> >> > > > > > > trying
> >> > > > > > > > > to
> >> > > > > > > > > > > > > > refactor the
> >> > > > > > > > > > > > > > > >> > > > current
> >> > > > > > > > > > > > > > > >> > > > > > > > > implementation as a first
> >> step.
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > Cheers,
> >> > > > > > > > > > > > > > > >> > > > > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > On Mon, Aug 29, 2022 at
> >> 3:52 PM
> >> > > > > David
> >> > > > > > > Jacot
> >> > > > > > > > > <
> >> > > > > > > > > > > > > > > >> djacot@confluent.io
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > Hi Luke,
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> >> > > machine
> >> > > > > are:
> >> > > > > > > > > > "Empty,
> >> > > > > > > > > > > > > > assigning,
> >> > > > > > > > > > > > > > > >> > > > > > reconciling,
> >> > > > > > > > > > > > > > > >> > > > > > > > > stable,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
> >> Consumer
> >> > > > > Group
> >> > > > > > > States
> >> > > > > > > > > > > > > section,
> >> > > > > > > > > > > > > > > >> right?
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > This sentence does not
> >> refer
> >> > > to
> >> > > > > those
> >> > > > > > > > > group
> >> > > > > > > > > > > > states
> >> > > > > > > > > > > > > > but
> >> > > > > > > > > > > > > > > >> rather
> >> > > > > > > > > > > > > > > >> > > > to a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > state machine replication
> >> > > (SMR).
> >> > > > > This
> >> > > > > > > > > > refers to
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> entire
> >> > > > > > > > > > > > > > > >> > > > state of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > group coordinator which
> >> is
> >> > > > > replicated
> >> > > > > > > via
> >> > > > > > > > > > the
> >> > > > > > > > > > > > log
> >> > > > > > > > > > > > > > > >> layer. I will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > clarify this in the KIP.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
> >> "each
> >> > > > > state
> >> > > > > > > > > machine
> >> > > > > > > > > > is
> >> > > > > > > > > > > > > > modelled
> >> > > > > > > > > > > > > > > >> as an
> >> > > > > > > > > > > > > > > >> > > > event
> >> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > The idea is to follow a
> >> model
> >> > > > > similar
> >> > > > > > > to
> >> > > > > > > > > > the new
> >> > > > > > > > > > > > > > quorum
> >> > > > > > > > > > > > > > > >> > > > > > controller. We
> >> > > > > > > > > > > > > > > >> > > > > > > > > > will have N threads to
> >> process
> >> > > > > events.
> >> > > > > > > > > Each
> >> > > > > > > > > > > > > > > >> __consumer_offsets
> >> > > > > > > > > > > > > > > >> > > > > > > > > > partition is assigned to
> >> a
> >> > > unique
> >> > > > > > > thread
> >> > > > > > > > > > and all
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> events
> >> > > > > > > > > > > > > > > >> > > > (e.g.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > requests, callbacks,
> >> etc.) are
> >> > > > > > > processed
> >> > > > > > > > > by
> >> > > > > > > > > > this
> >> > > > > > > > > > > > > > > >> thread. This
> >> > > > > > > > > > > > > > > >> > > > > > simplify
> >> > > > > > > > > > > > > > > >> > > > > > > > > > concurrency and will
> >> enable
> >> > > us to
> >> > > > > do
> >> > > > > > > > > > simulation
> >> > > > > > > > > > > > > > testing
> >> > > > > > > > > > > > > > > >> for the
> >> > > > > > > > > > > > > > > >> > > > > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > coordinator.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
> >> state
> >> > > > > machine
> >> > > > > > > per
> >> > > > > > > > > > > > > > > >> *__consumer_offsets*
> >> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine
> >> "per
> >> > > > > consumer
> >> > > > > > > group"
> >> > > > > > > > > > owned
> >> > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > coordinator?
> >> > > > > > > > > > > > > > > >> > > > > > > > > For
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> >> > > > > coordinator
> >> > > > > > > owns 2
> >> > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> groups, and
> >> > > > > > > > > > > > > > > >> > > > > > both
> >> > > > > > > > > > > > > > > >> > > > > > > > > exist in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> >> > > will we
> >> > > > > > > have 1
> >> > > > > > > > > > state
> >> > > > > > > > > > > > > > machine
> >> > > > > > > > > > > > > > > >> for it,
> >> > > > > > > > > > > > > > > >> > > > or
> >> > > > > > > > > > > > > > > >> > > > > > 2?
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.1. The confusion
> >> comes
> >> > > from
> >> > > > > > > there, I
> >> > > > > > > > > > > > think.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> >> > > > > > > > > > "*group.coordinator.threads"
> >> > > > > > > > > > > > *is
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> number
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
> >> machines.
> >> > > But
> >> > > > > I'm
> >> > > > > > > > > > wondering
> >> > > > > > > > > > > > if
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> purpose
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
> >> state of
> >> > > > > each
> >> > > > > > > > > consumer
> >> > > > > > > > > > > > group
> >> > > > > > > > > > > > > > (or
> >> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
> >> heavy
> >> > > > > > > computation,
> >> > > > > > > > > > why
> >> > > > > > > > > > > > > > should we
> >> > > > > > > > > > > > > > > >> need
> >> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.2. The idea is to
> >> have
> >> > > an
> >> > > > > > > ability to
> >> > > > > > > > > > shard
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > processing as
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > computation could be
> >> heavy.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
> >> session
> >> > > > > timeout,
> >> > > > > > > why
> >> > > > > > > > > > does
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> default
> >> > > > > > > > > > > > > > > >> > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
> >> (45s) and
> >> > > > > > > max(60s)? I
> >> > > > > > > > > > > > thought
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> min/max
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> >> > > lower/upper
> >> > > > > > > bound
> >> > > > > > > > > of
> >> > > > > > > > > > it,
> >> > > > > > > > > > > > > no?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > group.consumer.session.timeout.ms
> >> > > > > > > int
> >> > > > > > > > > > 30s The
> >> > > > > > > > > > > > > > > >> timeout to
> >> > > > > > > > > > > > > > > >> > > > detect
> >> > > > > > > > > > > > > > > >> > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> >> > > consumer
> >> > > > > > > group
> >> > > > > > > > > > > > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > group.consumer.min.session.timeout.ms
> >> > > > > > > > > > int 45s
> >> > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> minimum
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > group.consumer.max.session.timeout.ms
> >> > > > > > > > > > int 60s
> >> > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> maximum
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > This is indeed a
> >> mistake. The
> >> > > > > default
> >> > > > > > > > > > session
> >> > > > > > > > > > > > > > timeout
> >> > > > > > > > > > > > > > > >> should
> >> > > > > > > > > > > > > > > >> > > > be 45s
> >> > > > > > > > > > > > > > > >> > > > > > > > > > (the current default).
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default
> >> server side
> >> > > > > > > assignor
> >> > > > > > > > > are
> >> > > > > > > > > > > > > [range,
> >> > > > > > > > > > > > > > > >> uniform],
> >> > > > > > > > > > > > > > > >> > > > > > which means
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to
> >> "range"
> >> > > > > assignor.
> >> > > > > > > I'd
> >> > > > > > > > > > like to
> >> > > > > > > > > > > > > > know
> >> > > > > > > > > > > > > > > >> why not
> >> > > > > > > > > > > > > > > >> > > > > > uniform
> >> > > > > > > > > > > > > > > >> > > > > > > > > one? I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
> >> will
> >> > > > > choose
> >> > > > > > > > > uniform
> >> > > > > > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> (former
> >> > > > > > > > > > > > > > > >> > > > sticky
> >> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> >> > > distribution.
> >> > > > > Any
> >> > > > > > > > > other
> >> > > > > > > > > > > > reason
> >> > > > > > > > > > > > > > we
> >> > > > > > > > > > > > > > > >> choose
> >> > > > > > > > > > > > > > > >> > > > range
> >> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> group.consumer.assignors
> >> > > List
> >> > > > > range,
> >> > > > > > > > > > uniform
> >> > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> server side
> >> > > > > > > > > > > > > > > >> > > > > > assignors.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > The order on the server
> >> side
> >> > > has
> >> > > > > no
> >> > > > > > > > > > influence
> >> > > > > > > > > > > > > > because
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > client
> >> > > > > > > > > > > > > > > >> > > > > > must
> >> > > > > > > > > > > > > > > >> > > > > > > > > > chose the selector that
> >> he
> >> > > wants
> >> > > > > to
> >> > > > > > > use.
> >> > > > > > > > > > There
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > no
> >> > > > > > > > > > > > > > > >> default
> >> > > > > > > > > > > > > > > >> > > > in the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > current proposal. If the
> >> > > assignor
> >> > > > > is
> >> > > > > > > not
> >> > > > > > > > > > > > specified
> >> > > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > client,
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > request is rejected. The
> >> > > default
> >> > > > > > > client
> >> > > > > > > > > > value
> >> > > > > > > > > > > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > `group.remote.assignor`
> >> is
> >> > > > > `uniform`
> >> > > > > > > > > though.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for your very good
> >> > > > > comments,
> >> > > > > > > Luke.
> >> > > > > > > > > I
> >> > > > > > > > > > hope
> >> > > > > > > > > > > > > > that my
> >> > > > > > > > > > > > > > > >> > > > answers
> >> > > > > > > > > > > > > > > >> > > > > > help
> >> > > > > > > > > > > > > > > >> > > > > > > > > > to clarify things. I will
> >> > > update
> >> > > > > the
> >> > > > > > > KIP
> >> > > > > > > > > as
> >> > > > > > > > > > well
> >> > > > > > > > > > > > > > based
> >> > > > > > > > > > > > > > > >> on your
> >> > > > > > > > > > > > > > > >> > > > > > > > > > feedback.
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > Cheers,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > On Mon, Aug 22, 2022 at
> >> 9:29
> >> > > AM
> >> > > > > Luke
> >> > > > > > > Chen
> >> > > > > > > > > <
> >> > > > > > > > > > > > > > > >> showuon@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Hi David,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Thanks for the update.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Some more questions:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1. In Group Coordinator
> >> > > > > section, you
> >> > > > > > > > > > > > mentioned:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > The new group
> >> coordinator
> >> > > will
> >> > > > > > > have a
> >> > > > > > > > > > state
> >> > > > > > > > > > > > > > machine
> >> > > > > > > > > > > > > > > >> per
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets*
> >> > > partitions,
> >> > > > > > > where
> >> > > > > > > > > > each
> >> > > > > > > > > > > > > state
> >> > > > > > > > > > > > > > > >> machine is
> >> > > > > > > > > > > > > > > >> > > > > > modelled
> >> > > > > > > > > > > > > > > >> > > > > > > > > as an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > event loop. Those state
> >> > > machines
> >> > > > > > > will be
> >> > > > > > > > > > > > > executed
> >> > > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> *group.coordinator.threads*
> >> > > > > threads.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> >> > > machine
> >> > > > > are:
> >> > > > > > > > > > "Empty,
> >> > > > > > > > > > > > > > assigning,
> >> > > > > > > > > > > > > > > >> > > > > > reconciling,
> >> > > > > > > > > > > > > > > >> > > > > > > > > stable,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
> >> Consumer
> >> > > > > Group
> >> > > > > > > States
> >> > > > > > > > > > > > > section,
> >> > > > > > > > > > > > > > > >> right?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
> >> "each
> >> > > > > state
> >> > > > > > > > > machine
> >> > > > > > > > > > is
> >> > > > > > > > > > > > > > modelled
> >> > > > > > > > > > > > > > > >> as an
> >> > > > > > > > > > > > > > > >> > > > event
> >> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
> >> state
> >> > > > > machine
> >> > > > > > > per
> >> > > > > > > > > > > > > > > >> *__consumer_offsets*
> >> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine
> >> "per
> >> > > > > consumer
> >> > > > > > > group"
> >> > > > > > > > > > owned
> >> > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > coordinator?
> >> > > > > > > > > > > > > > > >> > > > > > > > > For
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> >> > > > > coordinator
> >> > > > > > > owns 2
> >> > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> groups, and
> >> > > > > > > > > > > > > > > >> > > > > > both
> >> > > > > > > > > > > > > > > >> > > > > > > > > exist in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> >> > > will we
> >> > > > > > > have 1
> >> > > > > > > > > > state
> >> > > > > > > > > > > > > > machine
> >> > > > > > > > > > > > > > > >> for it,
> >> > > > > > > > > > > > > > > >> > > > or
> >> > > > > > > > > > > > > > > >> > > > > > 2?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> >> > > > > > > > > > "*group.coordinator.threads"
> >> > > > > > > > > > > > *is
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> number
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
> >> machines.
> >> > > But
> >> > > > > I'm
> >> > > > > > > > > > wondering
> >> > > > > > > > > > > > if
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> purpose
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
> >> state of
> >> > > > > each
> >> > > > > > > > > consumer
> >> > > > > > > > > > > > group
> >> > > > > > > > > > > > > > (or
> >> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
> >> heavy
> >> > > > > > > computation,
> >> > > > > > > > > > why
> >> > > > > > > > > > > > > > should we
> >> > > > > > > > > > > > > > > >> need
> >> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 2. For the default
> >> value in
> >> > > the
> >> > > > > new
> >> > > > > > > > > > configs:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
> >> session
> >> > > > > timeout,
> >> > > > > > > why
> >> > > > > > > > > > does
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> default
> >> > > > > > > > > > > > > > > >> > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
> >> (45s) and
> >> > > > > > > max(60s)? I
> >> > > > > > > > > > > > thought
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> min/max
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> >> > > lower/upper
> >> > > > > > > bound
> >> > > > > > > > > of
> >> > > > > > > > > > it,
> >> > > > > > > > > > > > > no?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > group.consumer.session.timeout.ms
> >> > > > > > > int
> >> > > > > > > > > > 30s The
> >> > > > > > > > > > > > > > > >> timeout to
> >> > > > > > > > > > > > > > > >> > > > detect
> >> > > > > > > > > > > > > > > >> > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> >> > > consumer
> >> > > > > > > group
> >> > > > > > > > > > > > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > group.consumer.min.session.timeout.ms
> >> > > > > > > > > > int 45s
> >> > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> minimum
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > group.consumer.max.session.timeout.ms
> >> > > > > > > > > > int 60s
> >> > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> maximum
> >> > > > > > > > > > > > > > > >> > > > > > session
> >> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default
> >> server side
> >> > > > > > > assignor
> >> > > > > > > > > are
> >> > > > > > > > > > > > > [range,
> >> > > > > > > > > > > > > > > >> uniform],
> >> > > > > > > > > > > > > > > >> > > > > > which means
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to
> >> "range"
> >> > > > > assignor.
> >> > > > > > > I'd
> >> > > > > > > > > > like to
> >> > > > > > > > > > > > > > know
> >> > > > > > > > > > > > > > > >> why not
> >> > > > > > > > > > > > > > > >> > > > > > uniform
> >> > > > > > > > > > > > > > > >> > > > > > > > > one? I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
> >> will
> >> > > > > choose
> >> > > > > > > > > uniform
> >> > > > > > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> (former
> >> > > > > > > > > > > > > > > >> > > > sticky
> >> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> >> > > distribution.
> >> > > > > Any
> >> > > > > > > > > other
> >> > > > > > > > > > > > reason
> >> > > > > > > > > > > > > > we
> >> > > > > > > > > > > > > > > >> choose
> >> > > > > > > > > > > > > > > >> > > > range
> >> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> group.consumer.assignors
> >> > > List
> >> > > > > range,
> >> > > > > > > > > > uniform
> >> > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> server side
> >> > > > > > > > > > > > > > > >> > > > > > assignors.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Thank you.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > Luke
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Aug 22, 2022 at
> >> > > 2:10 PM
> >> > > > > Luke
> >> > > > > > > > > Chen
> >> > > > > > > > > > <
> >> > > > > > > > > > > > > > > >> showuon@gmail.com
> >> > > > > > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi Sagar,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > I have some thoughts
> >> about
> >> > > > > Kafka
> >> > > > > > > > > Connect
> >> > > > > > > > > > > > > > > >> integrating with
> >> > > > > > > > > > > > > > > >> > > > > > KIP-848,
> >> > > > > > > > > > > > > > > >> > > > > > > > > but I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > think we should have
> >> a
> >> > > > > separate
> >> > > > > > > > > > discussion
> >> > > > > > > > > > > > > > thread
> >> > > > > > > > > > > > > > > >> for the
> >> > > > > > > > > > > > > > > >> > > > Kafka
> >> > > > > > > > > > > > > > > >> > > > > > > > > Connect
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > KIP: Integrating
> >> Kafka
> >> > > Connect
> >> > > > > > > With
> >> > > > > > > > > New
> >> > > > > > > > > > > > > Consumer
> >> > > > > > > > > > > > > > > >> Rebalance
> >> > > > > > > > > > > > > > > >> > > > > > Protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > [1],
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > and let this
> >> discussion
> >> > > thread
> >> > > > > > > focus
> >> > > > > > > > > on
> >> > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> rebalance
> >> > > > > > > > > > > > > > > >> > > > > > protocol,
> >> > > > > > > > > > > > > > > >> > > > > > > > > WDYT?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > [1]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > >
> >> > >
> >> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > Thank you.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > Luke
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Aug 12, 2022
> >> at
> >> > > 9:31
> >> > > > > PM
> >> > > > > > > Sagar
> >> > > > > > > > > <
> >> > > > > > > > > > > > > > > >> > > > > > sagarmeansocean@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thank you
> >> Guozhang/David
> >> > > for
> >> > > > > the
> >> > > > > > > > > > feedback.
> >> > > > > > > > > > > > > > Looks
> >> > > > > > > > > > > > > > > >> like
> >> > > > > > > > > > > > > > > >> > > > there's
> >> > > > > > > > > > > > > > > >> > > > > > > > > agreement on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> using separate APIs
> >> for
> >> > > > > Connect.
> >> > > > > > > I
> >> > > > > > > > > > would
> >> > > > > > > > > > > > > > revisit
> >> > > > > > > > > > > > > > > >> the doc
> >> > > > > > > > > > > > > > > >> > > > and
> >> > > > > > > > > > > > > > > >> > > > > > see
> >> > > > > > > > > > > > > > > >> > > > > > > > > what
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> changes are to be
> >> made.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thanks!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Sagar.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> On Tue, Aug 9, 2022
> >> at
> >> > > 7:11
> >> > > > > PM
> >> > > > > > > David
> >> > > > > > > > > > Jacot
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> <dj...@confluent.io.invalid>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Hi Sagar,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Thanks for the
> >> feedback
> >> > > > > and the
> >> > > > > > > > > > document.
> >> > > > > > > > > > > > > > That's
> >> > > > > > > > > > > > > > > >> really
> >> > > > > > > > > > > > > > > >> > > > > > helpful. I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will take a look
> >> at it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Overall, it seems
> >> to me
> >> > > > > that
> >> > > > > > > both
> >> > > > > > > > > > Connect
> >> > > > > > > > > > > > > > and the
> >> > > > > > > > > > > > > > > >> > > > Consumer
> >> > > > > > > > > > > > > > > >> > > > > > could
> >> > > > > > > > > > > > > > > >> > > > > > > > > share
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the same
> >> underlying
> >> > > > > "engine".
> >> > > > > > > The
> >> > > > > > > > > > main
> >> > > > > > > > > > > > > > > >> difference is
> >> > > > > > > > > > > > > > > >> > > > that
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > Consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigns
> >> > > topic-partitions to
> >> > > > > > > members
> >> > > > > > > > > > > > whereas
> >> > > > > > > > > > > > > > > >> Connect
> >> > > > > > > > > > > > > > > >> > > > assigns
> >> > > > > > > > > > > > > > > >> > > > > > tasks
> >> > > > > > > > > > > > > > > >> > > > > > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > workers. I see two
> >> > > ways to
> >> > > > > move
> >> > > > > > > > > > forward:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) We extend the
> >> new
> >> > > > > proposed
> >> > > > > > > APIs
> >> > > > > > > > > to
> >> > > > > > > > > > > > > support
> >> > > > > > > > > > > > > > > >> different
> >> > > > > > > > > > > > > > > >> > > > > > resource
> >> > > > > > > > > > > > > > > >> > > > > > > > > types
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g. partitions,
> >> > > tasks,
> >> > > > > > > etc.); or
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 2) We use new
> >> dedicated
> >> > > > > APIs
> >> > > > > > > for
> >> > > > > > > > > > Connect.
> >> > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> dedicated
> >> > > > > > > > > > > > > > > >> > > > APIs
> >> > > > > > > > > > > > > > > >> > > > > > > > > would be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > similar to the new
> >> > > ones but
> >> > > > > > > > > > different on
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > content/resources and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they would rely
> >> on the
> >> > > same
> >> > > > > > > engine
> >> > > > > > > > > > on the
> >> > > > > > > > > > > > > > > >> coordinator
> >> > > > > > > > > > > > > > > >> > > > side.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I personally lean
> >> > > towards
> >> > > > > 2)
> >> > > > > > > > > because
> >> > > > > > > > > > I am
> >> > > > > > > > > > > > > > not a
> >> > > > > > > > > > > > > > > >> fan of
> >> > > > > > > > > > > > > > > >> > > > > > > > > overcharging
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > APIs to serve
> >> different
> >> > > > > > > purposes.
> >> > > > > > > > > > That
> >> > > > > > > > > > > > > being
> >> > > > > > > > > > > > > > > >> said, I am
> >> > > > > > > > > > > > > > > >> > > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > > opposed to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) if we can find
> >> an
> >> > > > > elegant
> >> > > > > > > way to
> >> > > > > > > > > > do
> >> > > > > > > > > > > > it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I think that we
> >> can
> >> > > > > continue to
> >> > > > > > > > > > discuss
> >> > > > > > > > > > > > it
> >> > > > > > > > > > > > > > here
> >> > > > > > > > > > > > > > > >> for now
> >> > > > > > > > > > > > > > > >> > > > in
> >> > > > > > > > > > > > > > > >> > > > > > order
> >> > > > > > > > > > > > > > > >> > > > > > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ensure that this
> >> KIP is
> >> > > > > > > compatible
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > what
> >> > > > > > > > > > > > > > we
> >> > > > > > > > > > > > > > > >> will do
> >> > > > > > > > > > > > > > > >> > > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > Connect in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the future.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Best,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > On Mon, Aug 8,
> >> 2022 at
> >> > > > > 2:41 PM
> >> > > > > > > > > David
> >> > > > > > > > > > > > Jacot
> >> > > > > > > > > > > > > <
> >> > > > > > > > > > > > > > > >> > > > > > djacot@confluent.io>
> >> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Hi all,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > I am back from
> >> > > vacation.
> >> > > > > I
> >> > > > > > > will
> >> > > > > > > > > go
> >> > > > > > > > > > > > > through
> >> > > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> address
> >> > > > > > > > > > > > > > > >> > > > > > your
> >> > > > > > > > > > > > > > > >> > > > > > > > > comments
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > in the coming
> >> days.
> >> > > > > Thanks
> >> > > > > > > for
> >> > > > > > > > > your
> >> > > > > > > > > > > > > > feedback.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Cheers,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Aug 3,
> >> 2022
> >> > > at
> >> > > > > 10:05
> >> > > > > > > PM
> >> > > > > > > > > > Gregory
> >> > > > > > > > > > > > > > Harris
> >> > > > > > > > > > > > > > > >> <
> >> > > > > > > > > > > > > > > >> > > > > > > > > gharris1727@gmail.com
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Hey All!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Thanks for
> >> the KIP,
> >> > > > > it's
> >> > > > > > > > > > wonderful to
> >> > > > > > > > > > > > > see
> >> > > > > > > > > > > > > > > >> > > > cooperative
> >> > > > > > > > > > > > > > > >> > > > > > > > > rebalancing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > making it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > down the
> >> stack!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > I had a few
> >> > > questions:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 1. The
> >> 'Rejected
> >> > > > > > > Alternatives'
> >> > > > > > > > > > > > section
> >> > > > > > > > > > > > > > > >> describes how
> >> > > > > > > > > > > > > > > >> > > > > > member
> >> > > > > > > > > > > > > > > >> > > > > > > > > epoch
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > advance in
> >> step
> >> > > with
> >> > > > > the
> >> > > > > > > group
> >> > > > > > > > > > epoch
> >> > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> assignment
> >> > > > > > > > > > > > > > > >> > > > > > epoch
> >> > > > > > > > > > > > > > > >> > > > > > > > > values. I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > that this is
> >> a good
> >> > > > > idea
> >> > > > > > > for
> >> > > > > > > > > the
> >> > > > > > > > > > > > > reasons
> >> > > > > > > > > > > > > > > >> described
> >> > > > > > > > > > > > > > > >> > > > in
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > KIP. When
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > protocol is
> >> > > > > incrementally
> >> > > > > > > > > > assigning
> >> > > > > > > > > > > > > > > >> partitions to a
> >> > > > > > > > > > > > > > > >> > > > > > worker,
> >> > > > > > > > > > > > > > > >> > > > > > > > > what
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> member
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > epoch does
> >> each
> >> > > > > incremental
> >> > > > > > > > > > > > assignment
> >> > > > > > > > > > > > > > use?
> >> > > > > > > > > > > > > > > >> Are
> >> > > > > > > > > > > > > > > >> > > > member
> >> > > > > > > > > > > > > > > >> > > > > > epochs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> re-used,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > a single
> >> member
> >> > > epoch
> >> > > > > can
> >> > > > > > > > > > correspond
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> multiple
> >> > > > > > > > > > > > > > > >> > > > > > different
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (monotonically
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > larger)
> >> > > assignments?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 2. Is the
> >> > > Assignor's
> >> > > > > > > 'Reason'
> >> > > > > > > > > > field
> >> > > > > > > > > > > > > > opaque
> >> > > > > > > > > > > > > > > >> to the
> >> > > > > > > > > > > > > > > >> > > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> coordinator? If
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > not, should
> >> custom
> >> > > > > > > client-side
> >> > > > > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > implementations
> >> > > > > > > > > > > > > > > >> > > > > > > > > interact
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Reason field,
> >> and
> >> > > how
> >> > > > > is
> >> > > > > > > its
> >> > > > > > > > > > common
> >> > > > > > > > > > > > > > meaning
> >> > > > > > > > > > > > > > > >> agreed
> >> > > > > > > > > > > > > > > >> > > > > > upon? If
> >> > > > > > > > > > > > > > > >> > > > > > > > > so, what
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > benefit of a
> >> > > distinct
> >> > > > > > > Reason
> >> > > > > > > > > > field
> >> > > > > > > > > > > > over
> >> > > > > > > > > > > > > > > >> including
> >> > > > > > > > > > > > > > > >> > > > such
> >> > > > > > > > > > > > > > > >> > > > > > > > > functionality
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > opaque
> >> metadata?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 3. The
> >> following is
> >> > > > > > > included in
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > KIP:
> >> > > > > > > > > > > > > > > >> "Thanks to
> >> > > > > > > > > > > > > > > >> > > > > > this, the
> >> > > > > > > > > > > > > > > >> > > > > > > > > input
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client side
> >> > > assignor is
> >> > > > > > > > > entirely
> >> > > > > > > > > > > > driven
> >> > > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > > >> the group
> >> > > > > > > > > > > > > > > >> > > > > > > > > coordinator.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > consumer is no
> >> > > longer
> >> > > > > > > > > > responsible for
> >> > > > > > > > > > > > > > > >> maintaining
> >> > > > > > > > > > > > > > > >> > > > any
> >> > > > > > > > > > > > > > > >> > > > > > state
> >> > > > > > > > > > > > > > > >> > > > > > > > > besides
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> its
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assigned
> >> > > partitions."
> >> > > > > Does
> >> > > > > > > this
> >> > > > > > > > > > mean
> >> > > > > > > > > > > > > > that the
> >> > > > > > > > > > > > > > > >> > > > > > client-side
> >> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> MAY
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > incorporate
> >> > > additional
> >> > > > > > > > > > non-Metadata
> >> > > > > > > > > > > > > state
> >> > > > > > > > > > > > > > > >> (such as
> >> > > > > > > > > > > > > > > >> > > > > > partition
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > throughput,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > cpu/memory
> >> metrics,
> >> > > > > config
> >> > > > > > > > > > topics,
> >> > > > > > > > > > > > > etc),
> >> > > > > > > > > > > > > > or
> >> > > > > > > > > > > > > > > >> that
> >> > > > > > > > > > > > > > > >> > > > > > additional
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > non-Metadata
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > state SHOULD
> >> NOT be
> >> > > > > used?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 4. I see that
> >> > > there are
> >> > > > > > > > > separate
> >> > > > > > > > > > > > > classes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > for
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > org.apache.kafka.server.group.consumer.PartitionAssignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > org.apache.kafka.clients.consumer.PartitionAssignor
> >> > > > > > > > > > > > > > > >> > > > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > seem to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > overlap
> >> > > significantly.
> >> > > > > Is
> >> > > > > > > it
> >> > > > > > > > > > possible
> >> > > > > > > > > > > > > for
> >> > > > > > > > > > > > > > > >> these two
> >> > > > > > > > > > > > > > > >> > > > > > > > > implementations
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > unified? This
> >> would
> >> > > > > serve
> >> > > > > > > to
> >> > > > > > > > > > promote
> >> > > > > > > > > > > > > > feature
> >> > > > > > > > > > > > > > > >> parity
> >> > > > > > > > > > > > > > > >> > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > server-side
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client-side
> >> > > assignors,
> >> > > > > and
> >> > > > > > > > > would
> >> > > > > > > > > > also
> >> > > > > > > > > > > > > > > >> facilitate
> >> > > > > > > > > > > > > > > >> > > > > > operational
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flexibility in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > certain
> >> > > situations. For
> >> > > > > > > > > example,
> >> > > > > > > > > > if a
> >> > > > > > > > > > > > > > > >> server-side
> >> > > > > > > > > > > > > > > >> > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > has some
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > poor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > behavior and
> >> needs
> >> > > a
> >> > > > > patch,
> >> > > > > > > > > > deploying
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> patched
> >> > > > > > > > > > > > > > > >> > > > > > assignor to
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and switching
> >> one
> >> > > > > consumer
> >> > > > > > > > > group
> >> > > > > > > > > > to a
> >> > > > > > > > > > > > > > > >> client-side
> >> > > > > > > > > > > > > > > >> > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > may be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > faster
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and less
> >> risky than
> >> > > > > > > patching
> >> > > > > > > > > all
> >> > > > > > > > > > of
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> brokers.
> >> > > > > > > > > > > > > > > >> > > > With
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > currently
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > proposed
> >> distinct
> >> > > > > APIs, a
> >> > > > > > > > > > non-trivial
> >> > > > > > > > > > > > > > > >> > > > reimplementation
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assembled,
> >> and if
> >> > > the
> >> > > > > two
> >> > > > > > > APIs
> >> > > > > > > > > > have
> >> > > > > > > > > > > > > > diverged
> >> > > > > > > > > > > > > > > >> > > > > > significantly,
> >> > > > > > > > > > > > > > > >> > > > > > > > > then it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > possible that
> >> a
> >> > > > > > > > > reimplementation
> >> > > > > > > > > > > > would
> >> > > > > > > > > > > > > > not be
> >> > > > > > > > > > > > > > > >> > > > possible.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > --
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Greg Harris
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > gharris1727@gmail.com
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > github.com/gharris1727
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Aug 3,
> >> > > 2022 at
> >> > > > > > > 8:39 AM
> >> > > > > > > > > > Sagar
> >> > > > > > > > > > > > <
> >> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Hi
> >> > > Guozhang/David,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > I created a
> >> > > > > confluence
> >> > > > > > > page
> >> > > > > > > > > to
> >> > > > > > > > > > > > > discuss
> >> > > > > > > > > > > > > > how
> >> > > > > > > > > > > > > > > >> Connect
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > based on
> >> the new
> >> > > > > > > rebalance
> >> > > > > > > > > > > > protocol.
> >> > > > > > > > > > > > > > > >> Here's the
> >> > > > > > > > > > > > > > > >> > > > page:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > >
> >> > >
> >> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > It's also
> >> pretty
> >> > > > > longish
> >> > > > > > > and
> >> > > > > > > > > I
> >> > > > > > > > > > have
> >> > > > > > > > > > > > > > tried
> >> > > > > > > > > > > > > > > >> to keep
> >> > > > > > > > > > > > > > > >> > > > a
> >> > > > > > > > > > > > > > > >> > > > > > format
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> similar to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > KIP-848.
> >> Let me
> >> > > know
> >> > > > > > > what you
> >> > > > > > > > > > > > think.
> >> > > > > > > > > > > > > > Also,
> >> > > > > > > > > > > > > > > >> do you
> >> > > > > > > > > > > > > > > >> > > > > > think this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > moved to a
> >> > > separate
> >> > > > > > > > > discussion
> >> > > > > > > > > > > > thread
> >> > > > > > > > > > > > > > or
> >> > > > > > > > > > > > > > > >> is this
> >> > > > > > > > > > > > > > > >> > > > one
> >> > > > > > > > > > > > > > > >> > > > > > fine?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Sagar.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul
> >> 26,
> >> > > 2022
> >> > > > > at
> >> > > > > > > 7:37
> >> > > > > > > > > AM
> >> > > > > > > > > > > > > Sagar <
> >> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Hello
> >> Guozhang,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thank you
> >> so
> >> > > much
> >> > > > > for
> >> > > > > > > the
> >> > > > > > > > > > doc on
> >> > > > > > > > > > > > > > Kafka
> >> > > > > > > > > > > > > > > >> Streams.
> >> > > > > > > > > > > > > > > >> > > > > > Sure, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > would do
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > analysis
> >> and
> >> > > come
> >> > > > > up
> >> > > > > > > with
> >> > > > > > > > > > such a
> >> > > > > > > > > > > > > > > >> document.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Sagar.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue,
> >> Jul 26,
> >> > > > > 2022 at
> >> > > > > > > > > 4:47
> >> > > > > > > > > > AM
> >> > > > > > > > > > > > > > Guozhang
> >> > > > > > > > > > > > > > > >> Wang <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wangguoz@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Hello
> >> Sagar,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> It would
> >> be
> >> > > great
> >> > > > > if
> >> > > > > > > you
> >> > > > > > > > > > could
> >> > > > > > > > > > > > > come
> >> > > > > > > > > > > > > > > >> back with
> >> > > > > > > > > > > > > > > >> > > > some
> >> > > > > > > > > > > > > > > >> > > > > > > > > analysis on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > how to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> implement the
> >> > > > > Connect
> >> > > > > > > side
> >> > > > > > > > > > > > > > integration
> >> > > > > > > > > > > > > > > >> with
> >> > > > > > > > > > > > > > > >> > > > the new
> >> > > > > > > > > > > > > > > >> > > > > > > > > protocol;
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> so
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > far
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> besides
> >> > > > > leveraging on
> >> > > > > > > the
> >> > > > > > > > > > new
> >> > > > > > > > > > > > > > "protocol
> >> > > > > > > > > > > > > > > >> type"
> >> > > > > > > > > > > > > > > >> > > > we
> >> > > > > > > > > > > > > > > >> > > > > > did not
> >> > > > > > > > > > > > > > > >> > > > > > > > > yet
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > through
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> Connect
> >> > > side
> >> > > > > > > > > > > > implementations.
> >> > > > > > > > > > > > > > For
> >> > > > > > > > > > > > > > > >> Streams
> >> > > > > > > > > > > > > > > >> > > > > > here's a
> >> > > > > > > > > > > > > > > >> > > > > > > > > draft of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> integration
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> plan:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > >
> >> > > > > > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > >
> >> > > > > > > > > > > > > > > >>
> >> > > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > >
> >> > >
> >> https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> just FYI
> >> for
> >> > > your
> >> > > > > > > analysis
> >> > > > > > > > > > on
> >> > > > > > > > > > > > > > Connect.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> On Tue,
> >> Jul
> >> > > 19,
> >> > > > > 2022
> >> > > > > > > at
> >> > > > > > > > > > 10:48 PM
> >> > > > > > > > > > > > > > Sagar <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > sagarmeansocean@gmail.com
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Hi
> >> David,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thank
> >> you
> >> > > for
> >> > > > > your
> >> > > > > > > > > > response.
> >> > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> reason I
> >> > > > > > > > > > > > > > > >> > > > thought
> >> > > > > > > > > > > > > > > >> > > > > > > > > connect can
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > also fit
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> into
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > this
> >> new
> >> > > scheme
> >> > > > > is
> >> > > > > > > that
> >> > > > > > > > > > even
> >> > > > > > > > > > > > > > today the
> >> > > > > > > > > > > > > > > >> > > > connect
> >> > > > > > > > > > > > > > > >> > > > > > uses a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> WorkerCoordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> extending
> >> > > from
> >> > > > > > > > > > > > > > AbstractCoordinator to
> >> > > > > > > > > > > > > > > >> empower
> >> > > > > > > > > > > > > > > >> > > > > > > > > rebalances of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > tasks/connectors.
> >> > > > > > > The
> >> > > > > > > > > > > > > > > >> WorkerCoordinator sets
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocolType()
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> connect
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > and
> >> uses the
> >> > > > > > > metadata()
> >> > > > > > > > > > method
> >> > > > > > > > > > > > > by
> >> > > > > > > > > > > > > > > >> plumbing
> >> > > > > > > > > > > > > > > >> > > > into
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > JoinGroupRequestProtocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > I
> >> think the
> >> > > > > changes
> >> > > > > > > to
> >> > > > > > > > > > support
> >> > > > > > > > > > > > > > > >> connect would
> >> > > > > > > > > > > > > > > >> > > > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > similar at a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > high
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > level
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> >> changes
> >> > > in
> >> > > > > > > streams
> >> > > > > > > > > > mainly
> >> > > > > > > > > > > > > > because
> >> > > > > > > > > > > > > > > >> of the
> >> > > > > > > > > > > > > > > >> > > > > > Client
> >> > > > > > > > > > > > > > > >> > > > > > > > > side
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assignors
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > being
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > used in
> >> > > both.
> >> > > > > At an
> >> > > > > > > > > > > > > implementation
> >> > > > > > > > > > > > > > > >> level, we
> >> > > > > > > > > > > > > > > >> > > > > > might
> >> > > > > > > > > > > > > > > >> > > > > > > > > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> make
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > a lot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> changes to
> >> > > get
> >> > > > > onto
> >> > > > > > > this
> >> > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> assignment
> >> > > > > > > > > > > > > > > >> > > > protocol
> >> > > > > > > > > > > > > > > >> > > > > > like
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> enhancing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> JoinGroup
> >> > > > > > > > > > request/response and
> >> > > > > > > > > > > > > > > >> SyncGroup and
> >> > > > > > > > > > > > > > > >> > > > > > using
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > ConsumerGroupHeartbeat
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > API etc
> >> > > again on
> >> > > > > > > similar
> >> > > > > > > > > > lines
> >> > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> streams (or
> >> > > > > > > > > > > > > > > >> > > > > > there
> >> > > > > > > > > > > > > > > >> > > > > > > > > might be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> deviations). I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > would
> >> try to
> >> > > > > > > perform a
> >> > > > > > > > > > > > detailed
> >> > > > > > > > > > > > > > > >> analysis of
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > same
> >> > > > > > > > > > > > > > > >> > > > > > > > > and we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> separate
> >> > > > > discussion
> >> > > > > > > > > > thread for
> >> > > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > > >> as that
> >> > > > > > > > > > > > > > > >> > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > derail this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > discussion
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> thread. Let
> >> > > me
> >> > > > > know
> >> > > > > > > if
> >> > > > > > > > > > that
> >> > > > > > > > > > > > > sounds
> >> > > > > > > > > > > > > > > >> good to
> >> > > > > > > > > > > > > > > >> > > > you.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thanks!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Sagar.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > On
> >> Fri, Jul
> >> > > 15,
> >> > > > > > > 2022 at
> >> > > > > > > > > > 5:47
> >> > > > > > > > > > > > PM
> >> > > > > > > > > > > > > > David
> >> > > > > > > > > > > > > > > >> Jacot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > <djacot@confluent.io.invalid
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Hi
> >> Sagar,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> Thanks for
> >> > > > > your
> >> > > > > > > > > > comments.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 1)
> >> Yes.
> >> > > That
> >> > > > > > > refers to
> >> > > > > > > > > > > > > > > >> `Assignment#error`.
> >> > > > > > > > > > > > > > > >> > > > > > Sure, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > can
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > mention it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 2)
> >> The
> >> > > idea
> >> > > > > is to
> >> > > > > > > > > > > > transition C
> >> > > > > > > > > > > > > > from
> >> > > > > > > > > > > > > > > >> his
> >> > > > > > > > > > > > > > > >> > > > current
> >> > > > > > > > > > > > > > > >> > > > > > > > > assignment
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > his
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> target
> >> > > > > assignment
> >> > > > > > > when
> >> > > > > > > > > > he
> >> > > > > > > > > > > > can
> >> > > > > > > > > > > > > > move
> >> > > > > > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > epoch 3.
> >> > > > > > > > > > > > > > > >> > > > > > When
> >> > > > > > > > > > > > > > > >> > > > > > > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > happens,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> member
> >> > > > > assignment
> >> > > > > > > is
> >> > > > > > > > > > updated
> >> > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> persisted
> >> > > > > > > > > > > > > > > >> > > > > > with all
> >> > > > > > > > > > > > > > > >> > > > > > > > > its
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigned
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> partitions
> >> > > > > even if
> >> > > > > > > > > they
> >> > > > > > > > > > are
> >> > > > > > > > > > > > > not
> >> > > > > > > > > > > > > > all
> >> > > > > > > > > > > > > > > >> revoked
> >> > > > > > > > > > > > > > > >> > > > > > yet. In
> >> > > > > > > > > > > > > > > >> > > > > > > > > other
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > words, the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> member
> >> > > > > assignment
> >> > > > > > > > > > becomes
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > target
> >> > > > > > > > > > > > > > > >> > > > > > assignment.
> >> > > > > > > > > > > > > > > >> > > > > > > > > This is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > basically
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > optimization
> >> > > > > to
> >> > > > > > > avoid
> >> > > > > > > > > > having
> >> > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> write all
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > changes to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> examples
> >> > > are
> >> > > > > > > based on
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > persisted
> >> > > > > > > > > > > > > > > >> state
> >> > > > > > > > > > > > > > > >> > > > so I
> >> > > > > > > > > > > > > > > >> > > > > > > > > understand
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> confusion.
> >> > > > > Let me
> >> > > > > > > see
> >> > > > > > > > > > if I
> >> > > > > > > > > > > > can
> >> > > > > > > > > > > > > > > >> improve
> >> > > > > > > > > > > > > > > >> > > > this in
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > description.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 3)
> >> > > Regarding
> >> > > > > > > Connect,
> >> > > > > > > > > it
> >> > > > > > > > > > > > could
> >> > > > > > > > > > > > > > > >> reuse the
> >> > > > > > > > > > > > > > > >> > > > > > protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > with a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client side
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> assignor
> >> > > if it
> >> > > > > > > fits in
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> protocol. The
> >> > > > > > > > > > > > > > > >> > > > > > assignment
> >> > > > > > > > > > > > > > > >> > > > > > > > > is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > topicid-partitions +
> >> > > > > > > > > > > > metadata,
> >> > > > > > > > > > > > > > could
> >> > > > > > > > > > > > > > > >> > > > Connect
> >> > > > > > > > > > > > > > > >> > > > > > fit
> >> > > > > > > > > > > > > > > >> > > > > > > > > into this?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Best,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > On
> >> Fri,
> >> > > Jul
> >> > > > > 15,
> >> > > > > > > 2022
> >> > > > > > > > > at
> >> > > > > > > > > > 1:55
> >> > > > > > > > > > > > > PM
> >> > > > > > > > > > > > > > > >> Sagar <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > sagarmeansocean@gmail.com>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Hi
> >> > > David,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> Thanks
> >> > > for
> >> > > > > the
> >> > > > > > > KIP.
> >> > > > > > > > > I
> >> > > > > > > > > > just
> >> > > > > > > > > > > > > had
> >> > > > > > > > > > > > > > > >> minor
> >> > > > > > > > > > > > > > > >> > > > > > observations:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 1)
> >> In
> >> > > the
> >> > > > > > > Assignment
> >> > > > > > > > > > Error
> >> > > > > > > > > > > > > > > >> section in
> >> > > > > > > > > > > > > > > >> > > > Client
> >> > > > > > > > > > > > > > > >> > > > > > Side
> >> > > > > > > > > > > > > > > >> > > > > > > > > mode
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Assignment
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> process,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > you
> >> > > > > mentioned
> >> > > > > > > => `In
> >> > > > > > > > > > this
> >> > > > > > > > > > > > > > case,
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > client
> >> > > > > > > > > > > > > > > >> > > > > > side
> >> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> can
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > return
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> error
> >> > > to the
> >> > > > > > > group
> >> > > > > > > > > > > > > > coordinator`.
> >> > > > > > > > > > > > > > > >> In this
> >> > > > > > > > > > > > > > > >> > > > > > case are
> >> > > > > > > > > > > > > > > >> > > > > > > > > you
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > referring to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> Assignor
> >> > > > > > > returning
> >> > > > > > > > > an
> >> > > > > > > > > > > > > > > >> AssignmentError
> >> > > > > > > > > > > > > > > >> > > > that's
> >> > > > > > > > > > > > > > > >> > > > > > > > > listed down
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > towards
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > end?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > If
> >> yes,
> >> > > do
> >> > > > > you
> >> > > > > > > think
> >> > > > > > > > > > it
> >> > > > > > > > > > > > > would
> >> > > > > > > > > > > > > > > >> make sense
> >> > > > > > > > > > > > > > > >> > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > mention this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > explicitly
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > here?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 2)
> >> In
> >> > > the
> >> > > > > Case
> >> > > > > > > > > Studies
> >> > > > > > > > > > > > > > section, I
> >> > > > > > > > > > > > > > > >> have a
> >> > > > > > > > > > > > > > > >> > > > > > slight
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> confusion,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > sure
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> if
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> others
> >> > > have
> >> > > > > the
> >> > > > > > > > > same.
> >> > > > > > > > > > > > > Consider
> >> > > > > > > > > > > > > > > >> this step:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> When B
> >> > > > > > > heartbeats,
> >> > > > > > > > > the
> >> > > > > > > > > > > > group
> >> > > > > > > > > > > > > > > >> coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > transitions him
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > epoch
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 3
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> because
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > B
> >> has no
> >> > > > > > > partitions
> >> > > > > > > > > to
> >> > > > > > > > > > > > > > revoke. It
> >> > > > > > > > > > > > > > > >> > > > persists
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > change and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > reply.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > Group
> >> > > > > > > (epoch=3)
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > A
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > B
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > C
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > Target
> >> > > > > > > > > Assignment
> >> > > > > > > > > > > > > > (epoch=3)
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > A -
> >> > > > > > > > > > > > > partitions=[foo-0]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > B -
> >> > > > > > > > > > > > > partitions=[foo-2]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > C -
> >> > > > > > > > > > > > > partitions=[foo-1]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > Member
> >> > > > > > > > > Assignment
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > A -
> >> > > > > > > epoch=2,
> >> > > > > > > > > > > > > > > >> partitions=[foo-0,
> >> > > > > > > > > > > > > > > >> > > > > > foo-1]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > B -
> >> > > > > > > epoch=3,
> >> > > > > > > > > > > > > > > >> partitions=[foo-2]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> -
> >> > > C -
> >> > > > > > > epoch=3,
> >> > > > > > > > > > > > > > > >> partitions=[foo-1]
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> When C
> >> > > > > > > heartbeats,
> >> > > > > > > > > it
> >> > > > > > > > > > > > > > transitions
> >> > > > > > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > epoch 3
> >> > > > > > > > > > > > > > > >> > > > > > but
> >> > > > > > > > > > > > > > > >> > > > > > > > > cannot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> get
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > foo-1
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> yet.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > Here,it's
> >> > > > > > > mentioned
> >> > > > > > > > > > that
> >> > > > > > > > > > > > > > member C
> >> > > > > > > > > > > > > > > >> can't
> >> > > > > > > > > > > > > > > >> > > > get
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > foo-1
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > partition
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > yet,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> based
> >> > > on the
> >> > > > > > > > > > description
> >> > > > > > > > > > > > > > above,
> >> > > > > > > > > > > > > > > >> it seems
> >> > > > > > > > > > > > > > > >> > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > already has
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Do you
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> would be
> >> > > > > better
> >> > > > > > > to
> >> > > > > > > > > > remove
> >> > > > > > > > > > > > it
> >> > > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> > > > populate it
> >> > > > > > > > > > > > > > > >> > > > > > only
> >> > > > > > > > > > > > > > > >> > > > > > > > > when it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > actually
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> gets
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > I
> >> see
> >> > > this
> >> > > > > in a
> >> > > > > > > lot
> >> > > > > > > > > of
> >> > > > > > > > > > > > other
> >> > > > > > > > > > > > > > > >> places, so
> >> > > > > > > > > > > > > > > >> > > > have
> >> > > > > > > > > > > > > > > >> > > > > > I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> understood it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> incorrectly
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > Regarding
> >> > > > > > > connect ,
> >> > > > > > > > > it
> >> > > > > > > > > > > > might
> >> > > > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> out of
> >> > > > > > > > > > > > > > > >> > > > scope
> >> > > > > > > > > > > > > > > >> > > > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > discussion,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > from
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> what I
> >> > > > > > > understood it
> >> > > > > > > > > > would
> >> > > > > > > > > > > > > > > >> probably be
> >> > > > > > > > > > > > > > > >> > > > > > running in
> >> > > > > > > > > > > > > > > >> > > > > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> assignor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > mode
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> even on
> >> > > the
> >> > > > > new
> >> > > > > > > > > > rebalance
> >> > > > > > > > > > > > > > > >> protocol as it
> >> > > > > > > > > > > > > > > >> > > > has
> >> > > > > > > > > > > > > > > >> > > > > > its
> >> > > > > > > > > > > > > > > >> > > > > > > > > own
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Custom
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > Assignors(Eager
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > and
> >> > > > > > > > > > > > IncrementalCooperative).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> Thanks!
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> Sagar.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > On
> >> Fri,
> >> > > Jul
> >> > > > > 15,
> >> > > > > > > 2022
> >> > > > > > > > > > at
> >> > > > > > > > > > > > 5:00
> >> > > > > > > > > > > > > > PM
> >> > > > > > > > > > > > > > > >> David
> >> > > > > > > > > > > > > > > >> > > > Jacot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > > > > > > <djacot@confluent.io.invalid
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> Thanks
> >> > > > > > > Hector! Our
> >> > > > > > > > > > goal
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> move
> >> > > > > > > > > > > > > > > >> > > > forward
> >> > > > > > > > > > > > > > > >> > > > > > with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > specialized API
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > instead of
> >> > > > > > > relying
> >> > > > > > > > > > on
> >> > > > > > > > > > > > one
> >> > > > > > > > > > > > > > > >> generic API.
> >> > > > > > > > > > > > > > > >> > > > For
> >> > > > > > > > > > > > > > > >> > > > > > > > > Connect, we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can apply
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> exact
> >> > > same
> >> > > > > > > pattern
> >> > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> reuse/share the
> >> > > > > > > > > > > > > > > >> > > > core
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > implementation on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> server
> >> > > > > side.
> >> > > > > > > For
> >> > > > > > > > > the
> >> > > > > > > > > > > > > schema
> >> > > > > > > > > > > > > > > >> registry, I
> >> > > > > > > > > > > > > > > >> > > > > > think
> >> > > > > > > > > > > > > > > >> > > > > > > > > that we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consider
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > having a
> >> > > > > > > tailored
> >> > > > > > > > > > API to
> >> > > > > > > > > > > > > do
> >> > > > > > > > > > > > > > > >> simple
> >> > > > > > > > > > > > > > > >> > > > > > > > > membership/leader
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > election.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> Best,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> On
> >> > > Fri,
> >> > > > > Jul
> >> > > > > > > 15,
> >> > > > > > > > > > 2022 at
> >> > > > > > > > > > > > > > 10:22
> >> > > > > > > > > > > > > > > >> AM Ismael
> >> > > > > > > > > > > > > > > >> > > > > > Juma <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ismael@juma.me.uk
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > Three
> >> > > > > quick
> >> > > > > > > > > > comments:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > 1.
> >> > > > > Regarding
> >> > > > > > > > > > > > > > > >> java.util.regex.Pattern
> >> > > > > > > > > > > > > > > >> > > > vs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> > > > > > > com.google.re2j.Pattern,
> >> > > > > > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > document the
> >> > > > > > > > > > > > differences
> >> > > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > detail
> >> > > > > > > > > > > > > > > >> > > > > > before
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deciding
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > way
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> or
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > another.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > That
> >> > > > > said,
> >> > > > > > > if
> >> > > > > > > > > > people
> >> > > > > > > > > > > > > pass
> >> > > > > > > > > > > > > > > >> > > > > > > > > java.util.regex.Pattern,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expect
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > their
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > semantics
> >> > > > > > > to be
> >> > > > > > > > > > > > honored.
> >> > > > > > > > > > > > > > If
> >> > > > > > > > > > > > > > > >> we are
> >> > > > > > > > > > > > > > > >> > > > doing
> >> > > > > > > > > > > > > > > >> > > > > > > > > something
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > different,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> then
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > should
> >> > > > > > > consider
> >> > > > > > > > > > adding
> >> > > > > > > > > > > > > an
> >> > > > > > > > > > > > > > > >> overload
> >> > > > > > > > > > > > > > > >> > > > with
> >> > > > > > > > > > > > > > > >> > > > > > our own
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Pattern
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > class
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > (I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > don't
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > think
> >> > > > > we'd
> >> > > > > > > want
> >> > > > > > > > > to
> >> > > > > > > > > > > > > expose
> >> > > > > > > > > > > > > > > >> re2j's at
> >> > > > > > > > > > > > > > > >> > > > this
> >> > > > > > > > > > > > > > > >> > > > > > > > > point).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > 2.
> >> > > > > Regarding
> >> > > > > > > > > topic
> >> > > > > > > > > > > > ids,
> >> > > > > > > > > > > > > > any
> >> > > > > > > > > > > > > > > >> major new
> >> > > > > > > > > > > > > > > >> > > > > > protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > integrate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > it
> >> > > and
> >> > > > > > > should
> >> > > > > > > > > > handle
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > topic
> >> > > > > > > > > > > > > > > >> > > > > > recreation case
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > correctly.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That's
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > main
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > part we
> >> > > > > > > need to
> >> > > > > > > > > > > > handle.
> >> > > > > > > > > > > > > I
> >> > > > > > > > > > > > > > > >> agree with
> >> > > > > > > > > > > > > > > >> > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > that we'd
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > want to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > add
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > to
> >> > > the
> >> > > > > > > relevant
> >> > > > > > > > > > > > > protocols
> >> > > > > > > > > > > > > > > >> that don't
> >> > > > > > > > > > > > > > > >> > > > > > have it
> >> > > > > > > > > > > > > > > >> > > > > > > > > yet and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > that we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > can
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> probably
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > focus
> >> > > > > on the
> >> > > > > > > > > > internals
> >> > > > > > > > > > > > > > versus
> >> > > > > > > > > > > > > > > >> adding
> >> > > > > > > > > > > > > > > >> > > > new
> >> > > > > > > > > > > > > > > >> > > > > > APIs
> >> > > > > > > > > > > > > > > >> > > > > > > > > to the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > (unless
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > we
> >> > > find
> >> > > > > that
> >> > > > > > > > > > adding
> >> > > > > > > > > > > > new
> >> > > > > > > > > > > > > > APIs
> >> > > > > > > > > > > > > > > >> is
> >> > > > > > > > > > > > > > > >> > > > required
> >> > > > > > > > > > > > > > > >> > > > > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> reasonable
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> semantics).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > 3.
> >> > > I am
> >> > > > > > > still
> >> > > > > > > > > not
> >> > > > > > > > > > sure
> >> > > > > > > > > > > > > > about
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > storing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> configs.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It's
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > powerful for
> >> > > > > > > > > > configs
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> > > > centralized
> >> > > > > > > > > > > > > > > >> > > > > > in the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> metadata
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> various
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > reasons
> >> > > > > > > > > > (auditability,
> >> > > > > > > > > > > > > > > >> visibility,
> >> > > > > > > > > > > > > > > >> > > > > > consistency,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> etc.).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> Similarly, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > am
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > sure
> >> > > > > about
> >> > > > > > > > > > > > automatically
> >> > > > > > > > > > > > > > > >> deleting
> >> > > > > > > > > > > > > > > >> > > > > > configs in a
> >> > > > > > > > > > > > > > > >> > > > > > > > > way
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cannot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > recovered. A
> >> > > > > > > > > good
> >> > > > > > > > > > > > > property
> >> > > > > > > > > > > > > > > >> for modern
> >> > > > > > > > > > > > > > > >> > > > > > systems
> >> > > > > > > > > > > > > > > >> > > > > > > > > is to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > minimize
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> number
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > unrecoverable
> >> > > > > > > > > data
> >> > > > > > > > > > > > loss
> >> > > > > > > > > > > > > > > >> scenarios.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > Ismael
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > On
> >> > > Wed,
> >> > > > > Jul
> >> > > > > > > 13,
> >> > > > > > > > > > 2022
> >> > > > > > > > > > > > at
> >> > > > > > > > > > > > > > 3:47
> >> > > > > > > > > > > > > > > >> PM David
> >> > > > > > > > > > > > > > > >> > > > > > Jacot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > > > > <djacot@confluent.io.invalid
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > Thanks
> >> > > > > > > > > > Guozhang. My
> >> > > > > > > > > > > > > > answers
> >> > > > > > > > > > > > > > > >> are
> >> > > > > > > > > > > > > > > >> > > > below:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > 1)
> >> > > > > the
> >> > > > > > > > > > migration
> >> > > > > > > > > > > > > path,
> >> > > > > > > > > > > > > > > >> especially
> >> > > > > > > > > > > > > > > >> > > > > > the last
> >> > > > > > > > > > > > > > > >> > > > > > > > > step
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > flag
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > to
> >> > > > > > > enable
> >> > > > > > > > > the
> >> > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> protocol, in
> >> > > > > > > > > > > > > > > >> > > > which
> >> > > > > > > > > > > > > > > >> > > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> both
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > protocols /
> >> > > > > > > > > > rpcs
> >> > > > > > > > > > > > and
> >> > > > > > > > > > > > > > old
> >> > > > > > > > > > > > > > > >> > > > protocols /
> >> > > > > > > > > > > > > > > >> > > > > > rpcs
> >> > > > > > > > > > > > > > > >> > > > > > > > > are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> same
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > group.
> >> > > > > > > How
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> coordinator could
> >> > > > > > > > > > > > > > > >> > > > > > "mimic"
> >> > > > > > > > > > > > > > > >> > > > > > > > > the old
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> using
> >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > new
> >> > > > > > > protocol
> >> > > > > > > > > > is
> >> > > > > > > > > > > > > > something
> >> > > > > > > > > > > > > > > >> we
> >> > > > > > > > > > > > > > > >> > > > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > present
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > Noted. I
> >> > > > > > > just
> >> > > > > > > > > > > > > published
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > >> new
> >> > > > > > > > > > > > > > > >> > > > version
> >> > > > > > > > > > > > > > > >> > > > > > of KIP
> >> > > > > > > > > > > > > > > >> > > > > > > > > which
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > includes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > details
> >> > > > > > > about
> >> > > > > > > > > > this.
> >> > > > > > > > > > > > > See
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > "Supporting
> >> > > > > > > > > > > > > > > >> > > > > > > > > Online
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> Upgrade"
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > and
> >> > > > > the
> >> > > > > > > > > > > > > "Compatibility,
> >> > > > > > > > > > > > > > > >> > > > Deprecation,
> >> > > > > > > > > > > > > > > >> > > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > Migration
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Plan". I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > I
> >> > > > > have to
> >> > > > > > > > > think
> >> > > > > > > > > > > > > through
> >> > > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > > >> few cases
> >> > > > > > > > > > > > > > > >> > > > > > now but
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overall idea
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > mechanism
> >> > > > > > > > > > should be
> >> > > > > > > > > > > > > > > >> understandable.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > 2)
> >> > > > > the
> >> > > > > > > usage
> >> > > > > > > > > > of
> >> > > > > > > > > > > > > topic
> >> > > > > > > > > > > > > > > >> ids. So
> >> > > > > > > > > > > > > > > >> > > > far as
> >> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > as
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > part of
> >> > > > > > > RPCs
> >> > > > > > > > > > and
> >> > > > > > > > > > > > > admin
> >> > > > > > > > > > > > > > > >> client,
> >> > > > > > > > > > > > > > > >> > > > but
> >> > > > > > > > > > > > > > > >> > > > > > they
> >> > > > > > > > > > > > > > > >> > > > > > > > > are not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> public
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > APIs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > to
> >> > > > > > > consumers
> >> > > > > > > > > > yet.
> >> > > > > > > > > > > > I
> >> > > > > > > > > > > > > > think
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > question is,
> >> > > > > > > > > > > > > > > >> > > > > > > > > first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > client
> >> > > > > > > to be
> >> > > > > > > > > > > > > > maintaining
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > names
> >> > > > > > > > > > > > > > > >> > > > > > -> ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > mapping
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > leverage
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > on
> >> > > > > all
> >> > > > > > > the
> >> > > > > > > > > > > > augmented
> >> > > > > > > > > > > > > > > >> existing
> >> > > > > > > > > > > > > > > >> > > > RPCs
> >> > > > > > > > > > > > > > > >> > > > > > and the
> >> > > > > > > > > > > > > > > >> > > > > > > > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> ids;
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > secondly,
> >> > > > > > > > > > should
> >> > > > > > > > > > > > we
> >> > > > > > > > > > > > > > ever
> >> > > > > > > > > > > > > > > >> consider
> >> > > > > > > > > > > > > > > >> > > > > > exposing
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > public
> >> > > > > > > APIs
> >> > > > > > > > > as
> >> > > > > > > > > > > > well
> >> > > > > > > > > > > > > > (both
> >> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > rebalance
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > listener for
> >> > > > > > > > > > cases
> >> > > > > > > > > > > > > > like
> >> > > > > > > > > > > > > > > >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > deletion-and-recreation).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > a)
> >> > > > > > > Assuming
> >> > > > > > > > > > that we
> >> > > > > > > > > > > > > > would
> >> > > > > > > > > > > > > > > >> include
> >> > > > > > > > > > > > > > > >> > > > > > converting
> >> > > > > > > > > > > > > > > >> > > > > > > > > all
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > offsets
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> related
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > RPCs
> >> > > > > to
> >> > > > > > > using
> >> > > > > > > > > > topic
> >> > > > > > > > > > > > > ids
> >> > > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > KIP,
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> able
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > fully
> >> > > > > > > operate
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > topic
> >> > > > > > > > > > > > > > > >> ids. That
> >> > > > > > > > > > > > > > > >> > > > > > being
> >> > > > > > > > > > > > > > > >> > > > > > > > > said, it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > still has
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> provide
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > the
> >> > > > > topics
> >> > > > > > > > > > names in
> >> > > > > > > > > > > > > > various
> >> > > > > > > > > > > > > > > >> APIs so
> >> > > > > > > > > > > > > > > >> > > > > > having a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> mapping
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > seems
> >> > > > > > > > > > inevitable to
> >> > > > > > > > > > > > > me.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > b) I
> >> > > > > don't
> >> > > > > > > > > have
> >> > > > > > > > > > a
> >> > > > > > > > > > > > > strong
> >> > > > > > > > > > > > > > > >> opinion on
> >> > > > > > > > > > > > > > > >> > > > > > this.
> >> > > > > > > > > > > > > > > >> > > > > > > > > Here I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wonder if
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > goes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > beyond the
> >> > > > > > > > > > scope of
> >> > > > > > > > > > > > > this
> >> > > > > > > > > > > > > > > >> KIP. I
> >> > > > > > > > > > > > > > > >> > > > would
> >> > > > > > > > > > > > > > > >> > > > > > rather
> >> > > > > > > > > > > > > > > >> > > > > > > > > focus
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> internals
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > here
> >> > > > > and
> >> > > > > > > we
> >> > > > > > > > > can
> >> > > > > > > > > > > > > consider
> >> > > > > > > > > > > > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > > > separately if
> >> > > > > > > > > > > > > > > >> > > > > > > > > we see
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > value in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> doing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > Coming
> >> > > > > > > back to
> >> > > > > > > > > > > > > Ismael's
> >> > > > > > > > > > > > > > > >> point about
> >> > > > > > > > > > > > > > > >> > > > > > using
> >> > > > > > > > > > > > > > > >> > > > > > > > > topic ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> ConsumerGroupHeartbeatRequest, I
> >> > > > > > > > > > > > > > > >> > > > think
> >> > > > > > > > > > > > > > > >> > > > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > there
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> advantage
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > favour of
> >> > > > > > > it.
> >> > > > > > > > > > The
> >> > > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> will
> >> > > > > > > > > > > > > > > >> > > > have
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opportunity
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > the
> >> > > > > topics
> >> > > > > > > > > > exists
> >> > > > > > > > > > > > > before
> >> > > > > > > > > > > > > > > >> passing
> >> > > > > > > > > > > > > > > >> > > > them
> >> > > > > > > > > > > > > > > >> > > > > > into
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > rebalance
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > protocol.
> >> > > > > > > > > > Obviously,
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > will
> >> > > > > > > > > > > > > > > >> > > > > > > > > also
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> notice
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > does
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > not
> >> > > > > really
> >> > > > > > > > > have
> >> > > > > > > > > > a
> >> > > > > > > > > > > > way
> >> > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> reject an
> >> > > > > > > > > > > > > > > >> > > > > > invalid
> >> > > > > > > > > > > > > > > >> > > > > > > > > topic in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> response.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > I'm
> >> > > > > > > agreeing
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > > David
> >> > > > > > > > > > > > > > > >> on all
> >> > > > > > > > > > > > > > > >> > > > other
> >> > > > > > > > > > > > > > > >> > > > > > minor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > `subscribe(Pattern)`
> >> > > > > > > > > > > > > > > >> question:
> >> > > > > > > > > > > > > > > >> > > > > > personally
> >> > > > > > > > > > > > > > > >> > > > > > > > > I think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> necessary
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > deprecate
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > subscribe
> >> > > > > > > > > > > > > > > >> API with
> >> > > > > > > > > > > > > > > >> > > > > > Pattern,
> >> > > > > > > > > > > > > > > >> > > > > > > > > but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > Pattern
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > while
> >> > > > > > > just
> >> > > > > > > > > > > > > documenting
> >> > > > > > > > > > > > > > > >> that our
> >> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
> >> by
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > server.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > Since
> >> > > > > > > the
> >> > > > > > > > > > > > > incompatible
> >> > > > > > > > > > > > > > > >> case is a
> >> > > > > > > > > > > > > > > >> > > > > > very rare
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > overloaded
> >> > > > > > > > > > > > `String`
> >> > > > > > > > > > > > > > based
> >> > > > > > > > > > > > > > > >> > > > > > subscription may
> >> > > > > > > > > > > > > > > >> > > > > > > > > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> vulnerable
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > various
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > invalid
> >> > > > > > > > > > regexes.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > That
> >> > > > > could
> >> > > > > > > > > > work. I
> >> > > > > > > > > > > > > have
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> look at
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > differences
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > between
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > two
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > engines to
> >> > > > > > > > > > better
> >> > > > > > > > > > > > > > > >> understand the
> >> > > > > > > > > > > > > > > >> > > > > > potential
> >> > > > > > > > > > > > > > > >> > > > > > > > > issues.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> My
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > understanding
> >> > > > > is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > that
> >> > > > > would
> >> > > > > > > > > work
> >> > > > > > > > > > for
> >> > > > > > > > > > > > > all
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> basic
> >> > > > > > > > > > > > > > > >> > > > > > regular
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expressions. The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > differences
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > between
> >> > > > > > > the
> >> > > > > > > > > two
> >> > > > > > > > > > are
> >> > > > > > > > > > > > > > mainly
> >> > > > > > > > > > > > > > > >> about
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > various
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> character
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> classes.
> >> I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > wonder
> >> > > > > > > what
> >> > > > > > > > > > other
> >> > > > > > > > > > > > > people
> >> > > > > > > > > > > > > > > >> think
> >> > > > > > > > > > > > > > > >> > > > about
> >> > > > > > > > > > > > > > > >> > > > > > this.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > Best,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > On
> >> > > > > Tue,
> >> > > > > > > Jul
> >> > > > > > > > > 12,
> >> > > > > > > > > > 2022
> >> > > > > > > > > > > > > at
> >> > > > > > > > > > > > > > > >> 11:28 PM
> >> > > > > > > > > > > > > > > >> > > > > > Guozhang
> >> > > > > > > > > > > > > > > >> > > > > > > > > Wang <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> > > > > wangguoz@gmail.com
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > Thanks
> >> > > > > > > > > David!
> >> > > > > > > > > > I
> >> > > > > > > > > > > > > think
> >> > > > > > > > > > > > > > on
> >> > > > > > > > > > > > > > > >> the high
> >> > > > > > > > > > > > > > > >> > > > > > level
> >> > > > > > > > > > > > > > > >> > > > > > > > > there are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > two meta
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> points we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> need
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > to
> >> > > > > > > > > concretize
> >> > > > > > > > > > a
> >> > > > > > > > > > > > bit
> >> > > > > > > > > > > > > > more:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > 1)
> >> > > > > the
> >> > > > > > > > > > migration
> >> > > > > > > > > > > > > path,
> >> > > > > > > > > > > > > > > >> especially
> >> > > > > > > > > > > > > > > >> > > > > > the last
> >> > > > > > > > > > > > > > > >> > > > > > > > > step
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > flag
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > to
> >> > > > > > > enable
> >> > > > > > > > > the
> >> > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> protocol, in
> >> > > > > > > > > > > > > > > >> > > > which
> >> > > > > > > > > > > > > > > >> > > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> both
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > protocols /
> >> > > > > > > > > > rpcs
> >> > > > > > > > > > > > and
> >> > > > > > > > > > > > > > old
> >> > > > > > > > > > > > > > > >> > > > protocols /
> >> > > > > > > > > > > > > > > >> > > > > > rpcs
> >> > > > > > > > > > > > > > > >> > > > > > > > > are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> same
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > group.
> >> > > > > > > How
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> coordinator could
> >> > > > > > > > > > > > > > > >> > > > > > "mimic"
> >> > > > > > > > > > > > > > > >> > > > > > > > > the old
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> using
> >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > new
> >> > > > > > > protocol
> >> > > > > > > > > > is
> >> > > > > > > > > > > > > > something
> >> > > > > > > > > > > > > > > >> we
> >> > > > > > > > > > > > > > > >> > > > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > present
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > 2)
> >> > > > > the
> >> > > > > > > usage
> >> > > > > > > > > > of
> >> > > > > > > > > > > > > topic
> >> > > > > > > > > > > > > > > >> ids. So
> >> > > > > > > > > > > > > > > >> > > > far as
> >> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > as
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > part of
> >> > > > > > > RPCs
> >> > > > > > > > > > and
> >> > > > > > > > > > > > > admin
> >> > > > > > > > > > > > > > > >> client,
> >> > > > > > > > > > > > > > > >> > > > but
> >> > > > > > > > > > > > > > > >> > > > > > they
> >> > > > > > > > > > > > > > > >> > > > > > > > > are not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> public
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > APIs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > to
> >> > > > > > > consumers
> >> > > > > > > > > > yet.
> >> > > > > > > > > > > > I
> >> > > > > > > > > > > > > > think
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > question is,
> >> > > > > > > > > > > > > > > >> > > > > > > > > first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > client
> >> > > > > > > to be
> >> > > > > > > > > > > > > > maintaining
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > names
> >> > > > > > > > > > > > > > > >> > > > > > -> ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > mapping
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > leverage
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > on
> >> > > > > all
> >> > > > > > > the
> >> > > > > > > > > > > > augmented
> >> > > > > > > > > > > > > > > >> existing
> >> > > > > > > > > > > > > > > >> > > > RPCs
> >> > > > > > > > > > > > > > > >> > > > > > and the
> >> > > > > > > > > > > > > > > >> > > > > > > > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> ids;
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > secondly,
> >> > > > > > > > > > should
> >> > > > > > > > > > > > we
> >> > > > > > > > > > > > > > ever
> >> > > > > > > > > > > > > > > >> consider
> >> > > > > > > > > > > > > > > >> > > > > > exposing
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > public
> >> > > > > > > APIs
> >> > > > > > > > > as
> >> > > > > > > > > > > > well
> >> > > > > > > > > > > > > > (both
> >> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > rebalance
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > listener for
> >> > > > > > > > > > cases
> >> > > > > > > > > > > > > > like
> >> > > > > > > > > > > > > > > >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > deletion-and-recreation).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > I'm
> >> > > > > > > agreeing
> >> > > > > > > > > > with
> >> > > > > > > > > > > > > > David
> >> > > > > > > > > > > > > > > >> on all
> >> > > > > > > > > > > > > > > >> > > > other
> >> > > > > > > > > > > > > > > >> > > > > > minor
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > `subscribe(Pattern)`
> >> > > > > > > > > > > > > > > >> question:
> >> > > > > > > > > > > > > > > >> > > > > > personally
> >> > > > > > > > > > > > > > > >> > > > > > > > > I think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> necessary
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > deprecate
> >> > > > > > > > > the
> >> > > > > > > > > > > > > > subscribe
> >> > > > > > > > > > > > > > > >> API with
> >> > > > > > > > > > > > > > > >> > > > > > Pattern,
> >> > > > > > > > > > > > > > > >> > > > > > > > > but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > Pattern
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > while
> >> > > > > > > just
> >> > > > > > > > > > > > > documenting
> >> > > > > > > > > > > > > > > >> that our
> >> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
> >> by
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > server.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > Since
> >> > > > > > > the
> >> > > > > > > > > > > > > incompatible
> >> > > > > > > > > > > > > > > >> case is a
> >> > > > > > > > > > > > > > > >> > > > > > very rare
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > overloaded
> >> > > > > > > > > > > > `String`
> >> > > > > > > > > > > > > > based
> >> > > > > > > > > > > > > > > >> > > > > > subscription may
> >> > > > > > > > > > > > > > > >> > > > > > > > > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> vulnerable
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > various
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > invalid
> >> > > > > > > > > > regexes.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > Guozhang
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > On
> >> > > > > Tue,
> >> > > > > > > Jul
> >> > > > > > > > > > 12,
> >> > > > > > > > > > > > 2022
> >> > > > > > > > > > > > > > at
> >> > > > > > > > > > > > > > > >> 5:23 AM
> >> > > > > > > > > > > > > > > >> > > > > > David Jacot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > > > > > > > > > > > <djacot@confluent.io.invalid
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > Hi
> >> > > > > > > Ismael,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > Thanks for
> >> > > > > > > > > > your
> >> > > > > > > > > > > > > > > >> feedback. Let
> >> > > > > > > > > > > > > > > >> > > > me
> >> > > > > > > > > > > > > > > >> > > > > > answer
> >> > > > > > > > > > > > > > > >> > > > > > > > > your
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > questions
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > inline.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 1. I
> >> > > > > > > > > think
> >> > > > > > > > > > > > it's
> >> > > > > > > > > > > > > > > >> premature to
> >> > > > > > > > > > > > > > > >> > > > > > talk about
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> target
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > versions
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > deprecation
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > removal
> >> > > > > > > > > > of the
> >> > > > > > > > > > > > > > > >> existing group
> >> > > > > > > > > > > > > > > >> > > > > > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Unlike
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > KRaft,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > affects a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > core
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > client
> >> > > > > > > > > > > > protocol
> >> > > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> hence
> >> > > > > > > > > > > > > > > >> > > > > > > > > deprecation/removal
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > heavily
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > dependent
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > how
> >> > > > > > > > > > quickly
> >> > > > > > > > > > > > > > > >> applications
> >> > > > > > > > > > > > > > > >> > > > migrate
> >> > > > > > > > > > > > > > > >> > > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > the new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > That
> >> > > > > > > makes
> >> > > > > > > > > > > > sense.
> >> > > > > > > > > > > > > I
> >> > > > > > > > > > > > > > will
> >> > > > > > > > > > > > > > > >> > > > remove it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 2.
> >> > > > > > > The
> >> > > > > > > > > KIP
> >> > > > > > > > > > > > says
> >> > > > > > > > > > > > > we
> >> > > > > > > > > > > > > > > >> intend to
> >> > > > > > > > > > > > > > > >> > > > > > release
> >> > > > > > > > > > > > > > > >> > > > > > > > > this in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 4.x, but
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> wasn't
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> made
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > clear
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > why. If
> >> > > > > > > > > we
> >> > > > > > > > > > > > added
> >> > > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > > >> as a
> >> > > > > > > > > > > > > > > >> > > > way to
> >> > > > > > > > > > > > > > > >> > > > > > > > > estimate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we'd
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> deprecate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > remove
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > the
> >> > > > > > > > > group
> >> > > > > > > > > > > > > > protocol, I
> >> > > > > > > > > > > > > > > >> also
> >> > > > > > > > > > > > > > > >> > > > > > suggest
> >> > > > > > > > > > > > > > > >> > > > > > > > > removing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this part.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Let me
> >> > > > > > > > > > explain
> >> > > > > > > > > > > > my
> >> > > > > > > > > > > > > > > >> reasoning. As
> >> > > > > > > > > > > > > > > >> > > > > > > > > explained, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> plan
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rewrite
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > coordinator
> >> > > > > > > > > > in
> >> > > > > > > > > > > > > Java
> >> > > > > > > > > > > > > > > >> while we
> >> > > > > > > > > > > > > > > >> > > > > > implement
> >> > > > > > > > > > > > > > > >> > > > > > > > > the new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> This
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > that
> >> > > > > > > the
> >> > > > > > > > > > > > internals
> >> > > > > > > > > > > > > > will
> >> > > > > > > > > > > > > > > >> be
> >> > > > > > > > > > > > > > > >> > > > slightly
> >> > > > > > > > > > > > > > > >> > > > > > > > > different
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> threading
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > model).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > Therefore, I
> >> > > > > > > > > > > > > wanted
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> tighten
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > switch from
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the old
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > coordinator
> >> > > > > > > > > > to
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > to a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > major
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> release.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > alternative
> >> > > > > > > > > > > > would
> >> > > > > > > > > > > > > > be to
> >> > > > > > > > > > > > > > > >> use a
> >> > > > > > > > > > > > > > > >> > > > flag
> >> > > > > > > > > > > > > > > >> > > > > > to do
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instead
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > relying
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > on the
> >> > > > > > > > > > software
> >> > > > > > > > > > > > > > upgrade.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 3.
> >> > > > > > > We
> >> > > > > > > > > > need to
> >> > > > > > > > > > > > > > flesh
> >> > > > > > > > > > > > > > > >> out the
> >> > > > > > > > > > > > > > > >> > > > > > details of
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > migration
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> story.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> sounds
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > like
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > we're
> >> > > > > > > > > > saying
> >> > > > > > > > > > > > we
> >> > > > > > > > > > > > > > will
> >> > > > > > > > > > > > > > > >> support
> >> > > > > > > > > > > > > > > >> > > > > > online
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> migrations.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> correct?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> We
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > explain
> >> > > > > > > > > > this
> >> > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> detail. It
> >> > > > > > > > > > > > > > > >> > > > could
> >> > > > > > > > > > > > > > > >> > > > > > also
> >> > > > > > > > > > > > > > > >> > > > > > > > > be done
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> separate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > KIP,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> if
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > it's
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > easier.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Yes,
> >> > > > > > > we
> >> > > > > > > > > will
> >> > > > > > > > > > > > > support
> >> > > > > > > > > > > > > > > >> online
> >> > > > > > > > > > > > > > > >> > > > > > migrations
> >> > > > > > > > > > > > > > > >> > > > > > > > > for the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > a
> >> > > > > > > group
> >> > > > > > > > > > using
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > old
> >> > > > > > > > > > > > > > > >> protocol
> >> > > > > > > > > > > > > > > >> > > > > > will be
> >> > > > > > > > > > > > > > > >> > > > > > > > > able to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Let me
> >> > > > > > > > > > briefly
> >> > > > > > > > > > > > > > explain
> >> > > > > > > > > > > > > > > >> how that
> >> > > > > > > > > > > > > > > >> > > > > > will work
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> though.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > It is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> basically
> >> > > a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > four
> >> > > > > > > step
> >> > > > > > > > > > > > process:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > 1. The
> >> > > > > > > > > > cluster
> >> > > > > > > > > > > > > must
> >> > > > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> > > > upgraded or
> >> > > > > > > > > > > > > > > >> > > > > > > > > rolled to a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > software
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> supporting
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > new
> >> > > > > > > group
> >> > > > > > > > > > > > > > coordinator.
> >> > > > > > > > > > > > > > > >> Both the
> >> > > > > > > > > > > > > > > >> > > > > > old and
> >> > > > > > > > > > > > > > > >> > > > > > > > > the new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > support
> >> > > > > > > > > the
> >> > > > > > > > > > old
> >> > > > > > > > > > > > > > > >> protocol and
> >> > > > > > > > > > > > > > > >> > > > rely
> >> > > > > > > > > > > > > > > >> > > > > > on the
> >> > > > > > > > > > > > > > > >> > > > > > > > > same
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > persisted
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> metadata
> >> > > so
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > they
> >> > > > > > > can
> >> > > > > > > > > > work
> >> > > > > > > > > > > > > > together.
> >> > > > > > > > > > > > > > > >> This
> >> > > > > > > > > > > > > > > >> > > > point
> >> > > > > > > > > > > > > > > >> > > > > > is an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> offline
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> migration.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > We
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> cannot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > do
> >> > > > > > > this
> >> > > > > > > > > one
> >> > > > > > > > > > live
> >> > > > > > > > > > > > > > > >> because it
> >> > > > > > > > > > > > > > > >> > > > would
> >> > > > > > > > > > > > > > > >> > > > > > require
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > shutting down
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> current
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > coordinator
> >> > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> starting up the
> >> > > > > > > > > > > > > > > >> > > > > > new one
> >> > > > > > > > > > > > > > > >> > > > > > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cause
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > unavailabilities.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > 2. The
> >> > > > > > > > > > cluster's
> >> > > > > > > > > > > > > > > >> metadata
> >> > > > > > > > > > > > > > > >> > > > > > version/IBP
> >> > > > > > > > > > > > > > > >> > > > > > > > > must be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> X
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> order
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > to
> >> > > > > > > enable
> >> > > > > > > > > > the
> >> > > > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> protocol.
> >> > > > > > > > > > > > > > > >> > > > This
> >> > > > > > > > > > > > > > > >> > > > > > cannot
> >> > > > > > > > > > > > > > > >> > > > > > > > > be done
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 1)
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > terminated
> >> > > > > > > > > > > > because
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> old
> >> > > > > > > > > > > > > > > >> > > > > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > doesn't
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > support
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > protocol.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > 3. The
> >> > > > > > > > > > consumers
> >> > > > > > > > > > > > > > must be
> >> > > > > > > > > > > > > > > >> > > > upgraded
> >> > > > > > > > > > > > > > > >> > > > > > to a
> >> > > > > > > > > > > > > > > >> > > > > > > > > version
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > supporting
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> online
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > migration
> >> > > > > > > > > > (must
> >> > > > > > > > > > > > > have
> >> > > > > > > > > > > > > > > >> KIP-792).
> >> > > > > > > > > > > > > > > >> > > > If
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > already
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> there.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > Nothing
> >> > > > > > > > > > must be
> >> > > > > > > > > > > > > > done at
> >> > > > > > > > > > > > > > > >> this
> >> > > > > > > > > > > > > > > >> > > > point.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > 4. The
> >> > > > > > > > > > consumers
> >> > > > > > > > > > > > > > must
> >> > > > > > > > > > > > > > > >> be rolled
> >> > > > > > > > > > > > > > > >> > > > > > with the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> turned
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > on.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > consumer
> >> > > > > > > > > > group
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > > > >> automatically
> >> > > > > > > > > > > > > > > >> > > > > > > > > converted when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > using
> >> > > > > > > the
> >> > > > > > > > > > new
> >> > > > > > > > > > > > > > protocol
> >> > > > > > > > > > > > > > > >> joins
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > group.
> >> > > > > > > > > > > > > > > >> > > > > > > > > While
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > members
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> using the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > old
> >> > > > > > > > > > protocol are
> >> > > > > > > > > > > > > > being
> >> > > > > > > > > > > > > > > >> > > > upgraded,
> >> > > > > > > > > > > > > > > >> > > > > > the old
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> proxied
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > into
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > new
> >> > > > > > > one.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Let me
> >> > > > > > > > > > clarify
> >> > > > > > > > > > > > all
> >> > > > > > > > > > > > > > of
> >> > > > > > > > > > > > > > > >> this in
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > KIP.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 4.
> >> > > > > > > I am
> >> > > > > > > > > > happy
> >> > > > > > > > > > > > > > that we
> >> > > > > > > > > > > > > > > >> are
> >> > > > > > > > > > > > > > > >> > > > > > pushing the
> >> > > > > > > > > > > > > > > >> > > > > > > > > pattern
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> subscriptions
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > server,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > but
> >> > > > > > > it
> >> > > > > > > > > > seems
> >> > > > > > > > > > > > > like
> >> > > > > > > > > > > > > > > >> there
> >> > > > > > > > > > > > > > > >> > > > could be
> >> > > > > > > > > > > > > > > >> > > > > > some
> >> > > > > > > > > > > > > > > >> > > > > > > > > tricky
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> compatibility
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > issues.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > Will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > have a
> >> > > > > > > > > > > > mechanism
> >> > > > > > > > > > > > > > for
> >> > > > > > > > > > > > > > > >> users to
> >> > > > > > > > > > > > > > > >> > > > > > detect
> >> > > > > > > > > > > > > > > >> > > > > > > > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > update
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> their
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > regex
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > before
> >> > > > > > > > > > > > switching
> >> > > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> the new
> >> > > > > > > > > > > > > > > >> > > > > > protocol?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > I
> >> > > > > > > think
> >> > > > > > > > > > that I
> >> > > > > > > > > > > > am
> >> > > > > > > > > > > > > a
> >> > > > > > > > > > > > > > bit
> >> > > > > > > > > > > > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > > > optimistic
> >> > > > > > > > > > > > > > > >> > > > > > > > > than
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> you
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> point. I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > believe
> >> > > > > > > > > > that the
> >> > > > > > > > > > > > > > > >> majority of
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > cases
> >> > > > > > > > > > > > > > > >> > > > > > > > > are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> simple
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > regexes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > which
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > work
> >> > > > > > > with
> >> > > > > > > > > > the
> >> > > > > > > > > > > > new
> >> > > > > > > > > > > > > > > >> engine. The
> >> > > > > > > > > > > > > > > >> > > > > > > > > coordinator will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > verify
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > regex
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> anyway
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > and
> >> > > > > > > reject
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> if the
> >> > > > > > > > > > > > > > > >> > > > > > regex is
> >> > > > > > > > > > > > > > > >> > > > > > > > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> valid.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Coming
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > back
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > migration
> >> > > > > > > > > > path,
> >> > > > > > > > > > > > in
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> worst
> >> > > > > > > > > > > > > > > >> > > > case,
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > joining
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > the
> >> > > > > > > group
> >> > > > > > > > > > will
> >> > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> rejected.
> >> > > > > > > > > > > > > > > >> > > > This
> >> > > > > > > > > > > > > > > >> > > > > > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > be used
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> last
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > defence, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > would
> >> > > > > > > say.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > One
> >> > > > > > > way
> >> > > > > > > > > for
> >> > > > > > > > > > > > > > customers to
> >> > > > > > > > > > > > > > > >> > > > validate
> >> > > > > > > > > > > > > > > >> > > > > > their
> >> > > > > > > > > > > > > > > >> > > > > > > > > regex
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> upgrading
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> their
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > prod
> >> > > > > > > would
> >> > > > > > > > > > be to
> >> > > > > > > > > > > > > > test
> >> > > > > > > > > > > > > > > >> them with
> >> > > > > > > > > > > > > > > >> > > > > > another
> >> > > > > > > > > > > > > > > >> > > > > > > > > group.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> For
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instance,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > could
> >> > > > > > > be
> >> > > > > > > > > > done
> >> > > > > > > > > > > > in a
> >> > > > > > > > > > > > > > > >> pre-prod
> >> > > > > > > > > > > > > > > >> > > > > > environment.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Another
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > way
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> would be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > extend the
> >> > > > > > > > > > > > > > > >> consumer-group tool
> >> > > > > > > > > > > > > > > >> > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > provide a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> regex
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> >> validation
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > mechanism.
> >> > > > > > > > > > Would
> >> > > > > > > > > > > > > > this be
> >> > > > > > > > > > > > > > > >> > > > enough in
> >> > > > > > > > > > > > > > > >> > > > > > your
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opinion?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 5.
> >> > > > > > > > > > Related to
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > last
> >> > > > > > > > > > > > > > > >> > > > question,
> >> > > > > > > > > > > > > > > >> > > > > > will
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Java
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> allow
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > users
> >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > stick
> >> > > > > > > > > > with the
> >> > > > > > > > > > > > > > > >> current regex
> >> > > > > > > > > > > > > > > >> > > > > > engine for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > compatibility
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> reasons?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> For
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > example,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > it
> >> > > > > > > may
> >> > > > > > > > > be
> >> > > > > > > > > > > > handy
> >> > > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> keep using
> >> > > > > > > > > > > > > > > >> > > > > > client
> >> > > > > > > > > > > > > > > >> > > > > > > > > based
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > regex at
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > keep
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > migrations
> >> > > > > > > > > > > > > simple
> >> > > > > > > > > > > > > > and
> >> > > > > > > > > > > > > > > >> then
> >> > > > > > > > > > > > > > > >> > > > > > migrate to
> >> > > > > > > > > > > > > > > >> > > > > > > > > server
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > based
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regexes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > as a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > second
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > step.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > I
> >> > > > > > > > > understand
> >> > > > > > > > > > > > your
> >> > > > > > > > > > > > > > point
> >> > > > > > > > > > > > > > > >> but I
> >> > > > > > > > > > > > > > > >> > > > am
> >> > > > > > > > > > > > > > > >> > > > > > > > > concerned that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > allow
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> users
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > to
> >> > > > > > > > > actually
> >> > > > > > > > > > stay
> >> > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> this mode.
> >> > > > > > > > > > > > > > > >> > > > That
> >> > > > > > > > > > > > > > > >> > > > > > > > > would go
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > against our
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> goal
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > simplifying
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > client
> >> > > > > > > > > > > > > > > >> because
> >> > > > > > > > > > > > > > > >> > > > we
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > have to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > continue
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> monitoring
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > the
> >> > > > > > > > > > metadata on
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> client
> >> > > > > > > > > > > > > > > >> > > > side. I
> >> > > > > > > > > > > > > > > >> > > > > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > rather
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not do
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 6.
> >> > > > > > > When
> >> > > > > > > > > > we say
> >> > > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > > >> the group
> >> > > > > > > > > > > > > > > >> > > > > > > > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > responsible
> >> > > > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > storing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > the
> >> > > > > > > > > > > > > configurations
> >> > > > > > > > > > > > > > > >> and that
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> configurations
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> deleted
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > group is
> >> > > > > > > > > > > > > deleted.
> >> > > > > > > > > > > > > > > >> Will a
> >> > > > > > > > > > > > > > > >> > > > > > transition to
> >> > > > > > > > > > > > > > > >> > > > > > > > > DEAD
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > trigger
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> >> deletion
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > configurations?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > That's
> >> > > > > > > > > > right.
> >> > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> > > > configurations
> >> > > > > > > > > > > > > > > >> > > > > > will be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deleted
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> group is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > deleted.
> >> > > > > > > > > > They go
> >> > > > > > > > > > > > > > > >> together.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 7.
> >> > > > > > > Will
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > choice to
> >> > > > > > > > > > > > > > > >> store
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > configs in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > make
> >> > > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > harder
> >> > > > > > > > > to
> >> > > > > > > > > > list
> >> > > > > > > > > > > > > all
> >> > > > > > > > > > > > > > > >> cluster
> >> > > > > > > > > > > > > > > >> > > > > > configs and
> >> > > > > > > > > > > > > > > >> > > > > > > > > their
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > values?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > I
> >> > > > > > > don't
> >> > > > > > > > > > think
> >> > > > > > > > > > > > so.
> >> > > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> group
> >> > > > > > > > > > > > > > > >> > > > > > > > > configurations are
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overrides
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> cluster
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > configs.
> >> > > > > > > > > If
> >> > > > > > > > > > you
> >> > > > > > > > > > > > > > want to
> >> > > > > > > > > > > > > > > >> know
> >> > > > > > > > > > > > > > > >> > > > all
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > overrides
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > you
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > to
> >> > > > > > > ask all
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > group
> >> > > > > > > > > > > > > > > >> > > > coordinators.
> >> > > > > > > > > > > > > > > >> > > > > > You
> >> > > > > > > > > > > > > > > >> > > > > > > > > cannot
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > rely on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> metadata
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> log
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > for
> >> > > > > > > > > > instance.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 8.
> >> > > > > > > How
> >> > > > > > > > > > would
> >> > > > > > > > > > > > > > someone
> >> > > > > > > > > > > > > > > >> > > > configure a
> >> > > > > > > > > > > > > > > >> > > > > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > before
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > starting
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > consumers?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > we
> >> > > > > > > > > > considered
> >> > > > > > > > > > > > > > > >> allowing the
> >> > > > > > > > > > > > > > > >> > > > > > explicit
> >> > > > > > > > > > > > > > > >> > > > > > > > > creation
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > groups?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > > > > > Alternatively,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > configs
> >> > > > > > > > > > could
> >> > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> decoupled
> >> > > > > > > > > > > > > > > >> > > > from
> >> > > > > > > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > lifecycle.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Yes.
> >> > > > > > > The
> >> > > > > > > > > > group
> >> > > > > > > > > > > > > will
> >> > > > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> > > > > > automatically
> >> > > > > > > > > > > > > > > >> > > > > > > > > created in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> However,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > the
> >> > > > > > > > > configs
> >> > > > > > > > > > will
> >> > > > > > > > > > > > > be
> >> > > > > > > > > > > > > > > >> lost after
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > retention
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > period of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > passes.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 9.
> >> > > > > > > Will
> >> > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > Consumer.subscribe
> >> > > > > > > > > > > > > > > >> > > > > > method
> >> > > > > > > > > > > > > > > >> > > > > > > > > for the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> client
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > still
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> take
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` of
> >> > > > > > > > > > > > > > > >> > > > do
> >> > > > > > > > > > > > > > > >> > > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > have to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > introduce
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> overload?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > That's a
> >> > > > > > > > > > very
> >> > > > > > > > > > > > > group
> >> > > > > > > > > > > > > > > >> question. I
> >> > > > > > > > > > > > > > > >> > > > > > forgot
> >> > > > > > > > > > > > > > > >> > > > > > > > > about
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > As
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` is
> >> > > > > > > > > > > > > > > >> > > > not
> >> > > > > > > > > > > > > > > >> > > > > > fully
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> compatible
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> engine
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > we
> >> > > > > > > plan to
> >> > > > > > > > > > use,
> >> > > > > > > > > > > > it
> >> > > > > > > > > > > > > > > >> might be
> >> > > > > > > > > > > > > > > >> > > > better
> >> > > > > > > > > > > > > > > >> > > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > deprecate
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> use an
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > overload
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > which
> >> > > > > > > > > takes
> >> > > > > > > > > > a
> >> > > > > > > > > > > > > > string.
> >> > > > > > > > > > > > > > > >> We would
> >> > > > > > > > > > > > > > > >> > > > > > rely on
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> server
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > validation.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > During the
> >> > > > > > > > > > > > > > migration, I
> >> > > > > > > > > > > > > > > >> think
> >> > > > > > > > > > > > > > > >> > > > that
> >> > > > > > > > > > > > > > > >> > > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > could
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> still
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > try to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> toString
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > regex
> >> > > > > > > and
> >> > > > > > > > > > use
> >> > > > > > > > > > > > it.
> >> > > > > > > > > > > > > > That
> >> > > > > > > > > > > > > > > >> should
> >> > > > > > > > > > > > > > > >> > > > > > work, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > think, in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> majority
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > cases.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 10.
> >> > > > > > > I
> >> > > > > > > > > > agree
> >> > > > > > > > > > > > with
> >> > > > > > > > > > > > > > > >> Justine
> >> > > > > > > > > > > > > > > >> > > > that we
> >> > > > > > > > > > > > > > > >> > > > > > > > > should be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > clearer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > about
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> reason
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > switch
> >> > > > > > > > > to
> >> > > > > > > > > > > > > > > >> > > > IBP/metadata.version
> >> > > > > > > > > > > > > > > >> > > > > > from the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Maybe
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> mean
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > can
> >> > > > > > > > > > switch the
> >> > > > > > > > > > > > > > > >> default for
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > feature
> >> > > > > > > > > > > > > > > >> > > > > > > > > flag
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > true
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> based
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > metadata.version
> >> > > > > > > > > > > > > > once
> >> > > > > > > > > > > > > > > >> we
> >> > > > > > > > > > > > > > > >> > > > want to
> >> > > > > > > > > > > > > > > >> > > > > > make
> >> > > > > > > > > > > > > > > >> > > > > > > > > it the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > default.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > My
> >> > > > > > > plan
> >> > > > > > > > > was
> >> > > > > > > > > > to
> >> > > > > > > > > > > > use
> >> > > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > > >> feature
> >> > > > > > > > > > > > > > > >> > > > > > flag
> >> > > > > > > > > > > > > > > >> > > > > > > > > mainly
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > during the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > development
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > phase. I
> >> > > > > > > > > > should
> >> > > > > > > > > > > > > not
> >> > > > > > > > > > > > > > have
> >> > > > > > > > > > > > > > > >> > > > mentioned
> >> > > > > > > > > > > > > > > >> > > > > > it, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > think,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > because
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > could
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> use
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > an
> >> > > > > > > > > internal
> >> > > > > > > > > > > > config
> >> > > > > > > > > > > > > > for
> >> > > > > > > > > > > > > > > >> it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 11.
> >> > > > > > > Some
> >> > > > > > > > > > of
> >> > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> protocol APIs
> >> > > > > > > > > > > > > > > >> > > > > > don't
> >> > > > > > > > > > > > > > > >> > > > > > > > > mention
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > required
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> ACLs, it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > would
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > good to
> >> > > > > > > > > > add
> >> > > > > > > > > > > > that
> >> > > > > > > > > > > > > > for
> >> > > > > > > > > > > > > > > >> > > > consistency.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Noted.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 12.
> >> > > > > > > It
> >> > > > > > > > > is
> >> > > > > > > > > > a
> >> > > > > > > > > > > > bit
> >> > > > > > > > > > > > > > odd
> >> > > > > > > > > > > > > > > >> that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> ConsumerGroupHeartbeat
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > requires
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > "Read
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> Group"
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > even
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > though
> >> > > > > > > > > it
> >> > > > > > > > > > > > seems
> >> > > > > > > > > > > > > > to do
> >> > > > > > > > > > > > > > > >> more
> >> > > > > > > > > > > > > > > >> > > > than
> >> > > > > > > > > > > > > > > >> > > > > > > > > reading.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > I
> >> > > > > > > agree.
> >> > > > > > > > > > This is
> >> > > > > > > > > > > > > > how the
> >> > > > > > > > > > > > > > > >> > > > current
> >> > > > > > > > > > > > > > > >> > > > > > > > > protocol works
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > We
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > only
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > require
> >> > > > > > > > > > "Read
> >> > > > > > > > > > > > > > Group" to
> >> > > > > > > > > > > > > > > >> join a
> >> > > > > > > > > > > > > > > >> > > > > > group. We
> >> > > > > > > > > > > > > > > >> > > > > > > > > could
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consider
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> changing
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> this
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > but I
> >> > > > > > > am
> >> > > > > > > > > not
> >> > > > > > > > > > > > sure
> >> > > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > > >> it is
> >> > > > > > > > > > > > > > > >> > > > worth
> >> > > > > > > > > > > > > > > >> > > > > > it.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 13.
> >> > > > > > > How
> >> > > > > > > > > is
> >> > > > > > > > > > > > topic
> >> > > > > > > > > > > > > > > >> recreation
> >> > > > > > > > > > > > > > > >> > > > > > handled by
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consumer
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > group
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > protocol?
> >> > > > > > > > > > It
> >> > > > > > > > > > > > > > would be
> >> > > > > > > > > > > > > > > >> good to
> >> > > > > > > > > > > > > > > >> > > > > > have a
> >> > > > > > > > > > > > > > > >> > > > > > > > > section
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Noted.
> >> > > > > > > > > From
> >> > > > > > > > > > a
> >> > > > > > > > > > > > > > protocol
> >> > > > > > > > > > > > > > > >> > > > > > perspective, the
> >> > > > > > > > > > > > > > > >> > > > > > > > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> have a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > topic
> >> > > > > > > id
> >> > > > > > > > > so
> >> > > > > > > > > > it
> >> > > > > > > > > > > > > will
> >> > > > > > > > > > > > > > > >> treat it
> >> > > > > > > > > > > > > > > >> > > > like a
> >> > > > > > > > > > > > > > > >> > > > > > > > > topic with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > different
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > name.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > only
> >> > > > > > > issue
> >> > > > > > > > > > is
> >> > > > > > > > > > > > that
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > fetch/commit
> >> > > > > > > > > > > > > > > >> > > > > > > > > offsets
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> APIs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > do not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > support
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > IDs
> >> > > > > > > so the
> >> > > > > > > > > > > > > consumer
> >> > > > > > > > > > > > > > > >> would
> >> > > > > > > > > > > > > > > >> > > > reuse the
> >> > > > > > > > > > > > > > > >> > > > > > > > > offsets
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> based
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > same.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > that
> >> > > > > > > we
> >> > > > > > > > > > should
> >> > > > > > > > > > > > > > update
> >> > > > > > > > > > > > > > > >> those
> >> > > > > > > > > > > > > > > >> > > > APIs
> >> > > > > > > > > > > > > > > >> > > > > > as well
> >> > > > > > > > > > > > > > > >> > > > > > > > > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> order
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> consistent
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> end
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > to
> >> > > > > > > end.
> >> > > > > > > > > That
> >> > > > > > > > > > > > would
> >> > > > > > > > > > > > > > > >> strengthen
> >> > > > > > > > > > > > > > > >> > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > semantics of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consumer.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 14.
> >> > > > > > > The
> >> > > > > > > > > > KIP
> >> > > > > > > > > > > > > > mentions
> >> > > > > > > > > > > > > > > >> we will
> >> > > > > > > > > > > > > > > >> > > > > > write the
> >> > > > > > > > > > > > > > > >> > > > > > > > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > coordinator
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Java.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> Even
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > though
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > this is
> >> > > > > > > > > an
> >> > > > > > > > > > > > > > > >> implementation
> >> > > > > > > > > > > > > > > >> > > > > > detail, do
> >> > > > > > > > > > > > > > > >> > > > > > > > > we plan
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > have a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> gradle
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > module
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > for
> >> > > > > > > it?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Yes.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 15.
> >> > > > > > > Do
> >> > > > > > > > > we
> >> > > > > > > > > > > > have a
> >> > > > > > > > > > > > > > > >> scalability
> >> > > > > > > > > > > > > > > >> > > > > > goal when
> >> > > > > > > > > > > > > > > >> > > > > > > > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> comes
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to how
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> many
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > members
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > new
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > group
> >> > > > > > > > > > protocol
> >> > > > > > > > > > > > > can
> >> > > > > > > > > > > > > > > >> support?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > We
> >> > > > > > > don't
> >> > > > > > > > > > have
> >> > > > > > > > > > > > > > numbers
> >> > > > > > > > > > > > > > > >> at the
> >> > > > > > > > > > > > > > > >> > > > > > moment. The
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> support
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> 1000s
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > of
> >> > > > > > > members
> >> > > > > > > > > > per
> >> > > > > > > > > > > > > > group.
> >> > > > > > > > > > > > > > > >> We will
> >> > > > > > > > > > > > > > > >> > > > > > measure
> >> > > > > > > > > > > > > > > >> > > > > > > > > this when
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > first
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > implementation.
> >> > > > > > > > > > > > > Note
> >> > > > > > > > > > > > > > > >> that we
> >> > > > > > > > > > > > > > > >> > > > might
> >> > > > > > > > > > > > > > > >> > > > > > have
> >> > > > > > > > > > > > > > > >> > > > > > > > > other
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > bottlenecks
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > down
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > road
> >> > > > > > > (e.g.
> >> > > > > > > > > > > > offset
> >> > > > > > > > > > > > > > > >> commits).
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > 16.
> >> > > > > > > Did
> >> > > > > > > > > we
> >> > > > > > > > > > > > > > consider
> >> > > > > > > > > > > > > > > >> having
> >> > > > > > > > > > > > > > > >> > > > > > > > > SubscribedTopidIds
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > > > SubscribedTopicNames
> >> > > > > > > > > > > > > > > >> in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> >> > > > > ConsumerGroupHeartbeatRequest?
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> idea
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > since we
> >> > > > > > > > > > have
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > > > >> resolve the
> >> > > > > > > > > > > > > > > >> > > > > > regex on
> >> > > > > > > > > > > > > > > >> > > > > > > > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > server, we
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> can do
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> same
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > the
> >> > > > > > > > > topic
> >> > > > > > > > > > > > name?
> >> > > > > > > > > > > > > > The
> >> > > > > > > > > > > > > > > >> > > > difference
> >> > > > > > > > > > > > > > > >> > > > > > is that
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> sending
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regex is
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > more
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > efficient
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > whereas
> >> > > > > > > > > > > > sending
> >> > > > > > > > > > > > > > the
> >> > > > > > > > > > > > > > > >> topic
> >> > > > > > > > > > > > > > > >> > > > names
> >> > > > > > > > > > > > > > > >> > > > > > is less
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > efficient.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > Furthermore,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > delete
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > recreation is
> >> > > > > > > > > > > > > > easier
> >> > > > > > > > > > > > > > > >> to
> >> > > > > > > > > > > > > > > >> > > > handle
> >> > > > > > > > > > > > > > > >> > > > > > if we
> >> > > > > > > > > > > > > > > >> > > > > > > > > have
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ids.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > The
> >> > > > > > > idea
> >> > > > > > > > > > was to
> >> > > > > > > > > > > > > > > >> consolidate the
> >> > > > > > > > > > > > > > > >> > > > > > metadata
> >> > > > > > > > > > > > > > > >> > > > > > > > > lookup
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> server
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > for
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> both
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > paths
> >> > > > > > > but
> >> > > > > > > > > I
> >> > > > > > > > > > do
> >> > > > > > > > > > > > > agree
> >> > > > > > > > > > > > > > > >> with your
> >> > > > > > > > > > > > > > > >> > > > > > point. As
> >> > > > > > > > > > > > > > > >> > > > > > > > > a
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> second
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > though,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > ids
> >> > > > > > > may be
> >> > > > > > > > > > > > better
> >> > > > > > > > > > > > > > here
> >> > > > > > > > > > > > > > > >> for the
> >> > > > > > > > > > > > > > > >> > > > > > delete and
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > recreation
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> Also, I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > suppose
> >> > > > > > > > > > that we
> >> > > > > > > > > > > > > may
> >> > > > > > > > > > > > > > > >> allow
> >> > > > > > > > > > > > > > > >> > > > users to
> >> > > > > > > > > > > > > > > >> > > > > > > > > subscribe
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> ids
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > future
> >> > > > > > > > > > because
> >> > > > > > > > > > > > > that
> >> > > > > > > > > > > > > > is
> >> > > > > > > > > > > > > > > >> the only
> >> > > > > > > > > > > > > > > >> > > > > > way to be
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> really
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > robust
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > re-creation.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > Best,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > On
> >> > > > > > > Tue,
> >> > > > > > > > > Jul
> >> > > > > > > > > > 12,
> >> > > > > > > > > > > > > > 2022 at
> >> > > > > > > > > > > > > > > >> 1:38 PM
> >> > > > > > > > > > > > > > > >> > > > > > David
> >> > > > > > > > > > > > > > > >> > > > > > > > > Jacot <
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> > > > > > > djacot@confluent.io>
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > >
> >> > > wrote:
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > Hi
> >> > > > > > > > > > Justine,
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > Thanks
> >> > > > > > > > > for
> >> > > > > > > > > > > > your
> >> > > > > > > > > > > > > > > >> comments.
> >> > > > > > > > > > > > > > > >> > > > Please
> >> > > > > > > > > > > > > > > >> > > > > > find
> >> > > > > > > > > > > > > > > >> > > > > > > > > my
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> answers
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > below.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > -
> >> > > > > > > Yes,
> >> > > > > > > > > > the new
> >> > > > > > > > > > > > > > > >> protocol
> >> > > > > > > > > > > > > > > >> > > > relies on
> >> > > > > > > > > > > > > > > >> > > > > > > > > topic IDs
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> >> exception
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> of the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > > > topic
> >> > > > > > > > > > names
> >> > > > > > > > > > > > > based
> >> > > > > > > > > > > > > > in
> >> > > > > > > > > > > > > > > >> the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> >> > > > > > > > > ConsumerGroupHeartbeatRequest.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > am
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> sure
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > if
> >> > > > > > > using
> >> > > > > > > > > > topic
> >> > > > > > > > > > > > > > names
> >> > > > > > > > > > > > > > > >> is the
> >> > > > > > > > > > > > > > > >> > > > > > right call
> >> > > > > > > > > > > > > > > >> > > > > > > > > here.
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> I
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> about
> >> > > it
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> >> > > >
> >> > > > >
> >> > > > > a
> >> > > > > > > little
> >> > > > > > > > > > more.
> >> > > > > > > > > > > > > > > >> Obviously, the
> >> > > > > > > > > > > > > > > >> > > > > > KIP does
> >> > > > > > > > > > > > > > > >> > > > > > > > > not
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change the
> >> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > -- Guozhang
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > -- Guozhang
> >> > >
> >> >
> >> >
> >> > --
> >> > -- Guozhang
> >>
> >
> >
> > --
> > -- Guozhang
> >
>
>
> --
> -- Guozhang
Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol
Posted by Guozhang Wang <wa...@gmail.com>.
Hi David,
I made another pass on the doc, and assuming we will change the
"TopicIdPartition" fields to "TopicPartition" for the consumer-side APIs, I
think this proposal looks great for me.
Guozhang
On Mon, Oct 3, 2022 at 9:36 AM Guozhang Wang <wa...@gmail.com> wrote:
> 3. Ah thanks, that's my mis-read then. I think keeping the current class
> and the params of `onAssignment` is fine then.
>
> On Mon, Oct 3, 2022 at 4:32 AM David Jacot <dj...@confluent.io.invalid>
> wrote:
>
>> 3. ConsumerGroupMetadata is already defined in the consumer [1]. I
>> realized that we pass it in the current assignor so I thought that we
>> could reuse it. The alternative would be to add the same information
>> in a class which includes the error and the member assignment as well.
>>
>> [1]
>> https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerGroupMetadata.java
>>
>> Best,
>> David
>>
>> On Mon, Oct 3, 2022 at 6:15 AM Guozhang Wang <wa...@gmail.com> wrote:
>> >
>> > 1. Yeah that makes sense. I originally concerned if there's any
>> consistency
>> > issues if the HB request with encoded topic IDs and the GPA/GIA RPCs
>> with
>> > encoded topic IDs use mappings from two different brokers, and one of
>> them
>> > is stale, but after thinking twice about it I think as long as we handle
>> > e.g. `unknown topic ids` from HB responses then we should be fine.
>> >
>> > 3. Ack, in that case I think we also need to define
>> "ConsumerGroupMetadata"
>> > which is passed in `onAssignment`. We have a schema defined in the
>> > persistent storage for different purposes, but not in the consumer APIs
>> > yet. I think that would just include a single "Map<String, Metadata>"?
>> >
>> > On Fri, Sep 30, 2022 at 5:51 AM David Jacot <djacot@confluent.io.invalid
>> >
>> > wrote:
>> >
>> > > Hey Guozhang,
>> > >
>> > > Thanks for your comments.
>> > >
>> > > 1. That's a very good question. I think that it is better to separate
>> > > the two concerns here. In my opinion, the GPA should only be used to
>> > > feed the assignor. I see the GPA as a way to call a remote function so
>> > > the GPA provides all its inputs to ensure that the assignor does not
>> > > depend on any local state. Knowing this, I would not use the GPA to
>> > > update the local state either. That is not its purpose. The consumer
>> > > is responsible to update its local metadata cache either when it gets
>> > > an unknown topic in the user-provided subscription or in the
>> > > coordinator-provided assignment. The consumer has to do this anyway to
>> > > get all topic metadata (e.g. leaders).
>> > >
>> > > 2. Interesting thought. Handling the translation is definitely
>> > > possible. I wonder if there are cases where having the topic ids is
>> > > beneficial. I am thinking about when a topic is deleted and recreated
>> > > right away. In this case, the current member assignment would still
>> > > have the old topic id and we would not be able to resolve it anymore.
>> > > I suppose that we should directly remove it from the current
>> > > assignment on the server side in this case and only add the new one.
>> > > As I first thought, that seems OK. The major downside is that the user
>> > > of the assignor would not know that the topic is different from what
>> > > it may have cached locally.
>> > >
>> > > We could say that we don't expose topic ids externally for now in
>> > > order to stay consistent with the other interfaces but it would be
>> > > annoying to add it back if we want to expose topic ids in the future.
>> > > I personally believe that we will have to expose them at some point.
>> > > For instance, in the rebalance listener, it would be useful to know
>> > > that a topic has been deleted and recreated. The only way to do this
>> > > is to know the topic id. Let me think a little more about this and
>> > > come back to you.
>> > >
>> > > 3. Sorry for this one. I was a typo. PartitionAssingor#assign should
>> > > return GroupAssignment.
>> > >
>> > > Best,
>> > > David
>> > >
>> > > On Thu, Sep 29, 2022 at 6:06 PM Guozhang Wang <wa...@gmail.com>
>> wrote:
>> > > >
>> > > > Thanks David, I read through the updated wiki, and I have a couple
>> > > > follow-up questions:
>> > > >
>> > > > 1. It seems we would have two potential sources to update the
>> consumer's
>> > > > internally maintained topic name -> topic ID mapping, which is used
>> for
>> > > > encoding subscribed topic Ids in the HB request, and offset commit
>> / list
>> > > > offset requests, etc. One is from metadata responses, and another
>> one is
>> > > > from the GPA (group prepare assignment). On the other hand, the
>> assignor
>> > > > APIs directly take parsed AssignmentMemberSpec and
>> > > AssignmentTopicMetadata
>> > > > from GPA responses directly instead of trying to consult the
>> consumer's
>> > > own
>> > > > cached mapping. So just to clarify, consumers would also update its
>> > > > internal mapping from the GPA responses as well, right?
>> > > >
>> > > > 2. I'm wondering if it's necessary to expose the topic IDs to the
>> > > assignor
>> > > > developers. What they need to do at the moment is to take the
>> mapping
>> > > from
>> > > > AssignmentTopicMetadata, and translate their decided assignment into
>> > > topic
>> > > > IDs when returning the MemberAssignment. We can instead just let
>> consumer
>> > > > to do such translation for them internally and still let the
>> assignor
>> > > > developers to only deal with the topic names. Of course that means
>> the
>> > > > client side assignor APIs would be a bit further diverged from the
>> > > > broker-side assignor such that:
>> > > >
>> > > > a) Topic metadata would be encoded just as String -> num.partitions;
>> > > > b) AssignmentMemberSpec contains subscribedTopicNames and
>> topicPartitions
>> > > > as target assignments;
>> > > > c) Returned MemberAssignment contains list of topicPartitions
>> instead of
>> > > > topicIdPartitions.
>> > > >
>> > > > Internally, consumers would convert the IDs to names when calling
>> the
>> > > > assign() function, and convert them back when sending the request;
>> and
>> > > > similarly it would convert the IDs to names when calling
>> onAssignment()
>> > > as
>> > > > well.
>> > > >
>> > > > In short, I'm wondering if we should bite the bullet for the users,
>> > > eating
>> > > > more complexities internally than pushing that to users.
>> > > >
>> > > > 3. This is a nit one: the return type of `Assignment
>> > > assign(AssignmentSpec
>> > > > assignmentSpec);` seems not defined in the doc. I think it would
>> just
>> > > > include two fields, as:
>> > > >
>> > > > class Assignment {
>> > > > GroupAssignment groupAssignment;
>> > > >
>> > > > ConsumerGroupMetadata consumerGroupMetadata;
>> > > > }
>> > > >
>> > > > Is that right?
>> > > >
>> > > > Guozhang
>> > > >
>> > > > On Thu, Sep 29, 2022 at 1:26 AM David Jacot
>> <djacot@confluent.io.invalid
>> > > >
>> > > > wrote:
>> > > >
>> > > > > Hi Guozhang,
>> > > > >
>> > > > > Thanks for your input.
>> > > > >
>> > > > > > 1) in HB request, when clients send the subscription, they are
>> still
>> > > > > > sending them in terms of topic names. That is, the consumer
>> would
>> > > not try
>> > > > > > to keep all the mappings from topic names to topic ids for the
>> > > subscribed
>> > > > > > topics.
>> > > > >
>> > > > > This is a mistake of mine. The heartbeat request should use Topic
>> Ids.
>> > > > > I forgot to change it. The idea is that the client will get the
>> > > > > metadata first to translate the topic names to topic ids and thus
>> also
>> > > > > have the mapping in its local metadata cache.
>> > > > >
>> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
>> > > already.
>> > > > > And
>> > > > > > fetch requests can directly rely on that since they just need
>> topic
>> > > IDs
>> > > > > and
>> > > > > > do not care what actual topic names are.
>> > > > >
>> > > > > That's right.
>> > > > >
>> > > > > > 3) in prepare-assignment responses, we return subscriptions as
>> topic
>> > > IDs
>> > > > > as
>> > > > > > well.
>> > > > >
>> > > > > That's right.
>> > > > >
>> > > > > > In a word, the consumer does not try to maintain the mapping of
>> topic
>> > > > > names
>> > > > > > to IDs at all, and even if they do, they will not expose it in
>> e.g.
>> > > > > > assignor APIs. But for client-side assignors like Streams, if
>> they
>> > > do not
>> > > > > > maintain the mapping from topic IDs to topic names, certain
>> logic
>> > > such as
>> > > > > > co-partitioning indeed cannot be executed.
>> > > > >
>> > > > > As I said, the consumer will have the mapping in its local
>> metadata
>> > > > > cache. It needs to get the partition metadata anyway and the
>> topic id
>> > > > > is part of it. However, I do agree that this mapping is not
>> exposed.
>> > > > >
>> > > > > > So I think we'd need to consider 1) use topic names in
>> subscriptions
>> > > in
>> > > > > the
>> > > > > > prepare-assignment responses, 2) use topic Names in client-side
>> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
>> > > > > > PartitionAssignor#TopicMetadata#topicName.
>> > > > >
>> > > > > 1) makes sense to me. I think that we could actually provide both
>> > > > > names and ids. Regarding 2), I think that I am slightly in favor
>> of
>> > > > > using both names and ids as well. Let me try to update the
>> protocol
>> > > > > and the interface towards this.
>> > > > >
>> > > > > Thanks,
>> > > > > David
>> > > > >
>> > > > > On Wed, Sep 28, 2022 at 9:35 PM Guozhang Wang <wangguoz@gmail.com
>> >
>> > > wrote:
>> > > > > >
>> > > > > > Hello Jun/David,
>> > > > > >
>> > > > > > Just to chime in here for Jun's comment #50, today Streams does
>> not
>> > > have
>> > > > > a
>> > > > > > way to map topic names to topic IDs, and in its topology
>> description
>> > > > > topics
>> > > > > > are encoded with names.
>> > > > > >
>> > > > > > What I originally thought was that, on the consumer side, we'd
>> need
>> > > to
>> > > > > > maintain this mapping from topic names to topic IDs anyways, and
>> > > Streams
>> > > > > > should leverage on the consumer's functionality to get the
>> > > corresponding
>> > > > > > topic names for the given topic IDs. But when I read through
>> the RPC
>> > > > > again
>> > > > > > I realized it may not be the case, since:
>> > > > > >
>> > > > > > 1) in HB request, when clients send the subscription, they are
>> still
>> > > > > > sending them in terms of topic names. That is, the consumer
>> would
>> > > not try
>> > > > > > to keep all the mappings from topic names to topic ids for the
>> > > subscribed
>> > > > > > topics.
>> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
>> > > already.
>> > > > > And
>> > > > > > fetch requests can directly rely on that since they just need
>> topic
>> > > IDs
>> > > > > and
>> > > > > > do not care what actual topic names are.
>> > > > > > 3) in prepare-assignment responses, we return subscriptions as
>> topic
>> > > IDs
>> > > > > as
>> > > > > > well.
>> > > > > >
>> > > > > > In a word, the consumer does not try to maintain the mapping of
>> topic
>> > > > > names
>> > > > > > to IDs at all, and even if they do, they will not expose it in
>> e.g.
>> > > > > > assignor APIs. But for client-side assignors like Streams, if
>> they
>> > > do not
>> > > > > > maintain the mapping from topic IDs to topic names, certain
>> logic
>> > > such as
>> > > > > > co-partitioning indeed cannot be executed.
>> > > > > >
>> > > > > > So I think we'd need to consider 1) use topic names in
>> subscriptions
>> > > in
>> > > > > the
>> > > > > > prepare-assignment responses, 2) use topic Names in client-side
>> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
>> > > > > > PartitionAssignor#TopicMetadata#topicName.
>> > > > > >
>> > > > > > On Wed, Sep 28, 2022 at 1:26 AM David Jacot
>> > > <djacot@confluent.io.invalid
>> > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Jun,
>> > > > > > >
>> > > > > > > Thanks for your comments.
>> > > > > > >
>> > > > > > > 48. Okay. There is a typo in the text:
>> > > > > > > > Note that the persisted target assignment of B already
>> includes
>> > > foo-2
>> > > > > > > here
>> > > > > > > but B does not get it yet because foo-2 is not free.
>> > > > > > >
>> > > > > > > It should be:
>> > > > > > > > Note that the persisted member assignment of B already
>> includes
>> > > foo-2
>> > > > > > > here
>> > > > > > > but B does not get it yet because foo-2 is not free.
>> > > > > > >
>> > > > > > > The examples are showing the persisted state in the
>> > > __consumer_offsets
>> > > > > > > topics. When a member transitions to the next epoch, we
>> directly
>> > > > > > > persist the whole target assignment in order to reduce the
>> number
>> > > of
>> > > > > > > writes and we manage dependencies in memory. So "B - epoch=2,
>> > > > > > > partitions=[foo-2]" is actually correct from a persisted
>> record
>> > > > > > > perspective. However, the in-memory state would be different
>> > > because
>> > > > > > > foo-2 is not yet available as you pointed out. The in-memory
>> state
>> > > > > > > would like as follow:
>> > > > > > >
>> > > > > > > Group (epoch=2)
>> > > > > > > A
>> > > > > > > B
>> > > > > > > Target Assignment (epoch=2)
>> > > > > > > A - partitions=[foo-0, foo-1]
>> > > > > > > B - partitions=[foo-2]
>> > > > > > > Member Assignment
>> > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2], pending
>> > > partitions=[]
>> > > > > > > B - epoch=2, partitions=[], pending partitions=[foo-2]
>> > > > > > >
>> > > > > > > I suppose that the confusion comes from this. I will go and
>> convert
>> > > > > > > all the examples to their in-memory representation in order to
>> > > reduce
>> > > > > > > the confusion.
>> > > > > > >
>> > > > > > > 50. Okay. I have to admit that I am not so familiar with the
>> > > internals
>> > > > > > > of KStreams. If I understand correctly, you're saying that
>> KStreams
>> > > > > > > may not have topic ids in its topology so topic names are
>> missing
>> > > here
>> > > > > > > to match the topics. I was hoping that KStreams would have
>> topic
>> > > ids
>> > > > > > > in the topology as well. If that is not the case, we can also
>> add
>> > > > > > > topic names on our side. I will check this.
>> > > > > > >
>> > > > > > > 53. It seems that I was not clear. Sorry for that. I did not
>> mean
>> > > to
>> > > > > > > say that MemberAssignment.metadata is not used by the group
>> > > > > > > coordinator. Let me re-explain the flow.
>> > > > > > >
>> > > > > > > PartitionAssignor.assign() returns an Assignment. Assignment
>> > > contains
>> > > > > > > MemberAssignments where each member has
>> MemberAssignment.metadata.
>> > > The
>> > > > > > > Assignment is installed by the client with the
>> > > > > > > ConsumerGroupInstallAssignment API and this overrides the
>> current
>> > > > > > > target assignment in the coordinator. Then the group
>> coordinator
>> > > uses
>> > > > > > > it to reconcile members. Therefore the target assignment will
>> > > > > > > eventually become the current member assignment and it will be
>> > > > > > > delivered to the member, including the original
>> > > > > > > MemberAssignment.metadata defines by the assignor.
>> > > > > > >
>> > > > > > > I hope that this helps to clarify how
>> MemberAssignment.metadata is
>> > > > > > > used afterwards.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > David
>> > > > > > >
>> > > > > > > On Tue, Sep 27, 2022 at 10:31 PM Jun Rao
>> <jun@confluent.io.invalid
>> > > >
>> > > > > wrote:
>> > > > > > > > Hi, David,
>> > > > > > > >
>> > > > > > > > Thanks for the reply. A few more follow up questions.
>> > > > > > > >
>> > > > > > > > 48. Hmm, so member assignment is what the client sees.
>> Then, I
>> > > am not
>> > > > > > > sure
>> > > > > > > > about the following. The text says B cannot get foo-2 yet.
>> But in
>> > > > > Member
>> > > > > > > > Assignment, B has foo-2.
>> > > > > > > >
>> > > > > > > > At this point B can transition to epoch 2 but cannot get
>> foo-2
>> > > until
>> > > > > A
>> > > > > > > > revokes it.
>> > > > > > > >
>> > > > > > > > Note that the persisted target assignment of B already
>> includes
>> > > foo-2
>> > > > > > > here
>> > > > > > > > but B does not get it yet because foo-2 is not free.
>> > > > > > > >
>> > > > > > > > Group (epoch=2)
>> > > > > > > > A
>> > > > > > > > B
>> > > > > > > > Target Assignment (epoch=2)
>> > > > > > > > A - partitions=[foo-0, foo-1]
>> > > > > > > > B - partitions=[foo-2]
>> > > > > > > > Member Assignment
>> > > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2]
>> > > > > > > > B - epoch=2, partitions=[foo-2]
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > 50. Hmm, I thought KStreams doesn't use range assignor all
>> the
>> > > time.
>> > > > > It
>> > > > > > > > does the assignment based on the topology. For example, if
>> you
>> > > are
>> > > > > > > joining
>> > > > > > > > two topics A and B and deriving the join result in a 3rd
>> topic C,
>> > > > > which
>> > > > > > > > will be aggregated. You want to use range to co-locate the
>> > > partitions
>> > > > > > > from
>> > > > > > > > topic A and B. However, for topic C, the partition could
>> > > potentially
>> > > > > be
>> > > > > > > > assigned in round robin. Without the topic name, KStreams
>> won't
>> > > know
>> > > > > > > which
>> > > > > > > > topics need to be co-located.
>> > > > > > > >
>> > > > > > > > 53. If MemberAssignment.metadata is not used by the group
>> > > > > coordinator,
>> > > > > > > > where is it being used?
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > >
>> > > > > > > > Jun
>> > > > > > > >
>> > > > > > > > On Tue, Sep 27, 2022 at 1:20 PM Jason Gustafson
>> > > > > > > <ja...@confluent.io.invalid>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi David,
>> > > > > > > > >
>> > > > > > > > > Thanks for the response. Replied below:
>> > > > > > > > >
>> > > > > > > > > > I don't have a strong opinion on this one. However, it
>> seems
>> > > that
>> > > > > > > this
>> > > > > > > > > could be a pandora box if our subset turns out to not be
>> > > sufficient
>> > > > > > > > > for some existing users. We will have to add more to it
>> and the
>> > > > > > > > > compatibility story between versions could be a real
>> > > nightmare. One
>> > > > > > > > > thing that we could do for sure is disabling all unicode
>> > > groups in
>> > > > > the
>> > > > > > > > > library. That seems possible and would already reduce the
>> > > scope.
>> > > > > > > > >
>> > > > > > > > > Changes to the regex syntax seems not a major problem to
>> me. We
>> > > > > would
>> > > > > > > bump
>> > > > > > > > > the protocol version as we usually do. Actually that begs
>> the
>> > > same
>> > > > > > > question
>> > > > > > > > > about RE2. How stable is the syntax and does it differ
>> > > depending
>> > > > > on the
>> > > > > > > > > version of the library? In general, I do agree that we'd
>> be
>> > > better
>> > > > > off
>> > > > > > > not
>> > > > > > > > > having to manage the syntax ourselves if we can convince
>> > > ourselves
>> > > > > that
>> > > > > > > > > compatibility won't be a problem and the client won't
>> > > eventually
>> > > > > need
>> > > > > > > to
>> > > > > > > > > have access to it.
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > Jason
>> > > > > > > > >
>> > > > > > > > > On Tue, Sep 27, 2022 at 6:21 AM David Jacot
>> > > > > > > <dj...@confluent.io.invalid>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi Luke,
>> > > > > > > > > >
>> > > > > > > > > > Thanks for your comments.
>> > > > > > > > > >
>> > > > > > > > > > Your suggestion makes sense. Jun's made the same one.
>> We are
>> > > also
>> > > > > > > > > > discussing whether we should directly return the group
>> state
>> > > in
>> > > > > the
>> > > > > > > > > > response and drop the prepare assignment api. It seems
>> that
>> > > the
>> > > > > > > > > > current consensus is to keep it though. If that stays
>> like
>> > > this,
>> > > > > I
>> > > > > > > > > > will apply your suggestion.
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > David
>> > > > > > > > > >
>> > > > > > > > > > On Tue, Sep 27, 2022 at 10:40 AM Luke Chen <
>> > > showuon@gmail.com>
>> > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > Hi David,
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks for your patience for answering all these
>> questions!
>> > > > > > > > > > >
>> > > > > > > > > > > One more question from me:
>> > > > > > > > > > > 1. When the selected consumer is going to be in
>> client-side
>> > > > > > > assigning,
>> > > > > > > > > we
>> > > > > > > > > > > will return COMPUTE_ASSIGNMENT in heartbeat response.
>> I'm
>> > > > > > > wondering if
>> > > > > > > > > we
>> > > > > > > > > > > could use a flag to indicate the assignment is
>> required.
>> > > The
>> > > > > > > rationale
>> > > > > > > > > is
>> > > > > > > > > > > when returning error, we lose a chance to catch up
>> with the
>> > > > > latest
>> > > > > > > > > target
>> > > > > > > > > > > assignment or revoke partitions. Since we expected the
>> > > record
>> > > > > > > fetching
>> > > > > > > > > is
>> > > > > > > > > > > not blocked by the rebalance/assignment computing, we
>> might
>> > > > > want to
>> > > > > > > > > allow
>> > > > > > > > > > > the members to update the member assignment in each
>> > > heartbeat.
>> > > > > > > > > > >
>> > > > > > > > > > > Luke
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, Sep 27, 2022 at 9:43 AM Jason Gustafson
>> > > > > > > > > > <ja...@confluent.io.invalid>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hey David,
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks for the response.
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Do you think that we could have a case where two
>> > > members
>> > > > > with
>> > > > > > > the
>> > > > > > > > > > same
>> > > > > > > > > > > > static id are active?
>> > > > > > > > > > > >
>> > > > > > > > > > > > I was just thinking about the handoff when we spin
>> up a
>> > > new
>> > > > > node
>> > > > > > > > > > before the
>> > > > > > > > > > > > old one completes shutdown. It's probably ok to
>> fence
>> > > the old
>> > > > > > > member
>> > > > > > > > > > > > abruptly. I do recall we had a few issues with this
>> > > handoff
>> > > > > in
>> > > > > > > the
>> > > > > > > > > > past. It
>> > > > > > > > > > > > was due to the fact that the replaced node might
>> have
>> > > been
>> > > > > the
>> > > > > > > > > leader.
>> > > > > > > > > > > > Since we've removed the leader notion, perhaps this
>> kind
>> > > of
>> > > > > issue
>> > > > > > > > > won't
>> > > > > > > > > > > > resurface.
>> > > > > > > > > > > >
>> > > > > > > > > > > > In regard to the RE2 library. I understand the
>> intent to
>> > > pass
>> > > > > > > this
>> > > > > > > > > > through
>> > > > > > > > > > > > as a string without any validation so that we don't
>> need
>> > > the
>> > > > > > > > > > dependence on
>> > > > > > > > > > > > the client. I do wonder if that will be restrictive
>> in
>> > > > > practice.
>> > > > > > > For
>> > > > > > > > > > > > example, if a regex changes on the client side,
>> would we
>> > > > > need to
>> > > > > > > > > > validate
>> > > > > > > > > > > > against the current assignment to see which
>> partitions to
>> > > > > > > continue
>> > > > > > > > > > > > fetching? Or would we instead invalidate the
>> complete
>> > > current
>> > > > > > > > > > assignment?
>> > > > > > > > > > > > Perhaps this is a rare case and we can do the
>> latter.
>> > > There
>> > > > > > > could be
>> > > > > > > > > > more
>> > > > > > > > > > > > situations though.
>> > > > > > > > > > > >
>> > > > > > > > > > > > As an alternative, I wonder if we could define a
>> > > minimalistic
>> > > > > > > regex
>> > > > > > > > > > syntax
>> > > > > > > > > > > > (a subset of RE2 say) that could map more easily to
>> regex
>> > > > > support
>> > > > > > > > > that
>> > > > > > > > > > > > already exists in clients? For example, in this
>> > > scenario, we
>> > > > > are
>> > > > > > > > > > matching
>> > > > > > > > > > > > topics, so we only need to be able to represent
>> topic
>> > > > > characters
>> > > > > > > and
>> > > > > > > > > > not
>> > > > > > > > > > > > the whole smorgasbord of whitespace and unicode
>> options.
>> > > > > Would it
>> > > > > > > > > make
>> > > > > > > > > > > > sense to have a reduced syntax which includes only
>> > > standard
>> > > > > topic
>> > > > > > > > > > > > characters, wildcards, character classes, and
>> > > repetitions?
>> > > > > > > > > > > >
>> > > > > > > > > > > > Best,
>> > > > > > > > > > > > Jason
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, Sep 22, 2022 at 9:02 AM Guozhang Wang <
>> > > > > > > wangguoz@gmail.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Hi David, thanks for all the detailed
>> explanations. I
>> > > think
>> > > > > > > they
>> > > > > > > > > all
>> > > > > > > > > > make
>> > > > > > > > > > > > > sense. Just want to have a couple follow-ups here:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > I don't really see the benefits here because
>> server
>> > > side
>> > > > > > > > > assignors
>> > > > > > > > > > > > > don't have metadata at all. They only assign
>> > > > > topic-partitions.
>> > > > > > > They
>> > > > > > > > > > > > > are not supposed to generate metadata nor to
>> receive
>> > > > > metadata
>> > > > > > > from
>> > > > > > > > > > the
>> > > > > > > > > > > > > members.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Just to clarify I was asking about the `version`
>> of the
>> > > > > > > assignor
>> > > > > > > > > > (i.e. up
>> > > > > > > > > > > > > to what version that the client would support),
>> and I
>> > > do
>> > > > > agree
>> > > > > > > we
>> > > > > > > > > > would
>> > > > > > > > > > > > not
>> > > > > > > > > > > > > need metadata. What I have in mind is that, for
>> some
>> > > > > specific
>> > > > > > > > > > built-in
>> > > > > > > > > > > > > broker-assignors, e.g. rack-aware assignors, if
>> it's
>> > > > > possible
>> > > > > > > that
>> > > > > > > > > > in a
>> > > > > > > > > > > > > newer version we would have a hierarchical rack ID
>> > > string
>> > > > > > > format,
>> > > > > > > > > > like
>> > > > > > > > > > > > > "tier1-tier2" etc, but if some client has not
>> upgraded
>> > > > > their
>> > > > > > > rack
>> > > > > > > > > ID
>> > > > > > > > > > > > > would still be in old format. In this case, the
>> broker
>> > > then
>> > > > > > > needs
>> > > > > > > > > to
>> > > > > > > > > > > > choose
>> > > > > > > > > > > > > the old versioned assignor. I'm probably making
>> > > something
>> > > > > up
>> > > > > > > here
>> > > > > > > > > for
>> > > > > > > > > > > > rack
>> > > > > > > > > > > > > aware assignors, but I'm wondering if in general
>> such
>> > > an
>> > > > > > > > > > "auto-downgrade"
>> > > > > > > > > > > > > behavior would be needed still for broker-side
>> > > assignor,
>> > > > > and
>> > > > > > > if yes
>> > > > > > > > > > would
>> > > > > > > > > > > > > "version" still be useful.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Yeah, that's right. Within a rebalance,
>> > > `onAssignment` is
>> > > > > > > called
>> > > > > > > > > > once
>> > > > > > > > > > > > > when the member transitions to a new epoch. This
>> one
>> > > > > contains
>> > > > > > > the
>> > > > > > > > > > full
>> > > > > > > > > > > > > metadata provided by the client side assignor.
>> Then,
>> > > > > > > `onAssignment`
>> > > > > > > > > > > > > can be called max N times where N is the number of
>> > > > > partitions
>> > > > > > > > > pending
>> > > > > > > > > > > > > revocation by other members. Let me try to clarify
>> > > this in
>> > > > > the
>> > > > > > > KIP.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Okay, my understanding is that the calling
>> ordering of
>> > > > > these
>> > > > > > > > > > callbacks
>> > > > > > > > > > > > > would be like the following:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > ----------------------------------------
>> > > > > > > > > > > > > onPartitionsRevoked(); // just once, since we
>> do not
>> > > > > really
>> > > > > > > need
>> > > > > > > > > > > > > to revoke incrementally.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > onAssignment(); // the first call, with epoch
>> > > > > incremented
>> > > > > > > > > > > > > onPartitionsAssigned(); // paired with the
>> > > onAssignment
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > onAssignment(); // the first
>> onAssignment
>> > > > > would
>> > > > > > > bump
>> > > > > > > > > up
>> > > > > > > > > > the
>> > > > > > > > > > > > > epoch, and the metadata reflected.
>> > > > > > > > > > > > > onPartitionsAssigned(); // each time we get an
>> > > additional
>> > > > > > > > > > assignment,
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > call onAssignment and then paired with an
>> > > > > onPartitionsAssigned
>> > > > > > > > > > > > > ...
>> > > > > > > > > > > > > onAssignment();
>> > > > > > > > > > > > > onPartitionsAssigned(); // on each of the
>> > > onAssignment
>> > > > > > > calls, the
>> > > > > > > > > > > > encoded
>> > > > > > > > > > > > > metadata would not change, only the incrementally
>> added
>> > > > > > > partitions
>> > > > > > > > > be
>> > > > > > > > > > > > > reflected
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Is that the case?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > I'm wondering if we would still call onAssignment
>> just
>> > > > > once,
>> > > > > > > that
>> > > > > > > > > > encodes
>> > > > > > > > > > > > > all the assignment for this rebalance, including
>> all
>> > > the
>> > > > > > > partitions
>> > > > > > > > > > that
>> > > > > > > > > > > > > should be assigned to the member but not yet
>> assigned
>> > > since
>> > > > > > > they
>> > > > > > > > > > have not
>> > > > > > > > > > > > > been revoked by others. In that case the call
>> ordering
>> > > > > would
>> > > > > > > be:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > ----------------------------------------
>> > > > > > > > > > > > > onPartitionsRevoked(); // just once
>> > > > > > > > > > > > > onAssignment(); // just once, with epoch
>> > > incremented,
>> > > > > and
>> > > > > > > > > metadata
>> > > > > > > > > > > > > encoded changed, the "assignment" field also
>> reflect
>> > > the
>> > > > > final
>> > > > > > > > > target
>> > > > > > > > > > > > > assignment
>> > > > > > > > > > > > > onPartitionsAssigned(); // multiple times, which
>> > > > > represent
>> > > > > > > > > > > > incrementally
>> > > > > > > > > > > > > added partitions
>> > > > > > > > > > > > > ...
>> > > > > > > > > > > > > onPartitionsAssigned();
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The motivation from this is that, most users
>> would only
>> > > > > > > implement
>> > > > > > > > > the
>> > > > > > > > > > > > > rebalance callback listeners and hence we'd
>> definitely
>> > > > > need to
>> > > > > > > make
>> > > > > > > > > > sure
>> > > > > > > > > > > > > the semantics of that does not change much, and
>> the
>> > > time
>> > > > > > > > > > > > > `onPartitionsAssigned` indicate the time when the
>> > > > > partitions
>> > > > > > > are
>> > > > > > > > > > actually
>> > > > > > > > > > > > > assigned to it; while for assignors, the
>> > > `onAssignment` is
>> > > > > > > used to
>> > > > > > > > > > > > indicate
>> > > > > > > > > > > > > what decision is made regarding for this member,
>> i.e.
>> > > when
>> > > > > the
>> > > > > > > > > > partitions
>> > > > > > > > > > > > > are decided to be given to it, but not necessarily
>> > > meant
>> > > > > that
>> > > > > > > it
>> > > > > > > > > has
>> > > > > > > > > > been
>> > > > > > > > > > > > > given, since that time should be determined by the
>> > > time of
>> > > > > > > > > > > > > `onPartitionsAssigned`. The benefits though,
>> would be
>> > > that
>> > > > > > > assignor
>> > > > > > > > > > > > > implementers would not need to reason which
>> > > `onAssignment`
>> > > > > > > would be
>> > > > > > > > > > the
>> > > > > > > > > > > > > last one for this epoch.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Thu, Sep 22, 2022 at 2:20 AM David Jacot
>> > > > > > > > > > <djacot@confluent.io.invalid
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hi Guozhang,
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 1) The client-side
>> "PartitionAssignor#Assignment"
>> > > has
>> > > > > an
>> > > > > > > Error
>> > > > > > > > > > byte
>> > > > > > > > > > > > > > field,
>> > > > > > > > > > > > > > > while the broker-side
>> > > "PartitionAssignor#Assignment"
>> > > > > does
>> > > > > > > not.
>> > > > > > > > > > And
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > rationale seems to be assuming that we should
>> > > always be
>> > > > > > > able to
>> > > > > > > > > > do
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > assignment at the broker-side assignor without
>> > > errors.
>> > > > > > > > > > Personally I
>> > > > > > > > > > > > > think
>> > > > > > > > > > > > > > > it's still potentially beneficial to add the
>> Error
>> > > > > field
>> > > > > > > even
>> > > > > > > > > for
>> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge
>> cases
>> > > where
>> > > > > some
>> > > > > > > > > > subscribed
>> > > > > > > > > > > > > > > topics are not recognized with the current
>> broker's
>> > > > > > > metadata.
>> > > > > > > > > > What do
>> > > > > > > > > > > > > you
>> > > > > > > > > > > > > > > think?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Yeah, that seems reasonable. However, I wonder
>> if it
>> > > > > would be
>> > > > > > > > > > better
>> > > > > > > > > > > > > > to use an exception on the server side. This is
>> what
>> > > we
>> > > > > > > usually
>> > > > > > > > > do
>> > > > > > > > > > for
>> > > > > > > > > > > > > > server side plugins. On the client side, we use
>> a
>> > > field
>> > > > > > > because
>> > > > > > > > > the
>> > > > > > > > > > > > > > errors are not defined in advance.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Your comment also makes me think about what we
>> > > should do
>> > > > > > > when the
>> > > > > > > > > > > > > > server side assignor fails. I suppose that we
>> have to
>> > > > > keep
>> > > > > > > the
>> > > > > > > > > > current
>> > > > > > > > > > > > > > assignment until a new event occurs. For
>> instance, in
>> > > > > your
>> > > > > > > > > example,
>> > > > > > > > > > > > > > the coordinator would have to trigger a
>> rebalance
>> > > when
>> > > > > > > > > unrecognized
>> > > > > > > > > > > > > > topics are available. This would be part of the
>> > > metadata
>> > > > > > > > > > monitoring.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
>> > > additional
>> > > > > > > fields
>> > > > > > > > > > > > > > > reason/version/metadata compared with the
>> > > broker-side
>> > > > > > > > > > GroupMember. I
>> > > > > > > > > > > > > > agree
>> > > > > > > > > > > > > > > that broker-side assignor would not need
>> > > > > reason/metadata
>> > > > > > > since
>> > > > > > > > > > they
>> > > > > > > > > > > > are
>> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
>> what
>> > > about
>> > > > > > > version?
>> > > > > > > > > > E.g.
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > > possible that we evolve our broker-side
>> built-in
>> > > > > assignor
>> > > > > > > but
>> > > > > > > > > > the old
>> > > > > > > > > > > > > > > versioned clients would not be able to work
>> with
>> > > the
>> > > > > new
>> > > > > > > > > > version, in
>> > > > > > > > > > > > > > which
>> > > > > > > > > > > > > > > case we need to let the broker being aware of
>> this
>> > > and
>> > > > > > > upgrade
>> > > > > > > > > > its
>> > > > > > > > > > > > > > behavior
>> > > > > > > > > > > > > > > to cooperate with the clients?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > I don't really see the benefits here because
>> server
>> > > side
>> > > > > > > > > assignors
>> > > > > > > > > > > > > > don't have metadata at all. They only assign
>> > > > > > > topic-partitions.
>> > > > > > > > > They
>> > > > > > > > > > > > > > are not supposed to generate metadata nor to
>> receive
>> > > > > metadata
>> > > > > > > > > from
>> > > > > > > > > > the
>> > > > > > > > > > > > > > members.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
>> client-side
>> > > > > > > "GroupMember",
>> > > > > > > > > > > > instead
>> > > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > including these three fields, what about just
>> > > adding
>> > > > > the
>> > > > > > > > > > "Metadata"
>> > > > > > > > > > > > > field
>> > > > > > > > > > > > > > > class which has these three fields? Also,
>> there
>> > > are two
>> > > > > > > > > > "Metadata"
>> > > > > > > > > > > > > > > currently in the APIs, the first is a class
>> that
>> > > > > encodes
>> > > > > > > > > > > > > > > reason/version/metadata, and the second is
>> just the
>> > > > > encoded
>> > > > > > > > > > metadata
>> > > > > > > > > > > > > > bytes.
>> > > > > > > > > > > > > > > I'm wondering what about just naming the
>> first as
>> > > > > > > > > memberMetadata,
>> > > > > > > > > > > > which
>> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
>> instead
>> > > > > naming the
>> > > > > > > > > > second
>> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > That's a good point. Let me try to rationalize
>> this
>> > > > > interface
>> > > > > > > > > > based on
>> > > > > > > > > > > > > > your suggestions.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > Best,
>> > > > > > > > > > > > > > David
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 9:21 PM Guozhang Wang <
>> > > > > > > > > wangguoz@gmail.com>
>> > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Hello David,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Just had a few more nit questions about the
>> public
>> > > > > APIs:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 1) The client-side
>> "PartitionAssignor#Assignment"
>> > > has
>> > > > > an
>> > > > > > > Error
>> > > > > > > > > > byte
>> > > > > > > > > > > > > > field,
>> > > > > > > > > > > > > > > while the broker-side
>> > > "PartitionAssignor#Assignment"
>> > > > > does
>> > > > > > > not.
>> > > > > > > > > > And
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > rationale seems to be assuming that we should
>> > > always be
>> > > > > > > able to
>> > > > > > > > > > do
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > assignment at the broker-side assignor without
>> > > errors.
>> > > > > > > > > > Personally I
>> > > > > > > > > > > > > think
>> > > > > > > > > > > > > > > it's still potentially beneficial to add the
>> Error
>> > > > > field
>> > > > > > > even
>> > > > > > > > > for
>> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge
>> cases
>> > > where
>> > > > > some
>> > > > > > > > > > subscribed
>> > > > > > > > > > > > > > > topics are not recognized with the current
>> broker's
>> > > > > > > metadata.
>> > > > > > > > > > What do
>> > > > > > > > > > > > > you
>> > > > > > > > > > > > > > > think?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
>> > > additional
>> > > > > > > fields
>> > > > > > > > > > > > > > > reason/version/metadata compared with the
>> > > broker-side
>> > > > > > > > > > GroupMember. I
>> > > > > > > > > > > > > > agree
>> > > > > > > > > > > > > > > that broker-side assignor would not need
>> > > > > reason/metadata
>> > > > > > > since
>> > > > > > > > > > they
>> > > > > > > > > > > > are
>> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
>> what
>> > > about
>> > > > > > > version?
>> > > > > > > > > > E.g.
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > > possible that we evolve our broker-side
>> built-in
>> > > > > assignor
>> > > > > > > but
>> > > > > > > > > > the old
>> > > > > > > > > > > > > > > versioned clients would not be able to work
>> with
>> > > the
>> > > > > new
>> > > > > > > > > > version, in
>> > > > > > > > > > > > > > which
>> > > > > > > > > > > > > > > case we need to let the broker being aware of
>> this
>> > > and
>> > > > > > > upgrade
>> > > > > > > > > > its
>> > > > > > > > > > > > > > behavior
>> > > > > > > > > > > > > > > to cooperate with the clients?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
>> client-side
>> > > > > > > "GroupMember",
>> > > > > > > > > > > > instead
>> > > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > including these three fields, what about just
>> > > adding
>> > > > > the
>> > > > > > > > > > "Metadata"
>> > > > > > > > > > > > > field
>> > > > > > > > > > > > > > > class which has these three fields? Also,
>> there
>> > > are two
>> > > > > > > > > > "Metadata"
>> > > > > > > > > > > > > > > currently in the APIs, the first is a class
>> that
>> > > > > encodes
>> > > > > > > > > > > > > > > reason/version/metadata, and the second is
>> just the
>> > > > > encoded
>> > > > > > > > > > metadata
>> > > > > > > > > > > > > > bytes.
>> > > > > > > > > > > > > > > I'm wondering what about just naming the
>> first as
>> > > > > > > > > memberMetadata,
>> > > > > > > > > > > > which
>> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
>> instead
>> > > > > naming the
>> > > > > > > > > > second
>> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 12:08 PM Guozhang
>> Wang <
>> > > > > > > > > > wangguoz@gmail.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Hello David,
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Thanks for bringing this question up. I
>> think the
>> > > > > main
>> > > > > > > > > > benefits as
>> > > > > > > > > > > > > you
>> > > > > > > > > > > > > > > > listed is 2) above if it stays; just to
>> clarify,
>> > > we
>> > > > > would
>> > > > > > > > > only
>> > > > > > > > > > be
>> > > > > > > > > > > > > able
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > > save one round trip if the rebalance is
>> still
>> > > > > triggered
>> > > > > > > by
>> > > > > > > > > the
>> > > > > > > > > > > > > broker;
>> > > > > > > > > > > > > > if
>> > > > > > > > > > > > > > > > the rebalance is triggered by the client
>> then the
>> > > > > > > > > > num.round.trips
>> > > > > > > > > > > > are
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > same:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > 1) With GroupPrepareAssignment:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
>> > > suppose
>> > > > > it has
>> > > > > > > > > > already
>> > > > > > > > > > > > > sent
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
>> first
>> > > since
>> > > > > > > only
>> > > > > > > > > one
>> > > > > > > > > > > > > request
>> > > > > > > > > > > > > > /
>> > > > > > > > > > > > > > > > response can be inflight with the
>> coordinator's
>> > > > > socket.
>> > > > > > > > > > > > > > > > T1: client receives the HB response, and
>> then
>> > > sends
>> > > > > the
>> > > > > > > > > > > > > > > > GroupPrepareAssignment request.
>> > > > > > > > > > > > > > > > T2: the GroupPrepareAssignment response is
>> > > returned.
>> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
>> sends a
>> > > > > > > > > > > > > > GroupInstallAssignment
>> > > > > > > > > > > > > > > > request.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > In total, two round trips.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > 2) Without GroupPrepareAssignment:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
>> > > suppose
>> > > > > it has
>> > > > > > > > > > already
>> > > > > > > > > > > > > sent
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
>> first
>> > > since
>> > > > > > > only
>> > > > > > > > > one
>> > > > > > > > > > > > > request
>> > > > > > > > > > > > > > /
>> > > > > > > > > > > > > > > > response can be inflight with the
>> coordinator's
>> > > > > socket.
>> > > > > > > > > > > > > > > > T1: client receives the HB response, and
>> then
>> > > sends
>> > > > > the
>> > > > > > > new
>> > > > > > > > > HB
>> > > > > > > > > > > > > request
>> > > > > > > > > > > > > > > > with the flag indicating a new rebalance
>> needed..
>> > > > > > > > > > > > > > > > T2: the HB response with the optional member
>> > > metadata
>> > > > > > > map is
>> > > > > > > > > > > > > returned.
>> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
>> sends a
>> > > > > > > > > > > > > > GroupInstallAssignment
>> > > > > > > > > > > > > > > > request.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > In total, two round trips as well.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > -----------------------------
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > So to complete the full picture here, we'd
>> need
>> > > to
>> > > > > modify
>> > > > > > > > > both
>> > > > > > > > > > HB
>> > > > > > > > > > > > > > request
>> > > > > > > > > > > > > > > > and response so that the client can also
>> > > indicate a
>> > > > > new
>> > > > > > > > > > rebalance
>> > > > > > > > > > > > via
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > HB request as well, right?
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Assuming all above is true, I think it's
>> okay to
>> > > > > merge
>> > > > > > > the
>> > > > > > > > > > > > > > > > GroupPrepareAssignment into HB given that
>> we can
>> > > > > make the
>> > > > > > > > > > > > additional
>> > > > > > > > > > > > > > fields
>> > > > > > > > > > > > > > > > encoding the full member (subscription)
>> metadata
>> > > and
>> > > > > > > topic
>> > > > > > > > > > metadata
>> > > > > > > > > > > > > as
>> > > > > > > > > > > > > > > > optional fields.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > On Mon, Sep 12, 2022 at 5:22 AM David Jacot
>> > > > > > > > > > > > > > <dj...@confluent.io.invalid>
>> > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> Hi all,
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> During an offline conversation, someone
>> asked
>> > > why we
>> > > > > > > need
>> > > > > > > > > the
>> > > > > > > > > > > > > > > >> ConsumerGroupPrepareAssignment API and
>> suggested
>> > > > > that we
>> > > > > > > > > could
>> > > > > > > > > > > > > instead
>> > > > > > > > > > > > > > > >> provide the group state in the heartbeat
>> > > response.
>> > > > > This
>> > > > > > > has
>> > > > > > > > > a
>> > > > > > > > > > few
>> > > > > > > > > > > > > > > >> advantages: 1) it does not require using a
>> > > special
>> > > > > error
>> > > > > > > > > code
>> > > > > > > > > > to
>> > > > > > > > > > > > > > > >> signal that a new assignment is required
>> as the
>> > > > > signal
>> > > > > > > would
>> > > > > > > > > > be
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> provided group state; 2) it removes one
>> round
>> > > trip
>> > > > > when
>> > > > > > > a
>> > > > > > > > > > client
>> > > > > > > > > > > > > side
>> > > > > > > > > > > > > > > >> assignor is used. The downside is that it
>> makes
>> > > the
>> > > > > > > > > heartbeat
>> > > > > > > > > > > > > > > >> response's definition quite large. I recall
>> > > that I
>> > > > > went
>> > > > > > > with
>> > > > > > > > > > the
>> > > > > > > > > > > > > > > >> current approach due to this.
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> Providing the group state in the heartbeat
>> > > response
>> > > > > is
>> > > > > > > > > > appealing.
>> > > > > > > > > > > > > What
>> > > > > > > > > > > > > > > >> do you guys think?
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> Best,
>> > > > > > > > > > > > > > > >> David
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> On Mon, Sep 12, 2022 at 2:17 PM David
>> Jacot <
>> > > > > > > > > > djacot@confluent.io>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > Hi Guozhang,
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > 1. I have added a reference to the
>> relevant
>> > > > > chapter
>> > > > > > > > > instead
>> > > > > > > > > > of
>> > > > > > > > > > > > > > > >> > repeating the whole thing. Does that
>> work for
>> > > you?
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > 2. The "Rebalance Triggers" section you
>> are
>> > > > > referring
>> > > > > > > to
>> > > > > > > > > is
>> > > > > > > > > > > > about
>> > > > > > > > > > > > > > when
>> > > > > > > > > > > > > > > >> > a rebalance should be triggered for the
>> > > > > non-upgraded
>> > > > > > > > > members
>> > > > > > > > > > > > using
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > old protocol. The section mentions that a
>> > > > > rebalance
>> > > > > > > must
>> > > > > > > > > be
>> > > > > > > > > > > > > > triggered
>> > > > > > > > > > > > > > > >> > when a new assignment is installed. This
>> > > implies
>> > > > > that
>> > > > > > > the
>> > > > > > > > > > group
>> > > > > > > > > > > > > > epoch
>> > > > > > > > > > > > > > > >> > was updated either by a native member or
>> a
>> > > > > > > non-upgraded
>> > > > > > > > > > member.
>> > > > > > > > > > > > > For
>> > > > > > > > > > > > > > > >> > the latter, the JoinGroup request would
>> be the
>> > > > > > > trigger. I
>> > > > > > > > > > have
>> > > > > > > > > > > > > > added a
>> > > > > > > > > > > > > > > >> > reference to the relevant chapter in the
>> > > > > "JoinGroup
>> > > > > > > > > > Handling"
>> > > > > > > > > > > > > > section
>> > > > > > > > > > > > > > > >> > as well. Does that make sense?
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > Thanks,
>> > > > > > > > > > > > > > > >> > David
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > On Fri, Sep 9, 2022 at 10:35 PM Guozhang
>> Wang
>> > > <
>> > > > > > > > > > > > wangguoz@gmail.com
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> wrote:
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > Hello David,
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > Alright I think that's sufficient.
>> Just to
>> > > make
>> > > > > that
>> > > > > > > > > > clear in
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > doc,
>> > > > > > > > > > > > > > > >> > > could we update:
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > 1) the heartbeat request handling
>> section,
>> > > > > stating
>> > > > > > > when
>> > > > > > > > > > > > > > coordinator
>> > > > > > > > > > > > > > > >> will
>> > > > > > > > > > > > > > > >> > > trigger rebalance based on the HB's
>> member
>> > > > > metadata
>> > > > > > > /
>> > > > > > > > > > reason?
>> > > > > > > > > > > > > > > >> > > 2) the "Rebalance Triggers" section to
>> > > include
>> > > > > what
>> > > > > > > we
>> > > > > > > > > > > > described
>> > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> "Group
>> > > > > > > > > > > > > > > >> > > Epoch - Trigger a rebalance" section as
>> > > well?
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > Guozhang
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > On Fri, Sep 9, 2022 at 1:28 AM David
>> Jacot
>> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
>> > > > > > > > > > > > > > > >> > > wrote:
>> > > > > > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > Hi Guozhang,
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > I thought that the assignor will
>> always be
>> > > > > > > consulted
>> > > > > > > > > > when
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > next
>> > > > > > > > > > > > > > > >> > > > heartbeat request is constructed. In
>> other
>> > > > > words,
>> > > > > > > > > > > > > > > >> > > > `PartitionAssignor#metadata` will be
>> > > called
>> > > > > for
>> > > > > > > every
>> > > > > > > > > > > > > heartbeat.
>> > > > > > > > > > > > > > > >> This
>> > > > > > > > > > > > > > > >> > > > gives the opportunity for the
>> assignor to
>> > > > > enforce
>> > > > > > > a
>> > > > > > > > > > > > rebalance
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > > >> > > > setting the reason to a non-zero
>> value or
>> > > by
>> > > > > > > changing
>> > > > > > > > > > the
>> > > > > > > > > > > > > > bytes. Do
>> > > > > > > > > > > > > > > >> > > > you think that this is not
>> sufficient?
>> > > Are you
>> > > > > > > > > > concerned by
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> delay?
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > Best,
>> > > > > > > > > > > > > > > >> > > > David
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > On Fri, Sep 9, 2022 at 7:10 AM
>> Guozhang
>> > > Wang <
>> > > > > > > > > > > > > > wangguoz@gmail.com>
>> > > > > > > > > > > > > > > >> wrote:
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > Hello David,
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > One of Jun's comments make me
>> thinking:
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > ```
>> > > > > > > > > > > > > > > >> > > > > In this case, a new assignment is
>> > > triggered
>> > > > > by
>> > > > > > > the
>> > > > > > > > > > client
>> > > > > > > > > > > > > side
>> > > > > > > > > > > > > > > >> > > > > assignor. When constructing the
>> HB, the
>> > > > > consumer
>> > > > > > > > > will
>> > > > > > > > > > > > always
>> > > > > > > > > > > > > > > >> consult
>> > > > > > > > > > > > > > > >> > > > > the client side assignor and
>> propagate
>> > > the
>> > > > > > > > > > information to
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > coordinator. In other words, we
>> don't
>> > > expect
>> > > > > > > users
>> > > > > > > > > to
>> > > > > > > > > > call
>> > > > > > > > > > > > > > > >> > > > > Consumer#enforceRebalance anymore.
>> > > > > > > > > > > > > > > >> > > > > ```
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > As I looked at the current
>> > > > > PartitionAssignor's
>> > > > > > > > > > interface,
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > > > >> actually do
>> > > > > > > > > > > > > > > >> > > > > not have a way yet to instruct how
>> to
>> > > > > construct
>> > > > > > > the
>> > > > > > > > > > next
>> > > > > > > > > > > > HB
>> > > > > > > > > > > > > > > >> request, e.g.
>> > > > > > > > > > > > > > > >> > > > > when the assignor wants to enforce
>> a new
>> > > > > > > rebalance
>> > > > > > > > > > with a
>> > > > > > > > > > > > > new
>> > > > > > > > > > > > > > > >> assignment,
>> > > > > > > > > > > > > > > >> > > > > we'd need some customizable APIs
>> inside
>> > > the
>> > > > > > > > > > > > > PartitionAssignor
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> indicate
>> > > > > > > > > > > > > > > >> > > > > the next HB telling broker about
>> so.
>> > > WDYT
>> > > > > about
>> > > > > > > > > adding
>> > > > > > > > > > > > such
>> > > > > > > > > > > > > an
>> > > > > > > > > > > > > > > >> API on the
>> > > > > > > > > > > > > > > >> > > > > PartitionAssignor?
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > Guozhang
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > On Tue, Sep 6, 2022 at 6:09 AM
>> David
>> > > Jacot
>> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
>> > > > > > > > > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > Hi Jun,
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > I have updated the KIP to
>> include your
>> > > > > > > feedback. I
>> > > > > > > > > > have
>> > > > > > > > > > > > > also
>> > > > > > > > > > > > > > > >> tried to
>> > > > > > > > > > > > > > > >> > > > > > clarify the parts which were not
>> > > cleared.
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > Best,
>> > > > > > > > > > > > > > > >> > > > > > David
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > On Fri, Sep 2, 2022 at 4:18 PM
>> David
>> > > > > Jacot <
>> > > > > > > > > > > > > > djacot@confluent.io
>> > > > > > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Hi Jun,
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Thanks for your feedback. Let
>> me
>> > > start
>> > > > > by
>> > > > > > > > > > answering
>> > > > > > > > > > > > your
>> > > > > > > > > > > > > > > >> questions
>> > > > > > > > > > > > > > > >> > > > > > > inline and I will update the
>> KIP
>> > > next
>> > > > > week.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
>> the
>> > > main
>> > > > > > > benefits
>> > > > > > > > > > of
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > KIP
>> > > > > > > > > > > > > > > >> seem to
>> > > > > > > > > > > > > > > >> > > > be
>> > > > > > > > > > > > > > > >> > > > > > fewer
>> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and
>> more
>> > > > > efficient
>> > > > > > > > > > support of
>> > > > > > > > > > > > > > > >> wildcard. A few
>> > > > > > > > > > > > > > > >> > > > > > > > comments below.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > I would also add that the KIP
>> > > removes
>> > > > > the
>> > > > > > > global
>> > > > > > > > > > sync
>> > > > > > > > > > > > > > barrier
>> > > > > > > > > > > > > > > >> in the
>> > > > > > > > > > > > > > > >> > > > > > > protocol which is essential to
>> > > improve
>> > > > > group
>> > > > > > > > > > stability
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> > > > > > > scalability, and the KIP also
>> > > > > simplifies the
>> > > > > > > > > > client by
>> > > > > > > > > > > > > > moving
>> > > > > > > > > > > > > > > >> most of
>> > > > > > > > > > > > > > > >> > > > > > > the logic to the server side.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 30.
>> ConsumerGroupHeartbeatRequest
>> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
>> > > singleton.
>> > > > > Do we
>> > > > > > > plan
>> > > > > > > > > > to
>> > > > > > > > > > > > > > support
>> > > > > > > > > > > > > > > >> rolling
>> > > > > > > > > > > > > > > >> > > > > > changing
>> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
>> the
>> > > > > > > consumers?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Definitely. The group
>> coordinator
>> > > will
>> > > > > use
>> > > > > > > the
>> > > > > > > > > > > > assignor
>> > > > > > > > > > > > > > used
>> > > > > > > > > > > > > > > >> by a
>> > > > > > > > > > > > > > > >> > > > > > > majority of the members. This
>> > > allows the
>> > > > > > > group
>> > > > > > > > > to
>> > > > > > > > > > move
>> > > > > > > > > > > > > > from
>> > > > > > > > > > > > > > > >> one
>> > > > > > > > > > > > > > > >> > > > > > > assignor to another by a roll.
>> This
>> > > is
>> > > > > > > explained
>> > > > > > > > > > in
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> Assignor
>> > > > > > > > > > > > > > > >> > > > > > > Selection chapter.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could
>> you
>> > > explain
>> > > > > > > whether
>> > > > > > > > > > it's
>> > > > > > > > > > > > > > > >> required in
>> > > > > > > > > > > > > > > >> > > > every
>> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios
>> when it
>> > > > > needs to
>> > > > > > > be
>> > > > > > > > > > filled?
>> > > > > > > > > > > > > For
>> > > > > > > > > > > > > > > >> example,
>> > > > > > > > > > > > > > > >> > > > it's
>> > > > > > > > > > > > > > > >> > > > > > not
>> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
>> TopicPartitions
>> > > > > needs to
>> > > > > > > be
>> > > > > > > > > > filled.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > The client is expected to set
>> those
>> > > > > fields
>> > > > > > > in
>> > > > > > > > > > case of
>> > > > > > > > > > > > a
>> > > > > > > > > > > > > > > >> connection
>> > > > > > > > > > > > > > > >> > > > > > > issue (e.g. timeout) or when
>> the
>> > > fields
>> > > > > have
>> > > > > > > > > > changed
>> > > > > > > > > > > > > since
>> > > > > > > > > > > > > > > >> the last
>> > > > > > > > > > > > > > > >> > > > > > > HB. The server populates those
>> > > fields as
>> > > > > > > long as
>> > > > > > > > > > the
>> > > > > > > > > > > > > > member
>> > > > > > > > > > > > > > > >> is not
>> > > > > > > > > > > > > > > >> > > > > > > fully reconciled - the member
>> should
>> > > > > > > acknowledge
>> > > > > > > > > > that
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > has
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > expected epoch and assignment.
>> I
>> > > will
>> > > > > > > clarify
>> > > > > > > > > > this in
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > KIP.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
>> > > protocol,
>> > > > > the
>> > > > > > > rack
>> > > > > > > > > > > > > affinity
>> > > > > > > > > > > > > > > >> between the
>> > > > > > > > > > > > > > > >> > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
>> considered
>> > > > > during
>> > > > > > > > > > fetching,
>> > > > > > > > > > > > but
>> > > > > > > > > > > > > > not
>> > > > > > > > > > > > > > > >> during
>> > > > > > > > > > > > > > > >> > > > > > assigning
>> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
>> > > Sometimes,
>> > > > > once
>> > > > > > > the
>> > > > > > > > > > > > > assignment
>> > > > > > > > > > > > > > is
>> > > > > > > > > > > > > > > >> made,
>> > > > > > > > > > > > > > > >> > > > there
>> > > > > > > > > > > > > > > >> > > > > > is
>> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
>> affinity
>> > > > > because
>> > > > > > > no
>> > > > > > > > > > replicas
>> > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > >> assigned
>> > > > > > > > > > > > > > > >> > > > > > partitions
>> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
>> > > > > wondering
>> > > > > > > if we
>> > > > > > > > > > should
>> > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > > >> > > > > > opportunity
>> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
>> rack
>> > > in
>> > > > > > > > > > GroupMember.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > That's an interesting idea. I
>> don't
>> > > see
>> > > > > any
>> > > > > > > > > issue
>> > > > > > > > > > with
>> > > > > > > > > > > > > > adding
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > rack
>> > > > > > > > > > > > > > > >> > > > > > > to the members. I will do so.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side,
>> often,
>> > > it's
>> > > > > > > useful to
>> > > > > > > > > > know
>> > > > > > > > > > > > how
>> > > > > > > > > > > > > > busy
>> > > > > > > > > > > > > > > >> a group
>> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
>> > > event
>> > > > > loop
>> > > > > > > > > model,
>> > > > > > > > > > it
>> > > > > > > > > > > > > seems
>> > > > > > > > > > > > > > > >> that we
>> > > > > > > > > > > > > > > >> > > > could
>> > > > > > > > > > > > > > > >> > > > > > add
>> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
>> fraction
>> > > of
>> > > > > the
>> > > > > > > time
>> > > > > > > > > > the
>> > > > > > > > > > > > > event
>> > > > > > > > > > > > > > > >> loop is
>> > > > > > > > > > > > > > > >> > > > doing
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > actual work.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > That's a great idea. I will
>> add it.
>> > > > > Thanks.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
>> > > > > coordinator
>> > > > > > > > > > failover
>> > > > > > > > > > > > > > > >> handling? For
>> > > > > > > > > > > > > > > >> > > > > > example,
>> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
>> check
>> > > if
>> > > > > any
>> > > > > > > group
>> > > > > > > > > > with
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> wildcard
>> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
>> > > matching
>> > > > > topic?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Sure. When the new group
>> coordinator
>> > > > > takes
>> > > > > > > over,
>> > > > > > > > > > it
>> > > > > > > > > > > > has
>> > > > > > > > > > > > > > to:
>> > > > > > > > > > > > > > > >> > > > > > > * Setup the session timeouts.
>> > > > > > > > > > > > > > > >> > > > > > > * Trigger a new assignment if a
>> > > client
>> > > > > side
>> > > > > > > > > > assignor
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > > used.
>> > > > > > > > > > > > > > > >> We
>> > > > > > > > > > > > > > > >> > > > don't
>> > > > > > > > > > > > > > > >> > > > > > > store the information about the
>> > > member
>> > > > > > > selected
>> > > > > > > > > > to run
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> assignment
>> > > > > > > > > > > > > > > >> > > > > > > so we have to start a new one.
>> > > > > > > > > > > > > > > >> > > > > > > * Update the topics metadata,
>> > > verify the
>> > > > > > > > > wildcard
>> > > > > > > > > > > > > > > >> subscriptions, and
>> > > > > > > > > > > > > > > >> > > > > > > trigger a rebalance if needed.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 34.
>> ConsumerGroupMetadataValue,
>> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
>> > > > > > > > > > > > > > > >> > > > > > > >
>> ConsumerGroupMemberMetadataValue:
>> > > > > Could we
>> > > > > > > > > > document
>> > > > > > > > > > > > > what
>> > > > > > > > > > > > > > > >> the epoch
>> > > > > > > > > > > > > > > >> > > > > > field
>> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
>> the
>> > > epoch
>> > > > > in
>> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
>> > > > > > > > > > > > > > > >> > > > > > reflect
>> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
>> > > about the
>> > > > > > > one in
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > ConsumerGroupPartitionMetadataValue
>> > > > > and
>> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Sure. I will clarify that but
>> it is
>> > > > > always
>> > > > > > > the
>> > > > > > > > > > latest
>> > > > > > > > > > > > > > group
>> > > > > > > > > > > > > > > >> epoch.
>> > > > > > > > > > > > > > > >> > > > > > > When the group state is
>> updated, the
>> > > > > group
>> > > > > > > epoch
>> > > > > > > > > > is
>> > > > > > > > > > > > > > bumped so
>> > > > > > > > > > > > > > > >> we use
>> > > > > > > > > > > > > > > >> > > > > > > that one for all the change
>> records
>> > > > > related
>> > > > > > > to
>> > > > > > > > > the
>> > > > > > > > > > > > > update.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
>> will
>> > > ensure
>> > > > > > > that
>> > > > > > > > > the
>> > > > > > > > > > > > > > following
>> > > > > > > > > > > > > > > >> > > > invariants
>> > > > > > > > > > > > > > > >> > > > > > are
>> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members
>> exists." It's
>> > > > > > > possible
>> > > > > > > > > for
>> > > > > > > > > > a
>> > > > > > > > > > > > > member
>> > > > > > > > > > > > > > > >> not to
>> > > > > > > > > > > > > > > >> > > > get any
>> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > That's right. Here I meant
>> that the
>> > > > > members
>> > > > > > > > > > provided
>> > > > > > > > > > > > by
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> assignor
>> > > > > > > > > > > > > > > >> > > > > > > in the assignment must exist
>> in the
>> > > > > group.
>> > > > > > > The
>> > > > > > > > > > > > assignor
>> > > > > > > > > > > > > > can
>> > > > > > > > > > > > > > > >> not make
>> > > > > > > > > > > > > > > >> > > > > > > up new member ids.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
>> > > with a
>> > > > > > > member
>> > > > > > > > > > epoch
>> > > > > > > > > > > > > > equals to
>> > > > > > > > > > > > > > > >> 0":
>> > > > > > > > > > > > > > > >> > > > When
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
>> member
>> > > id
>> > > > > > > would be
>> > > > > > > > > > used?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > A member is expected to
>> abandon all
>> > > its
>> > > > > > > > > > partitions and
>> > > > > > > > > > > > > > > >> rejoins when
>> > > > > > > > > > > > > > > >> > > > it
>> > > > > > > > > > > > > > > >> > > > > > > receives the
>> FENCED_MEMBER_EPOCH
>> > > error.
>> > > > > In
>> > > > > > > this
>> > > > > > > > > > case,
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > > coordinator will have removed
>> the
>> > > member
>> > > > > > > from
>> > > > > > > > > the
>> > > > > > > > > > > > group.
>> > > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> member
>> > > > > > > > > > > > > > > >> > > > > > > can rejoin the group with the
>> same
>> > > > > member
>> > > > > > > id but
>> > > > > > > > > > with
>> > > > > > > > > > > > 0
>> > > > > > > > > > > > > as
>> > > > > > > > > > > > > > > >> epoch. Let
>> > > > > > > > > > > > > > > >> > > > > > > me see if I can clarify this
>> in the
>> > > KIP.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users
>> will
>> > > have
>> > > > > the
>> > > > > > > > > ability
>> > > > > > > > > > to
>> > > > > > > > > > > > > > trigger a
>> > > > > > > > > > > > > > > >> > > > > > reassignment
>> > > > > > > > > > > > > > > >> > > > > > > > by either providing a
>> non-zero
>> > > reason
>> > > > > or
>> > > > > > > by
>> > > > > > > > > > updating
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> assignor
>> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems
>> to be
>> > > > > > > conflicting
>> > > > > > > > > > with
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> deprecation
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > In this case, a new assignment
>> is
>> > > > > triggered
>> > > > > > > by
>> > > > > > > > > the
>> > > > > > > > > > > > > client
>> > > > > > > > > > > > > > side
>> > > > > > > > > > > > > > > >> > > > > > > assignor. When constructing
>> the HB,
>> > > the
>> > > > > > > consumer
>> > > > > > > > > > will
>> > > > > > > > > > > > > > always
>> > > > > > > > > > > > > > > >> consult
>> > > > > > > > > > > > > > > >> > > > > > > the client side assignor and
>> > > propagate
>> > > > > the
>> > > > > > > > > > information
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > > coordinator. In other words, we
>> > > don't
>> > > > > expect
>> > > > > > > > > > users to
>> > > > > > > > > > > > > call
>> > > > > > > > > > > > > > > >> > > > > > > Consumer#enforceRebalance
>> anymore.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment
>> examples are
>> > > > > nice.
>> > > > > > > But
>> > > > > > > > > the
>> > > > > > > > > > > > > section
>> > > > > > > > > > > > > > > >> seems to
>> > > > > > > > > > > > > > > >> > > > have
>> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
>> transitions to
>> > > > > epoch
>> > > > > > > 2, B
>> > > > > > > > > > > > > > immediately
>> > > > > > > > > > > > > > > >> gets into
>> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1,
>> partitions=[foo-2]",
>> > > which
>> > > > > seems
>> > > > > > > > > > > > incorrect.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
>> transitions to
>> > > > > epoch
>> > > > > > > 3, C
>> > > > > > > > > > seems
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > > get
>> > > > > > > > > > > > > > > >> into
>> > > > > > > > > > > > > > > >> > > > > > epoch=3,
>> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
>> epoch
>> > > 3, C
>> > > > > > > still
>> > > > > > > > > > has A -
>> > > > > > > > > > > > > > > >> epoch=2,
>> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Sorry for that! I will revise
>> them.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
>> consumers:
>> > > Do
>> > > > > we
>> > > > > > > > > support
>> > > > > > > > > > the
>> > > > > > > > > > > > > > upgrade
>> > > > > > > > > > > > > > > >> from
>> > > > > > > > > > > > > > > >> > > > any
>> > > > > > > > > > > > > > > >> > > > > > old
>> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > We will support upgrading from
>> the
>> > > > > consumer
>> > > > > > > > > > protocol
>> > > > > > > > > > > > > > version
>> > > > > > > > > > > > > > > >> 3,
>> > > > > > > > > > > > > > > >> > > > > > > introduced in KIP-792. KIP-792
>> is
>> > > not
>> > > > > > > > > implemented
>> > > > > > > > > > yet
>> > > > > > > > > > > > so
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> earliest
>> > > > > > > > > > > > > > > >> > > > > > > version is unknown at the
>> moment.
>> > > This
>> > > > > is
>> > > > > > > > > > explained in
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> migration
>> > > > > > > > > > > > > > > >> > > > > > > plan chapter.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Thanks again for your feedback,
>> > > Jun. I
>> > > > > will
>> > > > > > > > > > update the
>> > > > > > > > > > > > > KIP
>> > > > > > > > > > > > > > > >> based on
>> > > > > > > > > > > > > > > >> > > > it
>> > > > > > > > > > > > > > > >> > > > > > > next week.
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > Best,
>> > > > > > > > > > > > > > > >> > > > > > > David
>> > > > > > > > > > > > > > > >> > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > On Thu, Sep 1, 2022 at 9:07 PM
>> Jun
>> > > Rao
>> > > > > > > > > > > > > > > >> <ju...@confluent.io.invalid>
>> > > > > > > > > > > > > > > >> > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > Hi, David,
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
>> the
>> > > main
>> > > > > > > benefits
>> > > > > > > > > > of
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > KIP
>> > > > > > > > > > > > > > > >> seem to
>> > > > > > > > > > > > > > > >> > > > be
>> > > > > > > > > > > > > > > >> > > > > > fewer
>> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and
>> more
>> > > > > efficient
>> > > > > > > > > > support of
>> > > > > > > > > > > > > > > >> wildcard. A few
>> > > > > > > > > > > > > > > >> > > > > > > > comments below.
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 30.
>> ConsumerGroupHeartbeatRequest
>> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
>> > > singleton.
>> > > > > Do we
>> > > > > > > plan
>> > > > > > > > > > to
>> > > > > > > > > > > > > > support
>> > > > > > > > > > > > > > > >> rolling
>> > > > > > > > > > > > > > > >> > > > > > changing
>> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
>> the
>> > > > > > > consumers?
>> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could
>> you
>> > > explain
>> > > > > > > whether
>> > > > > > > > > > it's
>> > > > > > > > > > > > > > > >> required in
>> > > > > > > > > > > > > > > >> > > > every
>> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios
>> when it
>> > > > > needs to
>> > > > > > > be
>> > > > > > > > > > filled?
>> > > > > > > > > > > > > For
>> > > > > > > > > > > > > > > >> example,
>> > > > > > > > > > > > > > > >> > > > it's
>> > > > > > > > > > > > > > > >> > > > > > not
>> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
>> TopicPartitions
>> > > > > needs to
>> > > > > > > be
>> > > > > > > > > > filled.
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
>> > > protocol,
>> > > > > the
>> > > > > > > rack
>> > > > > > > > > > > > > affinity
>> > > > > > > > > > > > > > > >> between the
>> > > > > > > > > > > > > > > >> > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
>> considered
>> > > > > during
>> > > > > > > > > > fetching,
>> > > > > > > > > > > > but
>> > > > > > > > > > > > > > not
>> > > > > > > > > > > > > > > >> during
>> > > > > > > > > > > > > > > >> > > > > > assigning
>> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
>> > > Sometimes,
>> > > > > once
>> > > > > > > the
>> > > > > > > > > > > > > assignment
>> > > > > > > > > > > > > > is
>> > > > > > > > > > > > > > > >> made,
>> > > > > > > > > > > > > > > >> > > > there
>> > > > > > > > > > > > > > > >> > > > > > is
>> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
>> affinity
>> > > > > because
>> > > > > > > no
>> > > > > > > > > > replicas
>> > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > >> assigned
>> > > > > > > > > > > > > > > >> > > > > > partitions
>> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
>> > > > > wondering
>> > > > > > > if we
>> > > > > > > > > > should
>> > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > > >> > > > > > opportunity
>> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
>> rack
>> > > in
>> > > > > > > > > > GroupMember.
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side,
>> often,
>> > > it's
>> > > > > > > useful to
>> > > > > > > > > > know
>> > > > > > > > > > > > how
>> > > > > > > > > > > > > > busy
>> > > > > > > > > > > > > > > >> a group
>> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
>> > > event
>> > > > > loop
>> > > > > > > > > model,
>> > > > > > > > > > it
>> > > > > > > > > > > > > seems
>> > > > > > > > > > > > > > > >> that we
>> > > > > > > > > > > > > > > >> > > > could
>> > > > > > > > > > > > > > > >> > > > > > add
>> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
>> fraction
>> > > of
>> > > > > the
>> > > > > > > time
>> > > > > > > > > > the
>> > > > > > > > > > > > > event
>> > > > > > > > > > > > > > > >> loop is
>> > > > > > > > > > > > > > > >> > > > doing
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > actual work.
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
>> > > > > coordinator
>> > > > > > > > > > failover
>> > > > > > > > > > > > > > > >> handling? For
>> > > > > > > > > > > > > > > >> > > > > > example,
>> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
>> check
>> > > if
>> > > > > any
>> > > > > > > group
>> > > > > > > > > > with
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> wildcard
>> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
>> > > matching
>> > > > > topic?
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 34.
>> ConsumerGroupMetadataValue,
>> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
>> > > > > > > > > > > > > > > >> > > > > > > >
>> ConsumerGroupMemberMetadataValue:
>> > > > > Could we
>> > > > > > > > > > document
>> > > > > > > > > > > > > what
>> > > > > > > > > > > > > > > >> the epoch
>> > > > > > > > > > > > > > > >> > > > > > field
>> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
>> the
>> > > epoch
>> > > > > in
>> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
>> > > > > > > > > > > > > > > >> > > > > > reflect
>> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
>> > > about the
>> > > > > > > one in
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > ConsumerGroupPartitionMetadataValue
>> > > > > and
>> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
>> will
>> > > ensure
>> > > > > > > that
>> > > > > > > > > the
>> > > > > > > > > > > > > > following
>> > > > > > > > > > > > > > > >> > > > invariants
>> > > > > > > > > > > > > > > >> > > > > > are
>> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members
>> exists." It's
>> > > > > > > possible
>> > > > > > > > > for
>> > > > > > > > > > a
>> > > > > > > > > > > > > member
>> > > > > > > > > > > > > > > >> not to
>> > > > > > > > > > > > > > > >> > > > get any
>> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
>> > > with a
>> > > > > > > member
>> > > > > > > > > > epoch
>> > > > > > > > > > > > > > equals to
>> > > > > > > > > > > > > > > >> 0":
>> > > > > > > > > > > > > > > >> > > > When
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
>> member
>> > > id
>> > > > > > > would be
>> > > > > > > > > > used?
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users
>> will
>> > > have
>> > > > > the
>> > > > > > > > > ability
>> > > > > > > > > > to
>> > > > > > > > > > > > > > trigger a
>> > > > > > > > > > > > > > > >> > > > > > reassignment
>> > > > > > > > > > > > > > > >> > > > > > > > by either providing a
>> non-zero
>> > > reason
>> > > > > or
>> > > > > > > by
>> > > > > > > > > > updating
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> assignor
>> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems
>> to be
>> > > > > > > conflicting
>> > > > > > > > > > with
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> deprecation
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment
>> examples are
>> > > > > nice.
>> > > > > > > But
>> > > > > > > > > the
>> > > > > > > > > > > > > section
>> > > > > > > > > > > > > > > >> seems to
>> > > > > > > > > > > > > > > >> > > > have
>> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
>> transitions to
>> > > > > epoch
>> > > > > > > 2, B
>> > > > > > > > > > > > > > immediately
>> > > > > > > > > > > > > > > >> gets into
>> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1,
>> partitions=[foo-2]",
>> > > which
>> > > > > seems
>> > > > > > > > > > > > incorrect.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
>> transitions to
>> > > > > epoch
>> > > > > > > 3, C
>> > > > > > > > > > seems
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > > get
>> > > > > > > > > > > > > > > >> into
>> > > > > > > > > > > > > > > >> > > > > > epoch=3,
>> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
>> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
>> epoch
>> > > 3, C
>> > > > > > > still
>> > > > > > > > > > has A -
>> > > > > > > > > > > > > > > >> epoch=2,
>> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
>> consumers:
>> > > Do
>> > > > > we
>> > > > > > > > > support
>> > > > > > > > > > the
>> > > > > > > > > > > > > > upgrade
>> > > > > > > > > > > > > > > >> from
>> > > > > > > > > > > > > > > >> > > > any
>> > > > > > > > > > > > > > > >> > > > > > old
>> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > Thanks,
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > Jun
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > On Mon, Aug 29, 2022 at 9:20
>> AM
>> > > David
>> > > > > > > Jacot
>> > > > > > > > > > > > > > > >> > > > > > <dj...@confluent.io.invalid>
>> > > > > > > > > > > > > > > >> > > > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > Hi all,
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > The KIP states that we will
>> > > > > > > re-implement the
>> > > > > > > > > > > > > > coordinator
>> > > > > > > > > > > > > > > >> in
>> > > > > > > > > > > > > > > >> > > > Java. I
>> > > > > > > > > > > > > > > >> > > > > > > > > discussed this offline
>> with a
>> > > few
>> > > > > folks
>> > > > > > > and
>> > > > > > > > > > folks
>> > > > > > > > > > > > > are
>> > > > > > > > > > > > > > > >> concerned
>> > > > > > > > > > > > > > > >> > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > we could introduce many
>> > > regressions
>> > > > > in
>> > > > > > > the
>> > > > > > > > > old
>> > > > > > > > > > > > > > protocol
>> > > > > > > > > > > > > > > >> if we do
>> > > > > > > > > > > > > > > >> > > > so.
>> > > > > > > > > > > > > > > >> > > > > > > > > Therefore, I am going to
>> remove
>> > > this
>> > > > > > > > > statement
>> > > > > > > > > > > > from
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> KIP. It
>> > > > > > > > > > > > > > > >> > > > is an
>> > > > > > > > > > > > > > > >> > > > > > > > > implementation detail
>> after all
>> > > so
>> > > > > it
>> > > > > > > does
>> > > > > > > > > not
>> > > > > > > > > > > > have
>> > > > > > > > > > > > > > to be
>> > > > > > > > > > > > > > > >> > > > decided at
>> > > > > > > > > > > > > > > >> > > > > > > > > this stage. We will likely
>> > > start by
>> > > > > > > trying
>> > > > > > > > > to
>> > > > > > > > > > > > > > refactor the
>> > > > > > > > > > > > > > > >> > > > current
>> > > > > > > > > > > > > > > >> > > > > > > > > implementation as a first
>> step.
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > Cheers,
>> > > > > > > > > > > > > > > >> > > > > > > > > David
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > On Mon, Aug 29, 2022 at
>> 3:52 PM
>> > > > > David
>> > > > > > > Jacot
>> > > > > > > > > <
>> > > > > > > > > > > > > > > >> djacot@confluent.io
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > Hi Luke,
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
>> > > machine
>> > > > > are:
>> > > > > > > > > > "Empty,
>> > > > > > > > > > > > > > assigning,
>> > > > > > > > > > > > > > > >> > > > > > reconciling,
>> > > > > > > > > > > > > > > >> > > > > > > > > stable,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
>> Consumer
>> > > > > Group
>> > > > > > > States
>> > > > > > > > > > > > > section,
>> > > > > > > > > > > > > > > >> right?
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > This sentence does not
>> refer
>> > > to
>> > > > > those
>> > > > > > > > > group
>> > > > > > > > > > > > states
>> > > > > > > > > > > > > > but
>> > > > > > > > > > > > > > > >> rather
>> > > > > > > > > > > > > > > >> > > > to a
>> > > > > > > > > > > > > > > >> > > > > > > > > > state machine replication
>> > > (SMR).
>> > > > > This
>> > > > > > > > > > refers to
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> entire
>> > > > > > > > > > > > > > > >> > > > state of
>> > > > > > > > > > > > > > > >> > > > > > > > > > group coordinator which
>> is
>> > > > > replicated
>> > > > > > > via
>> > > > > > > > > > the
>> > > > > > > > > > > > log
>> > > > > > > > > > > > > > > >> layer. I will
>> > > > > > > > > > > > > > > >> > > > > > > > > > clarify this in the KIP.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
>> "each
>> > > > > state
>> > > > > > > > > machine
>> > > > > > > > > > is
>> > > > > > > > > > > > > > modelled
>> > > > > > > > > > > > > > > >> as an
>> > > > > > > > > > > > > > > >> > > > event
>> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > The idea is to follow a
>> model
>> > > > > similar
>> > > > > > > to
>> > > > > > > > > > the new
>> > > > > > > > > > > > > > quorum
>> > > > > > > > > > > > > > > >> > > > > > controller. We
>> > > > > > > > > > > > > > > >> > > > > > > > > > will have N threads to
>> process
>> > > > > events.
>> > > > > > > > > Each
>> > > > > > > > > > > > > > > >> __consumer_offsets
>> > > > > > > > > > > > > > > >> > > > > > > > > > partition is assigned to
>> a
>> > > unique
>> > > > > > > thread
>> > > > > > > > > > and all
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> events
>> > > > > > > > > > > > > > > >> > > > (e.g.
>> > > > > > > > > > > > > > > >> > > > > > > > > > requests, callbacks,
>> etc.) are
>> > > > > > > processed
>> > > > > > > > > by
>> > > > > > > > > > this
>> > > > > > > > > > > > > > > >> thread. This
>> > > > > > > > > > > > > > > >> > > > > > simplify
>> > > > > > > > > > > > > > > >> > > > > > > > > > concurrency and will
>> enable
>> > > us to
>> > > > > do
>> > > > > > > > > > simulation
>> > > > > > > > > > > > > > testing
>> > > > > > > > > > > > > > > >> for the
>> > > > > > > > > > > > > > > >> > > > > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > coordinator.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
>> state
>> > > > > machine
>> > > > > > > per
>> > > > > > > > > > > > > > > >> *__consumer_offsets*
>> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine
>> "per
>> > > > > consumer
>> > > > > > > group"
>> > > > > > > > > > owned
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > coordinator?
>> > > > > > > > > > > > > > > >> > > > > > > > > For
>> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
>> > > > > coordinator
>> > > > > > > owns 2
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> groups, and
>> > > > > > > > > > > > > > > >> > > > > > both
>> > > > > > > > > > > > > > > >> > > > > > > > > exist in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
>> > > will we
>> > > > > > > have 1
>> > > > > > > > > > state
>> > > > > > > > > > > > > > machine
>> > > > > > > > > > > > > > > >> for it,
>> > > > > > > > > > > > > > > >> > > > or
>> > > > > > > > > > > > > > > >> > > > > > 2?
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.1. The confusion
>> comes
>> > > from
>> > > > > > > there, I
>> > > > > > > > > > > > think.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
>> > > > > > > > > > "*group.coordinator.threads"
>> > > > > > > > > > > > *is
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> number
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > threads
>> > > > > > > > > > > > > > > >> > > > > > > > > used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
>> machines.
>> > > But
>> > > > > I'm
>> > > > > > > > > > wondering
>> > > > > > > > > > > > if
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> purpose
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > threads
>> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
>> state of
>> > > > > each
>> > > > > > > > > consumer
>> > > > > > > > > > > > group
>> > > > > > > > > > > > > > (or
>> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
>> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
>> heavy
>> > > > > > > computation,
>> > > > > > > > > > why
>> > > > > > > > > > > > > > should we
>> > > > > > > > > > > > > > > >> need
>> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
>> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.2. The idea is to
>> have
>> > > an
>> > > > > > > ability to
>> > > > > > > > > > shard
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > processing as
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > computation could be
>> heavy.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
>> session
>> > > > > timeout,
>> > > > > > > why
>> > > > > > > > > > does
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> default
>> > > > > > > > > > > > > > > >> > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
>> (45s) and
>> > > > > > > max(60s)? I
>> > > > > > > > > > > > thought
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> min/max
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
>> > > lower/upper
>> > > > > > > bound
>> > > > > > > > > of
>> > > > > > > > > > it,
>> > > > > > > > > > > > > no?
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > group.consumer.session.timeout.ms
>> > > > > > > int
>> > > > > > > > > > 30s The
>> > > > > > > > > > > > > > > >> timeout to
>> > > > > > > > > > > > > > > >> > > > detect
>> > > > > > > > > > > > > > > >> > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
>> > > consumer
>> > > > > > > group
>> > > > > > > > > > > > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > group.consumer.min.session.timeout.ms
>> > > > > > > > > > int 45s
>> > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> minimum
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > group.consumer.max.session.timeout.ms
>> > > > > > > > > > int 60s
>> > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> maximum
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > This is indeed a
>> mistake. The
>> > > > > default
>> > > > > > > > > > session
>> > > > > > > > > > > > > > timeout
>> > > > > > > > > > > > > > > >> should
>> > > > > > > > > > > > > > > >> > > > be 45s
>> > > > > > > > > > > > > > > >> > > > > > > > > > (the current default).
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default
>> server side
>> > > > > > > assignor
>> > > > > > > > > are
>> > > > > > > > > > > > > [range,
>> > > > > > > > > > > > > > > >> uniform],
>> > > > > > > > > > > > > > > >> > > > > > which means
>> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to
>> "range"
>> > > > > assignor.
>> > > > > > > I'd
>> > > > > > > > > > like to
>> > > > > > > > > > > > > > know
>> > > > > > > > > > > > > > > >> why not
>> > > > > > > > > > > > > > > >> > > > > > uniform
>> > > > > > > > > > > > > > > >> > > > > > > > > one? I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
>> will
>> > > > > choose
>> > > > > > > > > uniform
>> > > > > > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> (former
>> > > > > > > > > > > > > > > >> > > > sticky
>> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
>> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
>> > > distribution.
>> > > > > Any
>> > > > > > > > > other
>> > > > > > > > > > > > reason
>> > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > >> choose
>> > > > > > > > > > > > > > > >> > > > range
>> > > > > > > > > > > > > > > >> > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> group.consumer.assignors
>> > > List
>> > > > > range,
>> > > > > > > > > > uniform
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> server side
>> > > > > > > > > > > > > > > >> > > > > > assignors.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > The order on the server
>> side
>> > > has
>> > > > > no
>> > > > > > > > > > influence
>> > > > > > > > > > > > > > because
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > client
>> > > > > > > > > > > > > > > >> > > > > > must
>> > > > > > > > > > > > > > > >> > > > > > > > > > chose the selector that
>> he
>> > > wants
>> > > > > to
>> > > > > > > use.
>> > > > > > > > > > There
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > no
>> > > > > > > > > > > > > > > >> default
>> > > > > > > > > > > > > > > >> > > > in the
>> > > > > > > > > > > > > > > >> > > > > > > > > > current proposal. If the
>> > > assignor
>> > > > > is
>> > > > > > > not
>> > > > > > > > > > > > specified
>> > > > > > > > > > > > > > by
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > client,
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > request is rejected. The
>> > > default
>> > > > > > > client
>> > > > > > > > > > value
>> > > > > > > > > > > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > > `group.remote.assignor`
>> is
>> > > > > `uniform`
>> > > > > > > > > though.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for your very good
>> > > > > comments,
>> > > > > > > Luke.
>> > > > > > > > > I
>> > > > > > > > > > hope
>> > > > > > > > > > > > > > that my
>> > > > > > > > > > > > > > > >> > > > answers
>> > > > > > > > > > > > > > > >> > > > > > help
>> > > > > > > > > > > > > > > >> > > > > > > > > > to clarify things. I will
>> > > update
>> > > > > the
>> > > > > > > KIP
>> > > > > > > > > as
>> > > > > > > > > > well
>> > > > > > > > > > > > > > based
>> > > > > > > > > > > > > > > >> on your
>> > > > > > > > > > > > > > > >> > > > > > > > > > feedback.
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > Cheers,
>> > > > > > > > > > > > > > > >> > > > > > > > > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > On Mon, Aug 22, 2022 at
>> 9:29
>> > > AM
>> > > > > Luke
>> > > > > > > Chen
>> > > > > > > > > <
>> > > > > > > > > > > > > > > >> showuon@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Hi David,
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Thanks for the update.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Some more questions:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1. In Group Coordinator
>> > > > > section, you
>> > > > > > > > > > > > mentioned:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > The new group
>> coordinator
>> > > will
>> > > > > > > have a
>> > > > > > > > > > state
>> > > > > > > > > > > > > > machine
>> > > > > > > > > > > > > > > >> per
>> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets*
>> > > partitions,
>> > > > > > > where
>> > > > > > > > > > each
>> > > > > > > > > > > > > state
>> > > > > > > > > > > > > > > >> machine is
>> > > > > > > > > > > > > > > >> > > > > > modelled
>> > > > > > > > > > > > > > > >> > > > > > > > > as an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > event loop. Those state
>> > > machines
>> > > > > > > will be
>> > > > > > > > > > > > > executed
>> > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> *group.coordinator.threads*
>> > > > > threads.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
>> > > machine
>> > > > > are:
>> > > > > > > > > > "Empty,
>> > > > > > > > > > > > > > assigning,
>> > > > > > > > > > > > > > > >> > > > > > reconciling,
>> > > > > > > > > > > > > > > >> > > > > > > > > stable,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
>> Consumer
>> > > > > Group
>> > > > > > > States
>> > > > > > > > > > > > > section,
>> > > > > > > > > > > > > > > >> right?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
>> "each
>> > > > > state
>> > > > > > > > > machine
>> > > > > > > > > > is
>> > > > > > > > > > > > > > modelled
>> > > > > > > > > > > > > > > >> as an
>> > > > > > > > > > > > > > > >> > > > event
>> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
>> state
>> > > > > machine
>> > > > > > > per
>> > > > > > > > > > > > > > > >> *__consumer_offsets*
>> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine
>> "per
>> > > > > consumer
>> > > > > > > group"
>> > > > > > > > > > owned
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > coordinator?
>> > > > > > > > > > > > > > > >> > > > > > > > > For
>> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
>> > > > > coordinator
>> > > > > > > owns 2
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> groups, and
>> > > > > > > > > > > > > > > >> > > > > > both
>> > > > > > > > > > > > > > > >> > > > > > > > > exist in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
>> > > will we
>> > > > > > > have 1
>> > > > > > > > > > state
>> > > > > > > > > > > > > > machine
>> > > > > > > > > > > > > > > >> for it,
>> > > > > > > > > > > > > > > >> > > > or
>> > > > > > > > > > > > > > > >> > > > > > 2?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
>> > > > > > > > > > "*group.coordinator.threads"
>> > > > > > > > > > > > *is
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> number
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > threads
>> > > > > > > > > > > > > > > >> > > > > > > > > used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
>> machines.
>> > > But
>> > > > > I'm
>> > > > > > > > > > wondering
>> > > > > > > > > > > > if
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> purpose
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > threads
>> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
>> state of
>> > > > > each
>> > > > > > > > > consumer
>> > > > > > > > > > > > group
>> > > > > > > > > > > > > > (or
>> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
>> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
>> heavy
>> > > > > > > computation,
>> > > > > > > > > > why
>> > > > > > > > > > > > > > should we
>> > > > > > > > > > > > > > > >> need
>> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
>> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 2. For the default
>> value in
>> > > the
>> > > > > new
>> > > > > > > > > > configs:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
>> session
>> > > > > timeout,
>> > > > > > > why
>> > > > > > > > > > does
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> default
>> > > > > > > > > > > > > > > >> > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
>> (45s) and
>> > > > > > > max(60s)? I
>> > > > > > > > > > > > thought
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> min/max
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
>> > > lower/upper
>> > > > > > > bound
>> > > > > > > > > of
>> > > > > > > > > > it,
>> > > > > > > > > > > > > no?
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > group.consumer.session.timeout.ms
>> > > > > > > int
>> > > > > > > > > > 30s The
>> > > > > > > > > > > > > > > >> timeout to
>> > > > > > > > > > > > > > > >> > > > detect
>> > > > > > > > > > > > > > > >> > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
>> > > consumer
>> > > > > > > group
>> > > > > > > > > > > > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > group.consumer.min.session.timeout.ms
>> > > > > > > > > > int 45s
>> > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> minimum
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > group.consumer.max.session.timeout.ms
>> > > > > > > > > > int 60s
>> > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> maximum
>> > > > > > > > > > > > > > > >> > > > > > session
>> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default
>> server side
>> > > > > > > assignor
>> > > > > > > > > are
>> > > > > > > > > > > > > [range,
>> > > > > > > > > > > > > > > >> uniform],
>> > > > > > > > > > > > > > > >> > > > > > which means
>> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to
>> "range"
>> > > > > assignor.
>> > > > > > > I'd
>> > > > > > > > > > like to
>> > > > > > > > > > > > > > know
>> > > > > > > > > > > > > > > >> why not
>> > > > > > > > > > > > > > > >> > > > > > uniform
>> > > > > > > > > > > > > > > >> > > > > > > > > one? I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
>> will
>> > > > > choose
>> > > > > > > > > uniform
>> > > > > > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> (former
>> > > > > > > > > > > > > > > >> > > > sticky
>> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
>> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
>> > > distribution.
>> > > > > Any
>> > > > > > > > > other
>> > > > > > > > > > > > reason
>> > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > >> choose
>> > > > > > > > > > > > > > > >> > > > range
>> > > > > > > > > > > > > > > >> > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> group.consumer.assignors
>> > > List
>> > > > > range,
>> > > > > > > > > > uniform
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> server side
>> > > > > > > > > > > > > > > >> > > > > > assignors.
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Thank you.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > Luke
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Aug 22, 2022 at
>> > > 2:10 PM
>> > > > > Luke
>> > > > > > > > > Chen
>> > > > > > > > > > <
>> > > > > > > > > > > > > > > >> showuon@gmail.com
>> > > > > > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi Sagar,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > I have some thoughts
>> about
>> > > > > Kafka
>> > > > > > > > > Connect
>> > > > > > > > > > > > > > > >> integrating with
>> > > > > > > > > > > > > > > >> > > > > > KIP-848,
>> > > > > > > > > > > > > > > >> > > > > > > > > but I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > think we should have
>> a
>> > > > > separate
>> > > > > > > > > > discussion
>> > > > > > > > > > > > > > thread
>> > > > > > > > > > > > > > > >> for the
>> > > > > > > > > > > > > > > >> > > > Kafka
>> > > > > > > > > > > > > > > >> > > > > > > > > Connect
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > KIP: Integrating
>> Kafka
>> > > Connect
>> > > > > > > With
>> > > > > > > > > New
>> > > > > > > > > > > > > Consumer
>> > > > > > > > > > > > > > > >> Rebalance
>> > > > > > > > > > > > > > > >> > > > > > Protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > [1],
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > and let this
>> discussion
>> > > thread
>> > > > > > > focus
>> > > > > > > > > on
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> rebalance
>> > > > > > > > > > > > > > > >> > > > > > protocol,
>> > > > > > > > > > > > > > > >> > > > > > > > > WDYT?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > [1]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > Thank you.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > Luke
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Aug 12, 2022
>> at
>> > > 9:31
>> > > > > PM
>> > > > > > > Sagar
>> > > > > > > > > <
>> > > > > > > > > > > > > > > >> > > > > > sagarmeansocean@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thank you
>> Guozhang/David
>> > > for
>> > > > > the
>> > > > > > > > > > feedback.
>> > > > > > > > > > > > > > Looks
>> > > > > > > > > > > > > > > >> like
>> > > > > > > > > > > > > > > >> > > > there's
>> > > > > > > > > > > > > > > >> > > > > > > > > agreement on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> using separate APIs
>> for
>> > > > > Connect.
>> > > > > > > I
>> > > > > > > > > > would
>> > > > > > > > > > > > > > revisit
>> > > > > > > > > > > > > > > >> the doc
>> > > > > > > > > > > > > > > >> > > > and
>> > > > > > > > > > > > > > > >> > > > > > see
>> > > > > > > > > > > > > > > >> > > > > > > > > what
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> changes are to be
>> made.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thanks!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Sagar.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> On Tue, Aug 9, 2022
>> at
>> > > 7:11
>> > > > > PM
>> > > > > > > David
>> > > > > > > > > > Jacot
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> <dj...@confluent.io.invalid>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Hi Sagar,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Thanks for the
>> feedback
>> > > > > and the
>> > > > > > > > > > document.
>> > > > > > > > > > > > > > That's
>> > > > > > > > > > > > > > > >> really
>> > > > > > > > > > > > > > > >> > > > > > helpful. I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will take a look
>> at it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Overall, it seems
>> to me
>> > > > > that
>> > > > > > > both
>> > > > > > > > > > Connect
>> > > > > > > > > > > > > > and the
>> > > > > > > > > > > > > > > >> > > > Consumer
>> > > > > > > > > > > > > > > >> > > > > > could
>> > > > > > > > > > > > > > > >> > > > > > > > > share
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the same
>> underlying
>> > > > > "engine".
>> > > > > > > The
>> > > > > > > > > > main
>> > > > > > > > > > > > > > > >> difference is
>> > > > > > > > > > > > > > > >> > > > that
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > Consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigns
>> > > topic-partitions to
>> > > > > > > members
>> > > > > > > > > > > > whereas
>> > > > > > > > > > > > > > > >> Connect
>> > > > > > > > > > > > > > > >> > > > assigns
>> > > > > > > > > > > > > > > >> > > > > > tasks
>> > > > > > > > > > > > > > > >> > > > > > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > workers. I see two
>> > > ways to
>> > > > > move
>> > > > > > > > > > forward:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) We extend the
>> new
>> > > > > proposed
>> > > > > > > APIs
>> > > > > > > > > to
>> > > > > > > > > > > > > support
>> > > > > > > > > > > > > > > >> different
>> > > > > > > > > > > > > > > >> > > > > > resource
>> > > > > > > > > > > > > > > >> > > > > > > > > types
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g. partitions,
>> > > tasks,
>> > > > > > > etc.); or
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 2) We use new
>> dedicated
>> > > > > APIs
>> > > > > > > for
>> > > > > > > > > > Connect.
>> > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> dedicated
>> > > > > > > > > > > > > > > >> > > > APIs
>> > > > > > > > > > > > > > > >> > > > > > > > > would be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > similar to the new
>> > > ones but
>> > > > > > > > > > different on
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > content/resources and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they would rely
>> on the
>> > > same
>> > > > > > > engine
>> > > > > > > > > > on the
>> > > > > > > > > > > > > > > >> coordinator
>> > > > > > > > > > > > > > > >> > > > side.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I personally lean
>> > > towards
>> > > > > 2)
>> > > > > > > > > because
>> > > > > > > > > > I am
>> > > > > > > > > > > > > > not a
>> > > > > > > > > > > > > > > >> fan of
>> > > > > > > > > > > > > > > >> > > > > > > > > overcharging
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > APIs to serve
>> different
>> > > > > > > purposes.
>> > > > > > > > > > That
>> > > > > > > > > > > > > being
>> > > > > > > > > > > > > > > >> said, I am
>> > > > > > > > > > > > > > > >> > > > not
>> > > > > > > > > > > > > > > >> > > > > > > > > opposed to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) if we can find
>> an
>> > > > > elegant
>> > > > > > > way to
>> > > > > > > > > > do
>> > > > > > > > > > > > it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I think that we
>> can
>> > > > > continue to
>> > > > > > > > > > discuss
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > here
>> > > > > > > > > > > > > > > >> for now
>> > > > > > > > > > > > > > > >> > > > in
>> > > > > > > > > > > > > > > >> > > > > > order
>> > > > > > > > > > > > > > > >> > > > > > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ensure that this
>> KIP is
>> > > > > > > compatible
>> > > > > > > > > > with
>> > > > > > > > > > > > > what
>> > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > >> will do
>> > > > > > > > > > > > > > > >> > > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > Connect in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the future.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Best,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > On Mon, Aug 8,
>> 2022 at
>> > > > > 2:41 PM
>> > > > > > > > > David
>> > > > > > > > > > > > Jacot
>> > > > > > > > > > > > > <
>> > > > > > > > > > > > > > > >> > > > > > djacot@confluent.io>
>> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Hi all,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > I am back from
>> > > vacation.
>> > > > > I
>> > > > > > > will
>> > > > > > > > > go
>> > > > > > > > > > > > > through
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> address
>> > > > > > > > > > > > > > > >> > > > > > your
>> > > > > > > > > > > > > > > >> > > > > > > > > comments
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > in the coming
>> days.
>> > > > > Thanks
>> > > > > > > for
>> > > > > > > > > your
>> > > > > > > > > > > > > > feedback.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Cheers,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Aug 3,
>> 2022
>> > > at
>> > > > > 10:05
>> > > > > > > PM
>> > > > > > > > > > Gregory
>> > > > > > > > > > > > > > Harris
>> > > > > > > > > > > > > > > >> <
>> > > > > > > > > > > > > > > >> > > > > > > > > gharris1727@gmail.com
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Hey All!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Thanks for
>> the KIP,
>> > > > > it's
>> > > > > > > > > > wonderful to
>> > > > > > > > > > > > > see
>> > > > > > > > > > > > > > > >> > > > cooperative
>> > > > > > > > > > > > > > > >> > > > > > > > > rebalancing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > making it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > down the
>> stack!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > I had a few
>> > > questions:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 1. The
>> 'Rejected
>> > > > > > > Alternatives'
>> > > > > > > > > > > > section
>> > > > > > > > > > > > > > > >> describes how
>> > > > > > > > > > > > > > > >> > > > > > member
>> > > > > > > > > > > > > > > >> > > > > > > > > epoch
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > advance in
>> step
>> > > with
>> > > > > the
>> > > > > > > group
>> > > > > > > > > > epoch
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> assignment
>> > > > > > > > > > > > > > > >> > > > > > epoch
>> > > > > > > > > > > > > > > >> > > > > > > > > values. I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > that this is
>> a good
>> > > > > idea
>> > > > > > > for
>> > > > > > > > > the
>> > > > > > > > > > > > > reasons
>> > > > > > > > > > > > > > > >> described
>> > > > > > > > > > > > > > > >> > > > in
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > KIP. When
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > protocol is
>> > > > > incrementally
>> > > > > > > > > > assigning
>> > > > > > > > > > > > > > > >> partitions to a
>> > > > > > > > > > > > > > > >> > > > > > worker,
>> > > > > > > > > > > > > > > >> > > > > > > > > what
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> member
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > epoch does
>> each
>> > > > > incremental
>> > > > > > > > > > > > assignment
>> > > > > > > > > > > > > > use?
>> > > > > > > > > > > > > > > >> Are
>> > > > > > > > > > > > > > > >> > > > member
>> > > > > > > > > > > > > > > >> > > > > > epochs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> re-used,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > a single
>> member
>> > > epoch
>> > > > > can
>> > > > > > > > > > correspond
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> multiple
>> > > > > > > > > > > > > > > >> > > > > > different
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (monotonically
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > larger)
>> > > assignments?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 2. Is the
>> > > Assignor's
>> > > > > > > 'Reason'
>> > > > > > > > > > field
>> > > > > > > > > > > > > > opaque
>> > > > > > > > > > > > > > > >> to the
>> > > > > > > > > > > > > > > >> > > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> coordinator? If
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > not, should
>> custom
>> > > > > > > client-side
>> > > > > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > implementations
>> > > > > > > > > > > > > > > >> > > > > > > > > interact
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Reason field,
>> and
>> > > how
>> > > > > is
>> > > > > > > its
>> > > > > > > > > > common
>> > > > > > > > > > > > > > meaning
>> > > > > > > > > > > > > > > >> agreed
>> > > > > > > > > > > > > > > >> > > > > > upon? If
>> > > > > > > > > > > > > > > >> > > > > > > > > so, what
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > benefit of a
>> > > distinct
>> > > > > > > Reason
>> > > > > > > > > > field
>> > > > > > > > > > > > over
>> > > > > > > > > > > > > > > >> including
>> > > > > > > > > > > > > > > >> > > > such
>> > > > > > > > > > > > > > > >> > > > > > > > > functionality
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > opaque
>> metadata?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 3. The
>> following is
>> > > > > > > included in
>> > > > > > > > > > the
>> > > > > > > > > > > > > KIP:
>> > > > > > > > > > > > > > > >> "Thanks to
>> > > > > > > > > > > > > > > >> > > > > > this, the
>> > > > > > > > > > > > > > > >> > > > > > > > > input
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client side
>> > > assignor is
>> > > > > > > > > entirely
>> > > > > > > > > > > > driven
>> > > > > > > > > > > > > > by
>> > > > > > > > > > > > > > > >> the group
>> > > > > > > > > > > > > > > >> > > > > > > > > coordinator.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > consumer is no
>> > > longer
>> > > > > > > > > > responsible for
>> > > > > > > > > > > > > > > >> maintaining
>> > > > > > > > > > > > > > > >> > > > any
>> > > > > > > > > > > > > > > >> > > > > > state
>> > > > > > > > > > > > > > > >> > > > > > > > > besides
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> its
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assigned
>> > > partitions."
>> > > > > Does
>> > > > > > > this
>> > > > > > > > > > mean
>> > > > > > > > > > > > > > that the
>> > > > > > > > > > > > > > > >> > > > > > client-side
>> > > > > > > > > > > > > > > >> > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> MAY
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > incorporate
>> > > additional
>> > > > > > > > > > non-Metadata
>> > > > > > > > > > > > > state
>> > > > > > > > > > > > > > > >> (such as
>> > > > > > > > > > > > > > > >> > > > > > partition
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > throughput,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > cpu/memory
>> metrics,
>> > > > > config
>> > > > > > > > > > topics,
>> > > > > > > > > > > > > etc),
>> > > > > > > > > > > > > > or
>> > > > > > > > > > > > > > > >> that
>> > > > > > > > > > > > > > > >> > > > > > additional
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > non-Metadata
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > state SHOULD
>> NOT be
>> > > > > used?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 4. I see that
>> > > there are
>> > > > > > > > > separate
>> > > > > > > > > > > > > classes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > for
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > org.apache.kafka.server.group.consumer.PartitionAssignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > org.apache.kafka.clients.consumer.PartitionAssignor
>> > > > > > > > > > > > > > > >> > > > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > seem to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > overlap
>> > > significantly.
>> > > > > Is
>> > > > > > > it
>> > > > > > > > > > possible
>> > > > > > > > > > > > > for
>> > > > > > > > > > > > > > > >> these two
>> > > > > > > > > > > > > > > >> > > > > > > > > implementations
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > unified? This
>> would
>> > > > > serve
>> > > > > > > to
>> > > > > > > > > > promote
>> > > > > > > > > > > > > > feature
>> > > > > > > > > > > > > > > >> parity
>> > > > > > > > > > > > > > > >> > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > server-side
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client-side
>> > > assignors,
>> > > > > and
>> > > > > > > > > would
>> > > > > > > > > > also
>> > > > > > > > > > > > > > > >> facilitate
>> > > > > > > > > > > > > > > >> > > > > > operational
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flexibility in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > certain
>> > > situations. For
>> > > > > > > > > example,
>> > > > > > > > > > if a
>> > > > > > > > > > > > > > > >> server-side
>> > > > > > > > > > > > > > > >> > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > has some
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > poor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > behavior and
>> needs
>> > > a
>> > > > > patch,
>> > > > > > > > > > deploying
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> patched
>> > > > > > > > > > > > > > > >> > > > > > assignor to
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and switching
>> one
>> > > > > consumer
>> > > > > > > > > group
>> > > > > > > > > > to a
>> > > > > > > > > > > > > > > >> client-side
>> > > > > > > > > > > > > > > >> > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > may be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > faster
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and less
>> risky than
>> > > > > > > patching
>> > > > > > > > > all
>> > > > > > > > > > of
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> brokers.
>> > > > > > > > > > > > > > > >> > > > With
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > currently
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > proposed
>> distinct
>> > > > > APIs, a
>> > > > > > > > > > non-trivial
>> > > > > > > > > > > > > > > >> > > > reimplementation
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assembled,
>> and if
>> > > the
>> > > > > two
>> > > > > > > APIs
>> > > > > > > > > > have
>> > > > > > > > > > > > > > diverged
>> > > > > > > > > > > > > > > >> > > > > > significantly,
>> > > > > > > > > > > > > > > >> > > > > > > > > then it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > possible that
>> a
>> > > > > > > > > reimplementation
>> > > > > > > > > > > > would
>> > > > > > > > > > > > > > not be
>> > > > > > > > > > > > > > > >> > > > possible.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > --
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Greg Harris
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > gharris1727@gmail.com
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > github.com/gharris1727
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Aug 3,
>> > > 2022 at
>> > > > > > > 8:39 AM
>> > > > > > > > > > Sagar
>> > > > > > > > > > > > <
>> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Hi
>> > > Guozhang/David,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > I created a
>> > > > > confluence
>> > > > > > > page
>> > > > > > > > > to
>> > > > > > > > > > > > > discuss
>> > > > > > > > > > > > > > how
>> > > > > > > > > > > > > > > >> Connect
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > based on
>> the new
>> > > > > > > rebalance
>> > > > > > > > > > > > protocol.
>> > > > > > > > > > > > > > > >> Here's the
>> > > > > > > > > > > > > > > >> > > > page:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > It's also
>> pretty
>> > > > > longish
>> > > > > > > and
>> > > > > > > > > I
>> > > > > > > > > > have
>> > > > > > > > > > > > > > tried
>> > > > > > > > > > > > > > > >> to keep
>> > > > > > > > > > > > > > > >> > > > a
>> > > > > > > > > > > > > > > >> > > > > > format
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> similar to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > KIP-848.
>> Let me
>> > > know
>> > > > > > > what you
>> > > > > > > > > > > > think.
>> > > > > > > > > > > > > > Also,
>> > > > > > > > > > > > > > > >> do you
>> > > > > > > > > > > > > > > >> > > > > > think this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > moved to a
>> > > separate
>> > > > > > > > > discussion
>> > > > > > > > > > > > thread
>> > > > > > > > > > > > > > or
>> > > > > > > > > > > > > > > >> is this
>> > > > > > > > > > > > > > > >> > > > one
>> > > > > > > > > > > > > > > >> > > > > > fine?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Sagar.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul
>> 26,
>> > > 2022
>> > > > > at
>> > > > > > > 7:37
>> > > > > > > > > AM
>> > > > > > > > > > > > > Sagar <
>> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Hello
>> Guozhang,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thank you
>> so
>> > > much
>> > > > > for
>> > > > > > > the
>> > > > > > > > > > doc on
>> > > > > > > > > > > > > > Kafka
>> > > > > > > > > > > > > > > >> Streams.
>> > > > > > > > > > > > > > > >> > > > > > Sure, I
>> > > > > > > > > > > > > > > >> > > > > > > > > would do
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > analysis
>> and
>> > > come
>> > > > > up
>> > > > > > > with
>> > > > > > > > > > such a
>> > > > > > > > > > > > > > > >> document.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Sagar.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue,
>> Jul 26,
>> > > > > 2022 at
>> > > > > > > > > 4:47
>> > > > > > > > > > AM
>> > > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > > > > >> Wang <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wangguoz@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Hello
>> Sagar,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> It would
>> be
>> > > great
>> > > > > if
>> > > > > > > you
>> > > > > > > > > > could
>> > > > > > > > > > > > > come
>> > > > > > > > > > > > > > > >> back with
>> > > > > > > > > > > > > > > >> > > > some
>> > > > > > > > > > > > > > > >> > > > > > > > > analysis on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > how to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> implement the
>> > > > > Connect
>> > > > > > > side
>> > > > > > > > > > > > > > integration
>> > > > > > > > > > > > > > > >> with
>> > > > > > > > > > > > > > > >> > > > the new
>> > > > > > > > > > > > > > > >> > > > > > > > > protocol;
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> so
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > far
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> besides
>> > > > > leveraging on
>> > > > > > > the
>> > > > > > > > > > new
>> > > > > > > > > > > > > > "protocol
>> > > > > > > > > > > > > > > >> type"
>> > > > > > > > > > > > > > > >> > > > we
>> > > > > > > > > > > > > > > >> > > > > > did not
>> > > > > > > > > > > > > > > >> > > > > > > > > yet
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > through
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> Connect
>> > > side
>> > > > > > > > > > > > implementations.
>> > > > > > > > > > > > > > For
>> > > > > > > > > > > > > > > >> Streams
>> > > > > > > > > > > > > > > >> > > > > > here's a
>> > > > > > > > > > > > > > > >> > > > > > > > > draft of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> integration
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> plan:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > >
>> > > > > > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > >
>> > > > > > > > > > > > > > > >>
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>> https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> just FYI
>> for
>> > > your
>> > > > > > > analysis
>> > > > > > > > > > on
>> > > > > > > > > > > > > > Connect.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> On Tue,
>> Jul
>> > > 19,
>> > > > > 2022
>> > > > > > > at
>> > > > > > > > > > 10:48 PM
>> > > > > > > > > > > > > > Sagar <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > sagarmeansocean@gmail.com
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Hi
>> David,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thank
>> you
>> > > for
>> > > > > your
>> > > > > > > > > > response.
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> reason I
>> > > > > > > > > > > > > > > >> > > > thought
>> > > > > > > > > > > > > > > >> > > > > > > > > connect can
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > also fit
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> into
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > this
>> new
>> > > scheme
>> > > > > is
>> > > > > > > that
>> > > > > > > > > > even
>> > > > > > > > > > > > > > today the
>> > > > > > > > > > > > > > > >> > > > connect
>> > > > > > > > > > > > > > > >> > > > > > uses a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> WorkerCoordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> extending
>> > > from
>> > > > > > > > > > > > > > AbstractCoordinator to
>> > > > > > > > > > > > > > > >> empower
>> > > > > > > > > > > > > > > >> > > > > > > > > rebalances of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > tasks/connectors.
>> > > > > > > The
>> > > > > > > > > > > > > > > >> WorkerCoordinator sets
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocolType()
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> connect
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > and
>> uses the
>> > > > > > > metadata()
>> > > > > > > > > > method
>> > > > > > > > > > > > > by
>> > > > > > > > > > > > > > > >> plumbing
>> > > > > > > > > > > > > > > >> > > > into
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > JoinGroupRequestProtocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > I
>> think the
>> > > > > changes
>> > > > > > > to
>> > > > > > > > > > support
>> > > > > > > > > > > > > > > >> connect would
>> > > > > > > > > > > > > > > >> > > > be
>> > > > > > > > > > > > > > > >> > > > > > > > > similar at a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > high
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > level
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
>> changes
>> > > in
>> > > > > > > streams
>> > > > > > > > > > mainly
>> > > > > > > > > > > > > > because
>> > > > > > > > > > > > > > > >> of the
>> > > > > > > > > > > > > > > >> > > > > > Client
>> > > > > > > > > > > > > > > >> > > > > > > > > side
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assignors
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > being
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > used in
>> > > both.
>> > > > > At an
>> > > > > > > > > > > > > implementation
>> > > > > > > > > > > > > > > >> level, we
>> > > > > > > > > > > > > > > >> > > > > > might
>> > > > > > > > > > > > > > > >> > > > > > > > > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> make
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > a lot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> changes to
>> > > get
>> > > > > onto
>> > > > > > > this
>> > > > > > > > > > new
>> > > > > > > > > > > > > > > >> assignment
>> > > > > > > > > > > > > > > >> > > > protocol
>> > > > > > > > > > > > > > > >> > > > > > like
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> enhancing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> JoinGroup
>> > > > > > > > > > request/response and
>> > > > > > > > > > > > > > > >> SyncGroup and
>> > > > > > > > > > > > > > > >> > > > > > using
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > ConsumerGroupHeartbeat
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > API etc
>> > > again on
>> > > > > > > similar
>> > > > > > > > > > lines
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> streams (or
>> > > > > > > > > > > > > > > >> > > > > > there
>> > > > > > > > > > > > > > > >> > > > > > > > > might be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> deviations). I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > would
>> try to
>> > > > > > > perform a
>> > > > > > > > > > > > detailed
>> > > > > > > > > > > > > > > >> analysis of
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > same
>> > > > > > > > > > > > > > > >> > > > > > > > > and we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> separate
>> > > > > discussion
>> > > > > > > > > > thread for
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > >> as that
>> > > > > > > > > > > > > > > >> > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > derail this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > discussion
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> thread. Let
>> > > me
>> > > > > know
>> > > > > > > if
>> > > > > > > > > > that
>> > > > > > > > > > > > > sounds
>> > > > > > > > > > > > > > > >> good to
>> > > > > > > > > > > > > > > >> > > > you.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thanks!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Sagar.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > On
>> Fri, Jul
>> > > 15,
>> > > > > > > 2022 at
>> > > > > > > > > > 5:47
>> > > > > > > > > > > > PM
>> > > > > > > > > > > > > > David
>> > > > > > > > > > > > > > > >> Jacot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > <djacot@confluent.io.invalid
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Hi
>> Sagar,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> Thanks for
>> > > > > your
>> > > > > > > > > > comments.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 1)
>> Yes.
>> > > That
>> > > > > > > refers to
>> > > > > > > > > > > > > > > >> `Assignment#error`.
>> > > > > > > > > > > > > > > >> > > > > > Sure, I
>> > > > > > > > > > > > > > > >> > > > > > > > > can
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > mention it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 2)
>> The
>> > > idea
>> > > > > is to
>> > > > > > > > > > > > transition C
>> > > > > > > > > > > > > > from
>> > > > > > > > > > > > > > > >> his
>> > > > > > > > > > > > > > > >> > > > current
>> > > > > > > > > > > > > > > >> > > > > > > > > assignment
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > his
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> target
>> > > > > assignment
>> > > > > > > when
>> > > > > > > > > > he
>> > > > > > > > > > > > can
>> > > > > > > > > > > > > > move
>> > > > > > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > epoch 3.
>> > > > > > > > > > > > > > > >> > > > > > When
>> > > > > > > > > > > > > > > >> > > > > > > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > happens,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> member
>> > > > > assignment
>> > > > > > > is
>> > > > > > > > > > updated
>> > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> persisted
>> > > > > > > > > > > > > > > >> > > > > > with all
>> > > > > > > > > > > > > > > >> > > > > > > > > its
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigned
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> partitions
>> > > > > even if
>> > > > > > > > > they
>> > > > > > > > > > are
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > all
>> > > > > > > > > > > > > > > >> revoked
>> > > > > > > > > > > > > > > >> > > > > > yet. In
>> > > > > > > > > > > > > > > >> > > > > > > > > other
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > words, the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> member
>> > > > > assignment
>> > > > > > > > > > becomes
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > target
>> > > > > > > > > > > > > > > >> > > > > > assignment.
>> > > > > > > > > > > > > > > >> > > > > > > > > This is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > basically
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > optimization
>> > > > > to
>> > > > > > > avoid
>> > > > > > > > > > having
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> write all
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > changes to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> examples
>> > > are
>> > > > > > > based on
>> > > > > > > > > > the
>> > > > > > > > > > > > > > persisted
>> > > > > > > > > > > > > > > >> state
>> > > > > > > > > > > > > > > >> > > > so I
>> > > > > > > > > > > > > > > >> > > > > > > > > understand
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> confusion.
>> > > > > Let me
>> > > > > > > see
>> > > > > > > > > > if I
>> > > > > > > > > > > > can
>> > > > > > > > > > > > > > > >> improve
>> > > > > > > > > > > > > > > >> > > > this in
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > description.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 3)
>> > > Regarding
>> > > > > > > Connect,
>> > > > > > > > > it
>> > > > > > > > > > > > could
>> > > > > > > > > > > > > > > >> reuse the
>> > > > > > > > > > > > > > > >> > > > > > protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > with a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client side
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> assignor
>> > > if it
>> > > > > > > fits in
>> > > > > > > > > > the
>> > > > > > > > > > > > > > > >> protocol. The
>> > > > > > > > > > > > > > > >> > > > > > assignment
>> > > > > > > > > > > > > > > >> > > > > > > > > is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > topicid-partitions +
>> > > > > > > > > > > > metadata,
>> > > > > > > > > > > > > > could
>> > > > > > > > > > > > > > > >> > > > Connect
>> > > > > > > > > > > > > > > >> > > > > > fit
>> > > > > > > > > > > > > > > >> > > > > > > > > into this?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Best,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > On
>> Fri,
>> > > Jul
>> > > > > 15,
>> > > > > > > 2022
>> > > > > > > > > at
>> > > > > > > > > > 1:55
>> > > > > > > > > > > > > PM
>> > > > > > > > > > > > > > > >> Sagar <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > sagarmeansocean@gmail.com>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Hi
>> > > David,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> Thanks
>> > > for
>> > > > > the
>> > > > > > > KIP.
>> > > > > > > > > I
>> > > > > > > > > > just
>> > > > > > > > > > > > > had
>> > > > > > > > > > > > > > > >> minor
>> > > > > > > > > > > > > > > >> > > > > > observations:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 1)
>> In
>> > > the
>> > > > > > > Assignment
>> > > > > > > > > > Error
>> > > > > > > > > > > > > > > >> section in
>> > > > > > > > > > > > > > > >> > > > Client
>> > > > > > > > > > > > > > > >> > > > > > Side
>> > > > > > > > > > > > > > > >> > > > > > > > > mode
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Assignment
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> process,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > you
>> > > > > mentioned
>> > > > > > > => `In
>> > > > > > > > > > this
>> > > > > > > > > > > > > > case,
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > client
>> > > > > > > > > > > > > > > >> > > > > > side
>> > > > > > > > > > > > > > > >> > > > > > > > > assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> can
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > return
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> error
>> > > to the
>> > > > > > > group
>> > > > > > > > > > > > > > coordinator`.
>> > > > > > > > > > > > > > > >> In this
>> > > > > > > > > > > > > > > >> > > > > > case are
>> > > > > > > > > > > > > > > >> > > > > > > > > you
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > referring to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> Assignor
>> > > > > > > returning
>> > > > > > > > > an
>> > > > > > > > > > > > > > > >> AssignmentError
>> > > > > > > > > > > > > > > >> > > > that's
>> > > > > > > > > > > > > > > >> > > > > > > > > listed down
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > towards
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > end?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > If
>> yes,
>> > > do
>> > > > > you
>> > > > > > > think
>> > > > > > > > > > it
>> > > > > > > > > > > > > would
>> > > > > > > > > > > > > > > >> make sense
>> > > > > > > > > > > > > > > >> > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > mention this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > explicitly
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > here?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 2)
>> In
>> > > the
>> > > > > Case
>> > > > > > > > > Studies
>> > > > > > > > > > > > > > section, I
>> > > > > > > > > > > > > > > >> have a
>> > > > > > > > > > > > > > > >> > > > > > slight
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> confusion,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > sure
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> if
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> others
>> > > have
>> > > > > the
>> > > > > > > > > same.
>> > > > > > > > > > > > > Consider
>> > > > > > > > > > > > > > > >> this step:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> When B
>> > > > > > > heartbeats,
>> > > > > > > > > the
>> > > > > > > > > > > > group
>> > > > > > > > > > > > > > > >> coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > transitions him
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > epoch
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 3
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> because
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > B
>> has no
>> > > > > > > partitions
>> > > > > > > > > to
>> > > > > > > > > > > > > > revoke. It
>> > > > > > > > > > > > > > > >> > > > persists
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > change and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > reply.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > Group
>> > > > > > > (epoch=3)
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > A
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > B
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > C
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > Target
>> > > > > > > > > Assignment
>> > > > > > > > > > > > > > (epoch=3)
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > A -
>> > > > > > > > > > > > > partitions=[foo-0]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > B -
>> > > > > > > > > > > > > partitions=[foo-2]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > C -
>> > > > > > > > > > > > > partitions=[foo-1]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > Member
>> > > > > > > > > Assignment
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > A -
>> > > > > > > epoch=2,
>> > > > > > > > > > > > > > > >> partitions=[foo-0,
>> > > > > > > > > > > > > > > >> > > > > > foo-1]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > B -
>> > > > > > > epoch=3,
>> > > > > > > > > > > > > > > >> partitions=[foo-2]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> -
>> > > C -
>> > > > > > > epoch=3,
>> > > > > > > > > > > > > > > >> partitions=[foo-1]
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> When C
>> > > > > > > heartbeats,
>> > > > > > > > > it
>> > > > > > > > > > > > > > transitions
>> > > > > > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > epoch 3
>> > > > > > > > > > > > > > > >> > > > > > but
>> > > > > > > > > > > > > > > >> > > > > > > > > cannot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> get
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > foo-1
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> yet.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > Here,it's
>> > > > > > > mentioned
>> > > > > > > > > > that
>> > > > > > > > > > > > > > member C
>> > > > > > > > > > > > > > > >> can't
>> > > > > > > > > > > > > > > >> > > > get
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > foo-1
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > partition
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > yet,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> based
>> > > on the
>> > > > > > > > > > description
>> > > > > > > > > > > > > > above,
>> > > > > > > > > > > > > > > >> it seems
>> > > > > > > > > > > > > > > >> > > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > already has
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Do you
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> would be
>> > > > > better
>> > > > > > > to
>> > > > > > > > > > remove
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> > > > populate it
>> > > > > > > > > > > > > > > >> > > > > > only
>> > > > > > > > > > > > > > > >> > > > > > > > > when it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > actually
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> gets
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > I
>> see
>> > > this
>> > > > > in a
>> > > > > > > lot
>> > > > > > > > > of
>> > > > > > > > > > > > other
>> > > > > > > > > > > > > > > >> places, so
>> > > > > > > > > > > > > > > >> > > > have
>> > > > > > > > > > > > > > > >> > > > > > I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> understood it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> incorrectly
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > Regarding
>> > > > > > > connect ,
>> > > > > > > > > it
>> > > > > > > > > > > > might
>> > > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> out of
>> > > > > > > > > > > > > > > >> > > > scope
>> > > > > > > > > > > > > > > >> > > > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > discussion,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > from
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> what I
>> > > > > > > understood it
>> > > > > > > > > > would
>> > > > > > > > > > > > > > > >> probably be
>> > > > > > > > > > > > > > > >> > > > > > running in
>> > > > > > > > > > > > > > > >> > > > > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> assignor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > mode
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> even on
>> > > the
>> > > > > new
>> > > > > > > > > > rebalance
>> > > > > > > > > > > > > > > >> protocol as it
>> > > > > > > > > > > > > > > >> > > > has
>> > > > > > > > > > > > > > > >> > > > > > its
>> > > > > > > > > > > > > > > >> > > > > > > > > own
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Custom
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > Assignors(Eager
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > and
>> > > > > > > > > > > > IncrementalCooperative).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> Thanks!
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> Sagar.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > On
>> Fri,
>> > > Jul
>> > > > > 15,
>> > > > > > > 2022
>> > > > > > > > > > at
>> > > > > > > > > > > > 5:00
>> > > > > > > > > > > > > > PM
>> > > > > > > > > > > > > > > >> David
>> > > > > > > > > > > > > > > >> > > > Jacot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > > > > > > <djacot@confluent.io.invalid
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> Thanks
>> > > > > > > Hector! Our
>> > > > > > > > > > goal
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> move
>> > > > > > > > > > > > > > > >> > > > forward
>> > > > > > > > > > > > > > > >> > > > > > with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > specialized API
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > instead of
>> > > > > > > relying
>> > > > > > > > > > on
>> > > > > > > > > > > > one
>> > > > > > > > > > > > > > > >> generic API.
>> > > > > > > > > > > > > > > >> > > > For
>> > > > > > > > > > > > > > > >> > > > > > > > > Connect, we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can apply
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> exact
>> > > same
>> > > > > > > pattern
>> > > > > > > > > > and
>> > > > > > > > > > > > > > > >> reuse/share the
>> > > > > > > > > > > > > > > >> > > > core
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > implementation on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> server
>> > > > > side.
>> > > > > > > For
>> > > > > > > > > the
>> > > > > > > > > > > > > schema
>> > > > > > > > > > > > > > > >> registry, I
>> > > > > > > > > > > > > > > >> > > > > > think
>> > > > > > > > > > > > > > > >> > > > > > > > > that we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consider
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > having a
>> > > > > > > tailored
>> > > > > > > > > > API to
>> > > > > > > > > > > > > do
>> > > > > > > > > > > > > > > >> simple
>> > > > > > > > > > > > > > > >> > > > > > > > > membership/leader
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > election.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> Best,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> On
>> > > Fri,
>> > > > > Jul
>> > > > > > > 15,
>> > > > > > > > > > 2022 at
>> > > > > > > > > > > > > > 10:22
>> > > > > > > > > > > > > > > >> AM Ismael
>> > > > > > > > > > > > > > > >> > > > > > Juma <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ismael@juma.me.uk
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > Three
>> > > > > quick
>> > > > > > > > > > comments:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > 1.
>> > > > > Regarding
>> > > > > > > > > > > > > > > >> java.util.regex.Pattern
>> > > > > > > > > > > > > > > >> > > > vs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> > > > > > > com.google.re2j.Pattern,
>> > > > > > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > document the
>> > > > > > > > > > > > differences
>> > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> more
>> > > > > > > > > > > > > > > >> > > > detail
>> > > > > > > > > > > > > > > >> > > > > > before
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deciding
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > way
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> or
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > another.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > That
>> > > > > said,
>> > > > > > > if
>> > > > > > > > > > people
>> > > > > > > > > > > > > pass
>> > > > > > > > > > > > > > > >> > > > > > > > > java.util.regex.Pattern,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expect
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > their
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > semantics
>> > > > > > > to be
>> > > > > > > > > > > > honored.
>> > > > > > > > > > > > > > If
>> > > > > > > > > > > > > > > >> we are
>> > > > > > > > > > > > > > > >> > > > doing
>> > > > > > > > > > > > > > > >> > > > > > > > > something
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > different,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> then
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > should
>> > > > > > > consider
>> > > > > > > > > > adding
>> > > > > > > > > > > > > an
>> > > > > > > > > > > > > > > >> overload
>> > > > > > > > > > > > > > > >> > > > with
>> > > > > > > > > > > > > > > >> > > > > > our own
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Pattern
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > class
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > (I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > don't
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > think
>> > > > > we'd
>> > > > > > > want
>> > > > > > > > > to
>> > > > > > > > > > > > > expose
>> > > > > > > > > > > > > > > >> re2j's at
>> > > > > > > > > > > > > > > >> > > > this
>> > > > > > > > > > > > > > > >> > > > > > > > > point).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > 2.
>> > > > > Regarding
>> > > > > > > > > topic
>> > > > > > > > > > > > ids,
>> > > > > > > > > > > > > > any
>> > > > > > > > > > > > > > > >> major new
>> > > > > > > > > > > > > > > >> > > > > > protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > integrate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > it
>> > > and
>> > > > > > > should
>> > > > > > > > > > handle
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > topic
>> > > > > > > > > > > > > > > >> > > > > > recreation case
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > correctly.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That's
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > main
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > part we
>> > > > > > > need to
>> > > > > > > > > > > > handle.
>> > > > > > > > > > > > > I
>> > > > > > > > > > > > > > > >> agree with
>> > > > > > > > > > > > > > > >> > > > > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > that we'd
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > want to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > add
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > to
>> > > the
>> > > > > > > relevant
>> > > > > > > > > > > > > protocols
>> > > > > > > > > > > > > > > >> that don't
>> > > > > > > > > > > > > > > >> > > > > > have it
>> > > > > > > > > > > > > > > >> > > > > > > > > yet and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > that we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > can
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> probably
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > focus
>> > > > > on the
>> > > > > > > > > > internals
>> > > > > > > > > > > > > > versus
>> > > > > > > > > > > > > > > >> adding
>> > > > > > > > > > > > > > > >> > > > new
>> > > > > > > > > > > > > > > >> > > > > > APIs
>> > > > > > > > > > > > > > > >> > > > > > > > > to the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > (unless
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > we
>> > > find
>> > > > > that
>> > > > > > > > > > adding
>> > > > > > > > > > > > new
>> > > > > > > > > > > > > > APIs
>> > > > > > > > > > > > > > > >> is
>> > > > > > > > > > > > > > > >> > > > required
>> > > > > > > > > > > > > > > >> > > > > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> reasonable
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> semantics).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > 3.
>> > > I am
>> > > > > > > still
>> > > > > > > > > not
>> > > > > > > > > > sure
>> > > > > > > > > > > > > > about
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > storing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> configs.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It's
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > powerful for
>> > > > > > > > > > configs
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> > > > centralized
>> > > > > > > > > > > > > > > >> > > > > > in the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> metadata
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> various
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > reasons
>> > > > > > > > > > (auditability,
>> > > > > > > > > > > > > > > >> visibility,
>> > > > > > > > > > > > > > > >> > > > > > consistency,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> etc.).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> Similarly, I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > am
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > sure
>> > > > > about
>> > > > > > > > > > > > automatically
>> > > > > > > > > > > > > > > >> deleting
>> > > > > > > > > > > > > > > >> > > > > > configs in a
>> > > > > > > > > > > > > > > >> > > > > > > > > way
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cannot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > recovered. A
>> > > > > > > > > good
>> > > > > > > > > > > > > property
>> > > > > > > > > > > > > > > >> for modern
>> > > > > > > > > > > > > > > >> > > > > > systems
>> > > > > > > > > > > > > > > >> > > > > > > > > is to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > minimize
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> number
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > unrecoverable
>> > > > > > > > > data
>> > > > > > > > > > > > loss
>> > > > > > > > > > > > > > > >> scenarios.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > Ismael
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > On
>> > > Wed,
>> > > > > Jul
>> > > > > > > 13,
>> > > > > > > > > > 2022
>> > > > > > > > > > > > at
>> > > > > > > > > > > > > > 3:47
>> > > > > > > > > > > > > > > >> PM David
>> > > > > > > > > > > > > > > >> > > > > > Jacot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > > > > <djacot@confluent.io.invalid
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > Thanks
>> > > > > > > > > > Guozhang. My
>> > > > > > > > > > > > > > answers
>> > > > > > > > > > > > > > > >> are
>> > > > > > > > > > > > > > > >> > > > below:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > 1)
>> > > > > the
>> > > > > > > > > > migration
>> > > > > > > > > > > > > path,
>> > > > > > > > > > > > > > > >> especially
>> > > > > > > > > > > > > > > >> > > > > > the last
>> > > > > > > > > > > > > > > >> > > > > > > > > step
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > flag
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > to
>> > > > > > > enable
>> > > > > > > > > the
>> > > > > > > > > > new
>> > > > > > > > > > > > > > > >> protocol, in
>> > > > > > > > > > > > > > > >> > > > which
>> > > > > > > > > > > > > > > >> > > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> both
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > protocols /
>> > > > > > > > > > rpcs
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > > old
>> > > > > > > > > > > > > > > >> > > > protocols /
>> > > > > > > > > > > > > > > >> > > > > > rpcs
>> > > > > > > > > > > > > > > >> > > > > > > > > are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> same
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > group.
>> > > > > > > How
>> > > > > > > > > the
>> > > > > > > > > > > > > > > >> coordinator could
>> > > > > > > > > > > > > > > >> > > > > > "mimic"
>> > > > > > > > > > > > > > > >> > > > > > > > > the old
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> using
>> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > new
>> > > > > > > protocol
>> > > > > > > > > > is
>> > > > > > > > > > > > > > something
>> > > > > > > > > > > > > > > >> we
>> > > > > > > > > > > > > > > >> > > > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > present
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > Noted. I
>> > > > > > > just
>> > > > > > > > > > > > > published
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > >> new
>> > > > > > > > > > > > > > > >> > > > version
>> > > > > > > > > > > > > > > >> > > > > > of KIP
>> > > > > > > > > > > > > > > >> > > > > > > > > which
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > includes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> more
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > details
>> > > > > > > about
>> > > > > > > > > > this.
>> > > > > > > > > > > > > See
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > "Supporting
>> > > > > > > > > > > > > > > >> > > > > > > > > Online
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> Upgrade"
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > and
>> > > > > the
>> > > > > > > > > > > > > "Compatibility,
>> > > > > > > > > > > > > > > >> > > > Deprecation,
>> > > > > > > > > > > > > > > >> > > > > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > Migration
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Plan". I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > I
>> > > > > have to
>> > > > > > > > > think
>> > > > > > > > > > > > > through
>> > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > >> few cases
>> > > > > > > > > > > > > > > >> > > > > > now but
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overall idea
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > mechanism
>> > > > > > > > > > should be
>> > > > > > > > > > > > > > > >> understandable.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > 2)
>> > > > > the
>> > > > > > > usage
>> > > > > > > > > > of
>> > > > > > > > > > > > > topic
>> > > > > > > > > > > > > > > >> ids. So
>> > > > > > > > > > > > > > > >> > > > far as
>> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > as
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > part of
>> > > > > > > RPCs
>> > > > > > > > > > and
>> > > > > > > > > > > > > admin
>> > > > > > > > > > > > > > > >> client,
>> > > > > > > > > > > > > > > >> > > > but
>> > > > > > > > > > > > > > > >> > > > > > they
>> > > > > > > > > > > > > > > >> > > > > > > > > are not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> public
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > APIs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > to
>> > > > > > > consumers
>> > > > > > > > > > yet.
>> > > > > > > > > > > > I
>> > > > > > > > > > > > > > think
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > question is,
>> > > > > > > > > > > > > > > >> > > > > > > > > first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > client
>> > > > > > > to be
>> > > > > > > > > > > > > > maintaining
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > names
>> > > > > > > > > > > > > > > >> > > > > > -> ids
>> > > > > > > > > > > > > > > >> > > > > > > > > mapping
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > leverage
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > on
>> > > > > all
>> > > > > > > the
>> > > > > > > > > > > > augmented
>> > > > > > > > > > > > > > > >> existing
>> > > > > > > > > > > > > > > >> > > > RPCs
>> > > > > > > > > > > > > > > >> > > > > > and the
>> > > > > > > > > > > > > > > >> > > > > > > > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> ids;
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > secondly,
>> > > > > > > > > > should
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > > ever
>> > > > > > > > > > > > > > > >> consider
>> > > > > > > > > > > > > > > >> > > > > > exposing
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > public
>> > > > > > > APIs
>> > > > > > > > > as
>> > > > > > > > > > > > well
>> > > > > > > > > > > > > > (both
>> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > rebalance
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > listener for
>> > > > > > > > > > cases
>> > > > > > > > > > > > > > like
>> > > > > > > > > > > > > > > >> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > deletion-and-recreation).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > a)
>> > > > > > > Assuming
>> > > > > > > > > > that we
>> > > > > > > > > > > > > > would
>> > > > > > > > > > > > > > > >> include
>> > > > > > > > > > > > > > > >> > > > > > converting
>> > > > > > > > > > > > > > > >> > > > > > > > > all
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > offsets
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> related
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > RPCs
>> > > > > to
>> > > > > > > using
>> > > > > > > > > > topic
>> > > > > > > > > > > > > ids
>> > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > > >> > > > KIP,
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> able
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > fully
>> > > > > > > operate
>> > > > > > > > > > with
>> > > > > > > > > > > > > topic
>> > > > > > > > > > > > > > > >> ids. That
>> > > > > > > > > > > > > > > >> > > > > > being
>> > > > > > > > > > > > > > > >> > > > > > > > > said, it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > still has
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> provide
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > the
>> > > > > topics
>> > > > > > > > > > names in
>> > > > > > > > > > > > > > various
>> > > > > > > > > > > > > > > >> APIs so
>> > > > > > > > > > > > > > > >> > > > > > having a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> mapping
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > seems
>> > > > > > > > > > inevitable to
>> > > > > > > > > > > > > me.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > b) I
>> > > > > don't
>> > > > > > > > > have
>> > > > > > > > > > a
>> > > > > > > > > > > > > strong
>> > > > > > > > > > > > > > > >> opinion on
>> > > > > > > > > > > > > > > >> > > > > > this.
>> > > > > > > > > > > > > > > >> > > > > > > > > Here I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wonder if
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > goes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > beyond the
>> > > > > > > > > > scope of
>> > > > > > > > > > > > > this
>> > > > > > > > > > > > > > > >> KIP. I
>> > > > > > > > > > > > > > > >> > > > would
>> > > > > > > > > > > > > > > >> > > > > > rather
>> > > > > > > > > > > > > > > >> > > > > > > > > focus
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> internals
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > here
>> > > > > and
>> > > > > > > we
>> > > > > > > > > can
>> > > > > > > > > > > > > consider
>> > > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > > >> > > > > > separately if
>> > > > > > > > > > > > > > > >> > > > > > > > > we see
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > value in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> doing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > Coming
>> > > > > > > back to
>> > > > > > > > > > > > > Ismael's
>> > > > > > > > > > > > > > > >> point about
>> > > > > > > > > > > > > > > >> > > > > > using
>> > > > > > > > > > > > > > > >> > > > > > > > > topic ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> ConsumerGroupHeartbeatRequest, I
>> > > > > > > > > > > > > > > >> > > > think
>> > > > > > > > > > > > > > > >> > > > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > there
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> advantage
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > favour of
>> > > > > > > it.
>> > > > > > > > > > The
>> > > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> will
>> > > > > > > > > > > > > > > >> > > > have
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opportunity
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > the
>> > > > > topics
>> > > > > > > > > > exists
>> > > > > > > > > > > > > before
>> > > > > > > > > > > > > > > >> passing
>> > > > > > > > > > > > > > > >> > > > them
>> > > > > > > > > > > > > > > >> > > > > > into
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > rebalance
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > protocol.
>> > > > > > > > > > Obviously,
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > will
>> > > > > > > > > > > > > > > >> > > > > > > > > also
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> notice
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > does
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > not
>> > > > > really
>> > > > > > > > > have
>> > > > > > > > > > a
>> > > > > > > > > > > > way
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> reject an
>> > > > > > > > > > > > > > > >> > > > > > invalid
>> > > > > > > > > > > > > > > >> > > > > > > > > topic in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> response.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > I'm
>> > > > > > > agreeing
>> > > > > > > > > > with
>> > > > > > > > > > > > > > David
>> > > > > > > > > > > > > > > >> on all
>> > > > > > > > > > > > > > > >> > > > other
>> > > > > > > > > > > > > > > >> > > > > > minor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > `subscribe(Pattern)`
>> > > > > > > > > > > > > > > >> question:
>> > > > > > > > > > > > > > > >> > > > > > personally
>> > > > > > > > > > > > > > > >> > > > > > > > > I think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> necessary
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > deprecate
>> > > > > > > > > the
>> > > > > > > > > > > > > > subscribe
>> > > > > > > > > > > > > > > >> API with
>> > > > > > > > > > > > > > > >> > > > > > Pattern,
>> > > > > > > > > > > > > > > >> > > > > > > > > but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > Pattern
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > while
>> > > > > > > just
>> > > > > > > > > > > > > documenting
>> > > > > > > > > > > > > > > >> that our
>> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
>> by
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > server.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > Since
>> > > > > > > the
>> > > > > > > > > > > > > incompatible
>> > > > > > > > > > > > > > > >> case is a
>> > > > > > > > > > > > > > > >> > > > > > very rare
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > overloaded
>> > > > > > > > > > > > `String`
>> > > > > > > > > > > > > > based
>> > > > > > > > > > > > > > > >> > > > > > subscription may
>> > > > > > > > > > > > > > > >> > > > > > > > > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> vulnerable
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > various
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > invalid
>> > > > > > > > > > regexes.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > That
>> > > > > could
>> > > > > > > > > > work. I
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> look at
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > differences
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > between
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > two
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > engines to
>> > > > > > > > > > better
>> > > > > > > > > > > > > > > >> understand the
>> > > > > > > > > > > > > > > >> > > > > > potential
>> > > > > > > > > > > > > > > >> > > > > > > > > issues.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> My
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > understanding
>> > > > > is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > that
>> > > > > would
>> > > > > > > > > work
>> > > > > > > > > > for
>> > > > > > > > > > > > > all
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> basic
>> > > > > > > > > > > > > > > >> > > > > > regular
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expressions. The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > differences
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > between
>> > > > > > > the
>> > > > > > > > > two
>> > > > > > > > > > are
>> > > > > > > > > > > > > > mainly
>> > > > > > > > > > > > > > > >> about
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > various
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> character
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> classes.
>> I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > wonder
>> > > > > > > what
>> > > > > > > > > > other
>> > > > > > > > > > > > > people
>> > > > > > > > > > > > > > > >> think
>> > > > > > > > > > > > > > > >> > > > about
>> > > > > > > > > > > > > > > >> > > > > > this.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > Best,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > On
>> > > > > Tue,
>> > > > > > > Jul
>> > > > > > > > > 12,
>> > > > > > > > > > 2022
>> > > > > > > > > > > > > at
>> > > > > > > > > > > > > > > >> 11:28 PM
>> > > > > > > > > > > > > > > >> > > > > > Guozhang
>> > > > > > > > > > > > > > > >> > > > > > > > > Wang <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> > > > > wangguoz@gmail.com
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > Thanks
>> > > > > > > > > David!
>> > > > > > > > > > I
>> > > > > > > > > > > > > think
>> > > > > > > > > > > > > > on
>> > > > > > > > > > > > > > > >> the high
>> > > > > > > > > > > > > > > >> > > > > > level
>> > > > > > > > > > > > > > > >> > > > > > > > > there are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > two meta
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> points we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> need
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > to
>> > > > > > > > > concretize
>> > > > > > > > > > a
>> > > > > > > > > > > > bit
>> > > > > > > > > > > > > > more:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > 1)
>> > > > > the
>> > > > > > > > > > migration
>> > > > > > > > > > > > > path,
>> > > > > > > > > > > > > > > >> especially
>> > > > > > > > > > > > > > > >> > > > > > the last
>> > > > > > > > > > > > > > > >> > > > > > > > > step
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > flag
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > to
>> > > > > > > enable
>> > > > > > > > > the
>> > > > > > > > > > new
>> > > > > > > > > > > > > > > >> protocol, in
>> > > > > > > > > > > > > > > >> > > > which
>> > > > > > > > > > > > > > > >> > > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> both
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > protocols /
>> > > > > > > > > > rpcs
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > > old
>> > > > > > > > > > > > > > > >> > > > protocols /
>> > > > > > > > > > > > > > > >> > > > > > rpcs
>> > > > > > > > > > > > > > > >> > > > > > > > > are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> same
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > group.
>> > > > > > > How
>> > > > > > > > > the
>> > > > > > > > > > > > > > > >> coordinator could
>> > > > > > > > > > > > > > > >> > > > > > "mimic"
>> > > > > > > > > > > > > > > >> > > > > > > > > the old
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> using
>> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > new
>> > > > > > > protocol
>> > > > > > > > > > is
>> > > > > > > > > > > > > > something
>> > > > > > > > > > > > > > > >> we
>> > > > > > > > > > > > > > > >> > > > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > present
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > 2)
>> > > > > the
>> > > > > > > usage
>> > > > > > > > > > of
>> > > > > > > > > > > > > topic
>> > > > > > > > > > > > > > > >> ids. So
>> > > > > > > > > > > > > > > >> > > > far as
>> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > as
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > part of
>> > > > > > > RPCs
>> > > > > > > > > > and
>> > > > > > > > > > > > > admin
>> > > > > > > > > > > > > > > >> client,
>> > > > > > > > > > > > > > > >> > > > but
>> > > > > > > > > > > > > > > >> > > > > > they
>> > > > > > > > > > > > > > > >> > > > > > > > > are not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> public
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > APIs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > to
>> > > > > > > consumers
>> > > > > > > > > > yet.
>> > > > > > > > > > > > I
>> > > > > > > > > > > > > > think
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > question is,
>> > > > > > > > > > > > > > > >> > > > > > > > > first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > client
>> > > > > > > to be
>> > > > > > > > > > > > > > maintaining
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > names
>> > > > > > > > > > > > > > > >> > > > > > -> ids
>> > > > > > > > > > > > > > > >> > > > > > > > > mapping
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > leverage
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > on
>> > > > > all
>> > > > > > > the
>> > > > > > > > > > > > augmented
>> > > > > > > > > > > > > > > >> existing
>> > > > > > > > > > > > > > > >> > > > RPCs
>> > > > > > > > > > > > > > > >> > > > > > and the
>> > > > > > > > > > > > > > > >> > > > > > > > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> ids;
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > secondly,
>> > > > > > > > > > should
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > > ever
>> > > > > > > > > > > > > > > >> consider
>> > > > > > > > > > > > > > > >> > > > > > exposing
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > public
>> > > > > > > APIs
>> > > > > > > > > as
>> > > > > > > > > > > > well
>> > > > > > > > > > > > > > (both
>> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > rebalance
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > listener for
>> > > > > > > > > > cases
>> > > > > > > > > > > > > > like
>> > > > > > > > > > > > > > > >> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > deletion-and-recreation).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > I'm
>> > > > > > > agreeing
>> > > > > > > > > > with
>> > > > > > > > > > > > > > David
>> > > > > > > > > > > > > > > >> on all
>> > > > > > > > > > > > > > > >> > > > other
>> > > > > > > > > > > > > > > >> > > > > > minor
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > `subscribe(Pattern)`
>> > > > > > > > > > > > > > > >> question:
>> > > > > > > > > > > > > > > >> > > > > > personally
>> > > > > > > > > > > > > > > >> > > > > > > > > I think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> necessary
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > deprecate
>> > > > > > > > > the
>> > > > > > > > > > > > > > subscribe
>> > > > > > > > > > > > > > > >> API with
>> > > > > > > > > > > > > > > >> > > > > > Pattern,
>> > > > > > > > > > > > > > > >> > > > > > > > > but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > Pattern
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > while
>> > > > > > > just
>> > > > > > > > > > > > > documenting
>> > > > > > > > > > > > > > > >> that our
>> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
>> by
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > server.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > Since
>> > > > > > > the
>> > > > > > > > > > > > > incompatible
>> > > > > > > > > > > > > > > >> case is a
>> > > > > > > > > > > > > > > >> > > > > > very rare
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > overloaded
>> > > > > > > > > > > > `String`
>> > > > > > > > > > > > > > based
>> > > > > > > > > > > > > > > >> > > > > > subscription may
>> > > > > > > > > > > > > > > >> > > > > > > > > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> vulnerable
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > various
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > invalid
>> > > > > > > > > > regexes.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > Guozhang
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > On
>> > > > > Tue,
>> > > > > > > Jul
>> > > > > > > > > > 12,
>> > > > > > > > > > > > 2022
>> > > > > > > > > > > > > > at
>> > > > > > > > > > > > > > > >> 5:23 AM
>> > > > > > > > > > > > > > > >> > > > > > David Jacot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > > > > > > > > > > > <djacot@confluent.io.invalid
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > Hi
>> > > > > > > Ismael,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > Thanks for
>> > > > > > > > > > your
>> > > > > > > > > > > > > > > >> feedback. Let
>> > > > > > > > > > > > > > > >> > > > me
>> > > > > > > > > > > > > > > >> > > > > > answer
>> > > > > > > > > > > > > > > >> > > > > > > > > your
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > questions
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > inline.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 1. I
>> > > > > > > > > think
>> > > > > > > > > > > > it's
>> > > > > > > > > > > > > > > >> premature to
>> > > > > > > > > > > > > > > >> > > > > > talk about
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> target
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > versions
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > deprecation
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > removal
>> > > > > > > > > > of the
>> > > > > > > > > > > > > > > >> existing group
>> > > > > > > > > > > > > > > >> > > > > > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Unlike
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > KRaft,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > affects a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > core
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > client
>> > > > > > > > > > > > protocol
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> hence
>> > > > > > > > > > > > > > > >> > > > > > > > > deprecation/removal
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > heavily
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > dependent
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > how
>> > > > > > > > > > quickly
>> > > > > > > > > > > > > > > >> applications
>> > > > > > > > > > > > > > > >> > > > migrate
>> > > > > > > > > > > > > > > >> > > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > the new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > That
>> > > > > > > makes
>> > > > > > > > > > > > sense.
>> > > > > > > > > > > > > I
>> > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > >> > > > remove it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 2.
>> > > > > > > The
>> > > > > > > > > KIP
>> > > > > > > > > > > > says
>> > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > >> intend to
>> > > > > > > > > > > > > > > >> > > > > > release
>> > > > > > > > > > > > > > > >> > > > > > > > > this in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 4.x, but
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> wasn't
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> made
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > clear
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > why. If
>> > > > > > > > > we
>> > > > > > > > > > > > added
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > >> as a
>> > > > > > > > > > > > > > > >> > > > way to
>> > > > > > > > > > > > > > > >> > > > > > > > > estimate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we'd
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> deprecate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > remove
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > the
>> > > > > > > > > group
>> > > > > > > > > > > > > > protocol, I
>> > > > > > > > > > > > > > > >> also
>> > > > > > > > > > > > > > > >> > > > > > suggest
>> > > > > > > > > > > > > > > >> > > > > > > > > removing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this part.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Let me
>> > > > > > > > > > explain
>> > > > > > > > > > > > my
>> > > > > > > > > > > > > > > >> reasoning. As
>> > > > > > > > > > > > > > > >> > > > > > > > > explained, I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> plan
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rewrite
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > coordinator
>> > > > > > > > > > in
>> > > > > > > > > > > > > Java
>> > > > > > > > > > > > > > > >> while we
>> > > > > > > > > > > > > > > >> > > > > > implement
>> > > > > > > > > > > > > > > >> > > > > > > > > the new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> This
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > that
>> > > > > > > the
>> > > > > > > > > > > > internals
>> > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > >> be
>> > > > > > > > > > > > > > > >> > > > slightly
>> > > > > > > > > > > > > > > >> > > > > > > > > different
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> threading
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > model).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > Therefore, I
>> > > > > > > > > > > > > wanted
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> tighten
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > switch from
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the old
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > coordinator
>> > > > > > > > > > to
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > new
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > to a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > major
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> release.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > alternative
>> > > > > > > > > > > > would
>> > > > > > > > > > > > > > be to
>> > > > > > > > > > > > > > > >> use a
>> > > > > > > > > > > > > > > >> > > > flag
>> > > > > > > > > > > > > > > >> > > > > > to do
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instead
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > relying
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > on the
>> > > > > > > > > > software
>> > > > > > > > > > > > > > upgrade.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 3.
>> > > > > > > We
>> > > > > > > > > > need to
>> > > > > > > > > > > > > > flesh
>> > > > > > > > > > > > > > > >> out the
>> > > > > > > > > > > > > > > >> > > > > > details of
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > migration
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> story.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> sounds
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > like
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > we're
>> > > > > > > > > > saying
>> > > > > > > > > > > > we
>> > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > >> support
>> > > > > > > > > > > > > > > >> > > > > > online
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> migrations.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> correct?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> We
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > explain
>> > > > > > > > > > this
>> > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> detail. It
>> > > > > > > > > > > > > > > >> > > > could
>> > > > > > > > > > > > > > > >> > > > > > also
>> > > > > > > > > > > > > > > >> > > > > > > > > be done
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> separate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > KIP,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> if
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > it's
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > easier.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Yes,
>> > > > > > > we
>> > > > > > > > > will
>> > > > > > > > > > > > > support
>> > > > > > > > > > > > > > > >> online
>> > > > > > > > > > > > > > > >> > > > > > migrations
>> > > > > > > > > > > > > > > >> > > > > > > > > for the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > a
>> > > > > > > group
>> > > > > > > > > > using
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > old
>> > > > > > > > > > > > > > > >> protocol
>> > > > > > > > > > > > > > > >> > > > > > will be
>> > > > > > > > > > > > > > > >> > > > > > > > > able to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Let me
>> > > > > > > > > > briefly
>> > > > > > > > > > > > > > explain
>> > > > > > > > > > > > > > > >> how that
>> > > > > > > > > > > > > > > >> > > > > > will work
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> though.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > It is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> basically
>> > > a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > four
>> > > > > > > step
>> > > > > > > > > > > > process:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > 1. The
>> > > > > > > > > > cluster
>> > > > > > > > > > > > > must
>> > > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> > > > upgraded or
>> > > > > > > > > > > > > > > >> > > > > > > > > rolled to a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > software
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> supporting
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > new
>> > > > > > > group
>> > > > > > > > > > > > > > coordinator.
>> > > > > > > > > > > > > > > >> Both the
>> > > > > > > > > > > > > > > >> > > > > > old and
>> > > > > > > > > > > > > > > >> > > > > > > > > the new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > will
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > support
>> > > > > > > > > the
>> > > > > > > > > > old
>> > > > > > > > > > > > > > > >> protocol and
>> > > > > > > > > > > > > > > >> > > > rely
>> > > > > > > > > > > > > > > >> > > > > > on the
>> > > > > > > > > > > > > > > >> > > > > > > > > same
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > persisted
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> metadata
>> > > so
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > they
>> > > > > > > can
>> > > > > > > > > > work
>> > > > > > > > > > > > > > together.
>> > > > > > > > > > > > > > > >> This
>> > > > > > > > > > > > > > > >> > > > point
>> > > > > > > > > > > > > > > >> > > > > > is an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> offline
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> migration.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > We
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> cannot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > do
>> > > > > > > this
>> > > > > > > > > one
>> > > > > > > > > > live
>> > > > > > > > > > > > > > > >> because it
>> > > > > > > > > > > > > > > >> > > > would
>> > > > > > > > > > > > > > > >> > > > > > require
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > shutting down
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> current
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > coordinator
>> > > > > > > > > > and
>> > > > > > > > > > > > > > > >> starting up the
>> > > > > > > > > > > > > > > >> > > > > > new one
>> > > > > > > > > > > > > > > >> > > > > > > > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cause
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > unavailabilities.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > 2. The
>> > > > > > > > > > cluster's
>> > > > > > > > > > > > > > > >> metadata
>> > > > > > > > > > > > > > > >> > > > > > version/IBP
>> > > > > > > > > > > > > > > >> > > > > > > > > must be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> X
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> order
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > to
>> > > > > > > enable
>> > > > > > > > > > the
>> > > > > > > > > > > > new
>> > > > > > > > > > > > > > > >> protocol.
>> > > > > > > > > > > > > > > >> > > > This
>> > > > > > > > > > > > > > > >> > > > > > cannot
>> > > > > > > > > > > > > > > >> > > > > > > > > be done
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 1)
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > terminated
>> > > > > > > > > > > > because
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> old
>> > > > > > > > > > > > > > > >> > > > > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > doesn't
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > support
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > protocol.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > 3. The
>> > > > > > > > > > consumers
>> > > > > > > > > > > > > > must be
>> > > > > > > > > > > > > > > >> > > > upgraded
>> > > > > > > > > > > > > > > >> > > > > > to a
>> > > > > > > > > > > > > > > >> > > > > > > > > version
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > supporting
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> online
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > migration
>> > > > > > > > > > (must
>> > > > > > > > > > > > > have
>> > > > > > > > > > > > > > > >> KIP-792).
>> > > > > > > > > > > > > > > >> > > > If
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > already
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> there.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > Nothing
>> > > > > > > > > > must be
>> > > > > > > > > > > > > > done at
>> > > > > > > > > > > > > > > >> this
>> > > > > > > > > > > > > > > >> > > > point.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > 4. The
>> > > > > > > > > > consumers
>> > > > > > > > > > > > > > must
>> > > > > > > > > > > > > > > >> be rolled
>> > > > > > > > > > > > > > > >> > > > > > with the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> turned
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > on.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > consumer
>> > > > > > > > > > group
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > > > >> automatically
>> > > > > > > > > > > > > > > >> > > > > > > > > converted when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > using
>> > > > > > > the
>> > > > > > > > > > new
>> > > > > > > > > > > > > > protocol
>> > > > > > > > > > > > > > > >> joins
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > group.
>> > > > > > > > > > > > > > > >> > > > > > > > > While
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > members
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> using the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > old
>> > > > > > > > > > protocol are
>> > > > > > > > > > > > > > being
>> > > > > > > > > > > > > > > >> > > > upgraded,
>> > > > > > > > > > > > > > > >> > > > > > the old
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> proxied
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > into
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > new
>> > > > > > > one.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Let me
>> > > > > > > > > > clarify
>> > > > > > > > > > > > all
>> > > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > >> this in
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > KIP.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 4.
>> > > > > > > I am
>> > > > > > > > > > happy
>> > > > > > > > > > > > > > that we
>> > > > > > > > > > > > > > > >> are
>> > > > > > > > > > > > > > > >> > > > > > pushing the
>> > > > > > > > > > > > > > > >> > > > > > > > > pattern
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> subscriptions
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > server,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > but
>> > > > > > > it
>> > > > > > > > > > seems
>> > > > > > > > > > > > > like
>> > > > > > > > > > > > > > > >> there
>> > > > > > > > > > > > > > > >> > > > could be
>> > > > > > > > > > > > > > > >> > > > > > some
>> > > > > > > > > > > > > > > >> > > > > > > > > tricky
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> compatibility
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > issues.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > Will
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > have a
>> > > > > > > > > > > > mechanism
>> > > > > > > > > > > > > > for
>> > > > > > > > > > > > > > > >> users to
>> > > > > > > > > > > > > > > >> > > > > > detect
>> > > > > > > > > > > > > > > >> > > > > > > > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > update
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> their
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > regex
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > before
>> > > > > > > > > > > > switching
>> > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> the new
>> > > > > > > > > > > > > > > >> > > > > > protocol?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > I
>> > > > > > > think
>> > > > > > > > > > that I
>> > > > > > > > > > > > am
>> > > > > > > > > > > > > a
>> > > > > > > > > > > > > > bit
>> > > > > > > > > > > > > > > >> more
>> > > > > > > > > > > > > > > >> > > > > > optimistic
>> > > > > > > > > > > > > > > >> > > > > > > > > than
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> you
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> point. I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > believe
>> > > > > > > > > > that the
>> > > > > > > > > > > > > > > >> majority of
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > cases
>> > > > > > > > > > > > > > > >> > > > > > > > > are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> simple
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > regexes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > which
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > work
>> > > > > > > with
>> > > > > > > > > > the
>> > > > > > > > > > > > new
>> > > > > > > > > > > > > > > >> engine. The
>> > > > > > > > > > > > > > > >> > > > > > > > > coordinator will
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > verify
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > regex
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> anyway
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > and
>> > > > > > > reject
>> > > > > > > > > > the
>> > > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> if the
>> > > > > > > > > > > > > > > >> > > > > > regex is
>> > > > > > > > > > > > > > > >> > > > > > > > > not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> valid.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Coming
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > back
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > migration
>> > > > > > > > > > path,
>> > > > > > > > > > > > in
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> worst
>> > > > > > > > > > > > > > > >> > > > case,
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > joining
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > the
>> > > > > > > group
>> > > > > > > > > > will
>> > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> rejected.
>> > > > > > > > > > > > > > > >> > > > This
>> > > > > > > > > > > > > > > >> > > > > > should
>> > > > > > > > > > > > > > > >> > > > > > > > > be used
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> last
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > defence, I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > would
>> > > > > > > say.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > One
>> > > > > > > way
>> > > > > > > > > for
>> > > > > > > > > > > > > > customers to
>> > > > > > > > > > > > > > > >> > > > validate
>> > > > > > > > > > > > > > > >> > > > > > their
>> > > > > > > > > > > > > > > >> > > > > > > > > regex
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> upgrading
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> their
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > prod
>> > > > > > > would
>> > > > > > > > > > be to
>> > > > > > > > > > > > > > test
>> > > > > > > > > > > > > > > >> them with
>> > > > > > > > > > > > > > > >> > > > > > another
>> > > > > > > > > > > > > > > >> > > > > > > > > group.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> For
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instance,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > could
>> > > > > > > be
>> > > > > > > > > > done
>> > > > > > > > > > > > in a
>> > > > > > > > > > > > > > > >> pre-prod
>> > > > > > > > > > > > > > > >> > > > > > environment.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Another
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > way
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> would be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > extend the
>> > > > > > > > > > > > > > > >> consumer-group tool
>> > > > > > > > > > > > > > > >> > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > provide a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> regex
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
>> validation
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > mechanism.
>> > > > > > > > > > Would
>> > > > > > > > > > > > > > this be
>> > > > > > > > > > > > > > > >> > > > enough in
>> > > > > > > > > > > > > > > >> > > > > > your
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opinion?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 5.
>> > > > > > > > > > Related to
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > last
>> > > > > > > > > > > > > > > >> > > > question,
>> > > > > > > > > > > > > > > >> > > > > > will
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Java
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> allow
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > users
>> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > stick
>> > > > > > > > > > with the
>> > > > > > > > > > > > > > > >> current regex
>> > > > > > > > > > > > > > > >> > > > > > engine for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > compatibility
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> reasons?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> For
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > example,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > it
>> > > > > > > may
>> > > > > > > > > be
>> > > > > > > > > > > > handy
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> keep using
>> > > > > > > > > > > > > > > >> > > > > > client
>> > > > > > > > > > > > > > > >> > > > > > > > > based
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > regex at
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > keep
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > migrations
>> > > > > > > > > > > > > simple
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > >> then
>> > > > > > > > > > > > > > > >> > > > > > migrate to
>> > > > > > > > > > > > > > > >> > > > > > > > > server
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > based
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regexes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > as a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > second
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > step.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > I
>> > > > > > > > > understand
>> > > > > > > > > > > > your
>> > > > > > > > > > > > > > point
>> > > > > > > > > > > > > > > >> but I
>> > > > > > > > > > > > > > > >> > > > am
>> > > > > > > > > > > > > > > >> > > > > > > > > concerned that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > allow
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> users
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > to
>> > > > > > > > > actually
>> > > > > > > > > > stay
>> > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> this mode.
>> > > > > > > > > > > > > > > >> > > > That
>> > > > > > > > > > > > > > > >> > > > > > > > > would go
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > against our
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> goal
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > simplifying
>> > > > > > > > > > the
>> > > > > > > > > > > > > > client
>> > > > > > > > > > > > > > > >> because
>> > > > > > > > > > > > > > > >> > > > we
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > have to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > continue
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> monitoring
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > the
>> > > > > > > > > > metadata on
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> client
>> > > > > > > > > > > > > > > >> > > > side. I
>> > > > > > > > > > > > > > > >> > > > > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > rather
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not do
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 6.
>> > > > > > > When
>> > > > > > > > > > we say
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > >> the group
>> > > > > > > > > > > > > > > >> > > > > > > > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> will
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > responsible
>> > > > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > storing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > the
>> > > > > > > > > > > > > configurations
>> > > > > > > > > > > > > > > >> and that
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> configurations
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> deleted
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > group is
>> > > > > > > > > > > > > deleted.
>> > > > > > > > > > > > > > > >> Will a
>> > > > > > > > > > > > > > > >> > > > > > transition to
>> > > > > > > > > > > > > > > >> > > > > > > > > DEAD
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > trigger
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
>> deletion
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > configurations?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > That's
>> > > > > > > > > > right.
>> > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> > > > configurations
>> > > > > > > > > > > > > > > >> > > > > > will be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deleted
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> group is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > deleted.
>> > > > > > > > > > They go
>> > > > > > > > > > > > > > > >> together.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 7.
>> > > > > > > Will
>> > > > > > > > > > the
>> > > > > > > > > > > > > > choice to
>> > > > > > > > > > > > > > > >> store
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > configs in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > make
>> > > > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > harder
>> > > > > > > > > to
>> > > > > > > > > > list
>> > > > > > > > > > > > > all
>> > > > > > > > > > > > > > > >> cluster
>> > > > > > > > > > > > > > > >> > > > > > configs and
>> > > > > > > > > > > > > > > >> > > > > > > > > their
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > values?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > I
>> > > > > > > don't
>> > > > > > > > > > think
>> > > > > > > > > > > > so.
>> > > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> group
>> > > > > > > > > > > > > > > >> > > > > > > > > configurations are
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overrides
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> cluster
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > configs.
>> > > > > > > > > If
>> > > > > > > > > > you
>> > > > > > > > > > > > > > want to
>> > > > > > > > > > > > > > > >> know
>> > > > > > > > > > > > > > > >> > > > all
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > overrides
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > you
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > to
>> > > > > > > ask all
>> > > > > > > > > > the
>> > > > > > > > > > > > > group
>> > > > > > > > > > > > > > > >> > > > coordinators.
>> > > > > > > > > > > > > > > >> > > > > > You
>> > > > > > > > > > > > > > > >> > > > > > > > > cannot
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > rely on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> metadata
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> log
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > for
>> > > > > > > > > > instance.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 8.
>> > > > > > > How
>> > > > > > > > > > would
>> > > > > > > > > > > > > > someone
>> > > > > > > > > > > > > > > >> > > > configure a
>> > > > > > > > > > > > > > > >> > > > > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > before
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > starting
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > consumers?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > we
>> > > > > > > > > > considered
>> > > > > > > > > > > > > > > >> allowing the
>> > > > > > > > > > > > > > > >> > > > > > explicit
>> > > > > > > > > > > > > > > >> > > > > > > > > creation
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > groups?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > > > > > Alternatively,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > configs
>> > > > > > > > > > could
>> > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> decoupled
>> > > > > > > > > > > > > > > >> > > > from
>> > > > > > > > > > > > > > > >> > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > lifecycle.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Yes.
>> > > > > > > The
>> > > > > > > > > > group
>> > > > > > > > > > > > > will
>> > > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> > > > > > automatically
>> > > > > > > > > > > > > > > >> > > > > > > > > created in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> However,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > the
>> > > > > > > > > configs
>> > > > > > > > > > will
>> > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > >> lost after
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > retention
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > period of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > passes.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 9.
>> > > > > > > Will
>> > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > Consumer.subscribe
>> > > > > > > > > > > > > > > >> > > > > > method
>> > > > > > > > > > > > > > > >> > > > > > > > > for the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> client
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > still
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> take
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` of
>> > > > > > > > > > > > > > > >> > > > do
>> > > > > > > > > > > > > > > >> > > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > have to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > introduce
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> overload?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > That's a
>> > > > > > > > > > very
>> > > > > > > > > > > > > group
>> > > > > > > > > > > > > > > >> question. I
>> > > > > > > > > > > > > > > >> > > > > > forgot
>> > > > > > > > > > > > > > > >> > > > > > > > > about
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > As
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` is
>> > > > > > > > > > > > > > > >> > > > not
>> > > > > > > > > > > > > > > >> > > > > > fully
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> compatible
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> engine
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > we
>> > > > > > > plan to
>> > > > > > > > > > use,
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > > >> might be
>> > > > > > > > > > > > > > > >> > > > better
>> > > > > > > > > > > > > > > >> > > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > deprecate
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> use an
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > overload
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > which
>> > > > > > > > > takes
>> > > > > > > > > > a
>> > > > > > > > > > > > > > string.
>> > > > > > > > > > > > > > > >> We would
>> > > > > > > > > > > > > > > >> > > > > > rely on
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> server
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > validation.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > During the
>> > > > > > > > > > > > > > migration, I
>> > > > > > > > > > > > > > > >> think
>> > > > > > > > > > > > > > > >> > > > that
>> > > > > > > > > > > > > > > >> > > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > could
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> still
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > try to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> toString
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > regex
>> > > > > > > and
>> > > > > > > > > > use
>> > > > > > > > > > > > it.
>> > > > > > > > > > > > > > That
>> > > > > > > > > > > > > > > >> should
>> > > > > > > > > > > > > > > >> > > > > > work, I
>> > > > > > > > > > > > > > > >> > > > > > > > > think, in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> majority
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
>> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > cases.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 10.
>> > > > > > > I
>> > > > > > > > > > agree
>> > > > > > > > > > > > with
>> > > > > > > > > > > > > > > >> Justine
>> > > > > > > > > > > > > > > >> > > > that we
>> > > > > > > > > > > > > > > >> > > > > > > > > should be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > clearer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > about
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> reason
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > switch
>> > > > > > > > > to
>> > > > > > > > > > > > > > > >> > > > IBP/metadata.version
>> > > > > > > > > > > > > > > >> > > > > > from the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Maybe
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> mean
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > can
>> > > > > > > > > > switch the
>> > > > > > > > > > > > > > > >> default for
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > feature
>> > > > > > > > > > > > > > > >> > > > > > > > > flag
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > true
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> based
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > metadata.version
>> > > > > > > > > > > > > > once
>> > > > > > > > > > > > > > > >> we
>> > > > > > > > > > > > > > > >> > > > want to
>> > > > > > > > > > > > > > > >> > > > > > make
>> > > > > > > > > > > > > > > >> > > > > > > > > it the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > default.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > My
>> > > > > > > plan
>> > > > > > > > > was
>> > > > > > > > > > to
>> > > > > > > > > > > > use
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > >> feature
>> > > > > > > > > > > > > > > >> > > > > > flag
>> > > > > > > > > > > > > > > >> > > > > > > > > mainly
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > during the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > development
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > phase. I
>> > > > > > > > > > should
>> > > > > > > > > > > > > not
>> > > > > > > > > > > > > > have
>> > > > > > > > > > > > > > > >> > > > mentioned
>> > > > > > > > > > > > > > > >> > > > > > it, I
>> > > > > > > > > > > > > > > >> > > > > > > > > think,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > because
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > could
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> use
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > an
>> > > > > > > > > internal
>> > > > > > > > > > > > config
>> > > > > > > > > > > > > > for
>> > > > > > > > > > > > > > > >> it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 11.
>> > > > > > > Some
>> > > > > > > > > > of
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> protocol APIs
>> > > > > > > > > > > > > > > >> > > > > > don't
>> > > > > > > > > > > > > > > >> > > > > > > > > mention
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > required
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> ACLs, it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > would
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > good to
>> > > > > > > > > > add
>> > > > > > > > > > > > that
>> > > > > > > > > > > > > > for
>> > > > > > > > > > > > > > > >> > > > consistency.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Noted.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 12.
>> > > > > > > It
>> > > > > > > > > is
>> > > > > > > > > > a
>> > > > > > > > > > > > bit
>> > > > > > > > > > > > > > odd
>> > > > > > > > > > > > > > > >> that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> ConsumerGroupHeartbeat
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > requires
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > "Read
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> Group"
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > even
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > though
>> > > > > > > > > it
>> > > > > > > > > > > > seems
>> > > > > > > > > > > > > > to do
>> > > > > > > > > > > > > > > >> more
>> > > > > > > > > > > > > > > >> > > > than
>> > > > > > > > > > > > > > > >> > > > > > > > > reading.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > I
>> > > > > > > agree.
>> > > > > > > > > > This is
>> > > > > > > > > > > > > > how the
>> > > > > > > > > > > > > > > >> > > > current
>> > > > > > > > > > > > > > > >> > > > > > > > > protocol works
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > We
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > only
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > require
>> > > > > > > > > > "Read
>> > > > > > > > > > > > > > Group" to
>> > > > > > > > > > > > > > > >> join a
>> > > > > > > > > > > > > > > >> > > > > > group. We
>> > > > > > > > > > > > > > > >> > > > > > > > > could
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consider
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> changing
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> this
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > but I
>> > > > > > > am
>> > > > > > > > > not
>> > > > > > > > > > > > sure
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > >> it is
>> > > > > > > > > > > > > > > >> > > > worth
>> > > > > > > > > > > > > > > >> > > > > > it.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 13.
>> > > > > > > How
>> > > > > > > > > is
>> > > > > > > > > > > > topic
>> > > > > > > > > > > > > > > >> recreation
>> > > > > > > > > > > > > > > >> > > > > > handled by
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consumer
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > group
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > protocol?
>> > > > > > > > > > It
>> > > > > > > > > > > > > > would be
>> > > > > > > > > > > > > > > >> good to
>> > > > > > > > > > > > > > > >> > > > > > have a
>> > > > > > > > > > > > > > > >> > > > > > > > > section
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Noted.
>> > > > > > > > > From
>> > > > > > > > > > a
>> > > > > > > > > > > > > > protocol
>> > > > > > > > > > > > > > > >> > > > > > perspective, the
>> > > > > > > > > > > > > > > >> > > > > > > > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> have a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > topic
>> > > > > > > id
>> > > > > > > > > so
>> > > > > > > > > > it
>> > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > >> treat it
>> > > > > > > > > > > > > > > >> > > > like a
>> > > > > > > > > > > > > > > >> > > > > > > > > topic with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > different
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > name.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > only
>> > > > > > > issue
>> > > > > > > > > > is
>> > > > > > > > > > > > that
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > fetch/commit
>> > > > > > > > > > > > > > > >> > > > > > > > > offsets
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> APIs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > do not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > support
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > IDs
>> > > > > > > so the
>> > > > > > > > > > > > > consumer
>> > > > > > > > > > > > > > > >> would
>> > > > > > > > > > > > > > > >> > > > reuse the
>> > > > > > > > > > > > > > > >> > > > > > > > > offsets
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> based
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > same.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > that
>> > > > > > > we
>> > > > > > > > > > should
>> > > > > > > > > > > > > > update
>> > > > > > > > > > > > > > > >> those
>> > > > > > > > > > > > > > > >> > > > APIs
>> > > > > > > > > > > > > > > >> > > > > > as well
>> > > > > > > > > > > > > > > >> > > > > > > > > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> order
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> consistent
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> end
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > to
>> > > > > > > end.
>> > > > > > > > > That
>> > > > > > > > > > > > would
>> > > > > > > > > > > > > > > >> strengthen
>> > > > > > > > > > > > > > > >> > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > semantics of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consumer.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 14.
>> > > > > > > The
>> > > > > > > > > > KIP
>> > > > > > > > > > > > > > mentions
>> > > > > > > > > > > > > > > >> we will
>> > > > > > > > > > > > > > > >> > > > > > write the
>> > > > > > > > > > > > > > > >> > > > > > > > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > coordinator
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Java.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> Even
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > though
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > this is
>> > > > > > > > > an
>> > > > > > > > > > > > > > > >> implementation
>> > > > > > > > > > > > > > > >> > > > > > detail, do
>> > > > > > > > > > > > > > > >> > > > > > > > > we plan
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > have a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> gradle
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > module
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > for
>> > > > > > > it?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Yes.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 15.
>> > > > > > > Do
>> > > > > > > > > we
>> > > > > > > > > > > > have a
>> > > > > > > > > > > > > > > >> scalability
>> > > > > > > > > > > > > > > >> > > > > > goal when
>> > > > > > > > > > > > > > > >> > > > > > > > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> comes
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to how
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> many
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > members
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > new
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > group
>> > > > > > > > > > protocol
>> > > > > > > > > > > > > can
>> > > > > > > > > > > > > > > >> support?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > We
>> > > > > > > don't
>> > > > > > > > > > have
>> > > > > > > > > > > > > > numbers
>> > > > > > > > > > > > > > > >> at the
>> > > > > > > > > > > > > > > >> > > > > > moment. The
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> support
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> 1000s
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > of
>> > > > > > > members
>> > > > > > > > > > per
>> > > > > > > > > > > > > > group.
>> > > > > > > > > > > > > > > >> We will
>> > > > > > > > > > > > > > > >> > > > > > measure
>> > > > > > > > > > > > > > > >> > > > > > > > > this when
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > first
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > implementation.
>> > > > > > > > > > > > > Note
>> > > > > > > > > > > > > > > >> that we
>> > > > > > > > > > > > > > > >> > > > might
>> > > > > > > > > > > > > > > >> > > > > > have
>> > > > > > > > > > > > > > > >> > > > > > > > > other
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > bottlenecks
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > down
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > road
>> > > > > > > (e.g.
>> > > > > > > > > > > > offset
>> > > > > > > > > > > > > > > >> commits).
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > 16.
>> > > > > > > Did
>> > > > > > > > > we
>> > > > > > > > > > > > > > consider
>> > > > > > > > > > > > > > > >> having
>> > > > > > > > > > > > > > > >> > > > > > > > > SubscribedTopidIds
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > > > SubscribedTopicNames
>> > > > > > > > > > > > > > > >> in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
>> > > > > ConsumerGroupHeartbeatRequest?
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> idea
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > since we
>> > > > > > > > > > have
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > > > >> resolve the
>> > > > > > > > > > > > > > > >> > > > > > regex on
>> > > > > > > > > > > > > > > >> > > > > > > > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > server, we
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> can do
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> same
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > the
>> > > > > > > > > topic
>> > > > > > > > > > > > name?
>> > > > > > > > > > > > > > The
>> > > > > > > > > > > > > > > >> > > > difference
>> > > > > > > > > > > > > > > >> > > > > > is that
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> sending
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regex is
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > more
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > efficient
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > whereas
>> > > > > > > > > > > > sending
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > >> topic
>> > > > > > > > > > > > > > > >> > > > names
>> > > > > > > > > > > > > > > >> > > > > > is less
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > efficient.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > Furthermore,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > delete
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > recreation is
>> > > > > > > > > > > > > > easier
>> > > > > > > > > > > > > > > >> to
>> > > > > > > > > > > > > > > >> > > > handle
>> > > > > > > > > > > > > > > >> > > > > > if we
>> > > > > > > > > > > > > > > >> > > > > > > > > have
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ids.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > The
>> > > > > > > idea
>> > > > > > > > > > was to
>> > > > > > > > > > > > > > > >> consolidate the
>> > > > > > > > > > > > > > > >> > > > > > metadata
>> > > > > > > > > > > > > > > >> > > > > > > > > lookup
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> server
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > for
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> both
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > paths
>> > > > > > > but
>> > > > > > > > > I
>> > > > > > > > > > do
>> > > > > > > > > > > > > agree
>> > > > > > > > > > > > > > > >> with your
>> > > > > > > > > > > > > > > >> > > > > > point. As
>> > > > > > > > > > > > > > > >> > > > > > > > > a
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> second
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > though,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > ids
>> > > > > > > may be
>> > > > > > > > > > > > better
>> > > > > > > > > > > > > > here
>> > > > > > > > > > > > > > > >> for the
>> > > > > > > > > > > > > > > >> > > > > > delete and
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > recreation
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> Also, I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > suppose
>> > > > > > > > > > that we
>> > > > > > > > > > > > > may
>> > > > > > > > > > > > > > > >> allow
>> > > > > > > > > > > > > > > >> > > > users to
>> > > > > > > > > > > > > > > >> > > > > > > > > subscribe
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> ids
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > future
>> > > > > > > > > > because
>> > > > > > > > > > > > > that
>> > > > > > > > > > > > > > is
>> > > > > > > > > > > > > > > >> the only
>> > > > > > > > > > > > > > > >> > > > > > way to be
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> really
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > robust
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > re-creation.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > Best,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > On
>> > > > > > > Tue,
>> > > > > > > > > Jul
>> > > > > > > > > > 12,
>> > > > > > > > > > > > > > 2022 at
>> > > > > > > > > > > > > > > >> 1:38 PM
>> > > > > > > > > > > > > > > >> > > > > > David
>> > > > > > > > > > > > > > > >> > > > > > > > > Jacot <
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> > > > > > > djacot@confluent.io>
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > >
>> > > wrote:
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > Hi
>> > > > > > > > > > Justine,
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > Thanks
>> > > > > > > > > for
>> > > > > > > > > > > > your
>> > > > > > > > > > > > > > > >> comments.
>> > > > > > > > > > > > > > > >> > > > Please
>> > > > > > > > > > > > > > > >> > > > > > find
>> > > > > > > > > > > > > > > >> > > > > > > > > my
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> answers
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > below.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > -
>> > > > > > > Yes,
>> > > > > > > > > > the new
>> > > > > > > > > > > > > > > >> protocol
>> > > > > > > > > > > > > > > >> > > > relies on
>> > > > > > > > > > > > > > > >> > > > > > > > > topic IDs
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
>> exception
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> of the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > > > topic
>> > > > > > > > > > names
>> > > > > > > > > > > > > based
>> > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > >> the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
>> > > > > > > > > ConsumerGroupHeartbeatRequest.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > am
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> sure
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > if
>> > > > > > > using
>> > > > > > > > > > topic
>> > > > > > > > > > > > > > names
>> > > > > > > > > > > > > > > >> is the
>> > > > > > > > > > > > > > > >> > > > > > right call
>> > > > > > > > > > > > > > > >> > > > > > > > > here.
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> I
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> about
>> > > it
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
>> > > >
>> > > > >
>> > > > > a
>> > > > > > > little
>> > > > > > > > > > more.
>> > > > > > > > > > > > > > > >> Obviously, the
>> > > > > > > > > > > > > > > >> > > > > > KIP does
>> > > > > > > > > > > > > > > >> > > > > > > > > not
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change the
>> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > -- Guozhang
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > -- Guozhang
>> > >
>> >
>> >
>> > --
>> > -- Guozhang
>>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol
Posted by Guozhang Wang <wa...@gmail.com>.
3. Ah thanks, that's my mis-read then. I think keeping the current class
and the params of `onAssignment` is fine then.
On Mon, Oct 3, 2022 at 4:32 AM David Jacot <dj...@confluent.io.invalid>
wrote:
> 3. ConsumerGroupMetadata is already defined in the consumer [1]. I
> realized that we pass it in the current assignor so I thought that we
> could reuse it. The alternative would be to add the same information
> in a class which includes the error and the member assignment as well.
>
> [1]
> https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerGroupMetadata.java
>
> Best,
> David
>
> On Mon, Oct 3, 2022 at 6:15 AM Guozhang Wang <wa...@gmail.com> wrote:
> >
> > 1. Yeah that makes sense. I originally concerned if there's any
> consistency
> > issues if the HB request with encoded topic IDs and the GPA/GIA RPCs with
> > encoded topic IDs use mappings from two different brokers, and one of
> them
> > is stale, but after thinking twice about it I think as long as we handle
> > e.g. `unknown topic ids` from HB responses then we should be fine.
> >
> > 3. Ack, in that case I think we also need to define
> "ConsumerGroupMetadata"
> > which is passed in `onAssignment`. We have a schema defined in the
> > persistent storage for different purposes, but not in the consumer APIs
> > yet. I think that would just include a single "Map<String, Metadata>"?
> >
> > On Fri, Sep 30, 2022 at 5:51 AM David Jacot <djacot@confluent.io.invalid
> >
> > wrote:
> >
> > > Hey Guozhang,
> > >
> > > Thanks for your comments.
> > >
> > > 1. That's a very good question. I think that it is better to separate
> > > the two concerns here. In my opinion, the GPA should only be used to
> > > feed the assignor. I see the GPA as a way to call a remote function so
> > > the GPA provides all its inputs to ensure that the assignor does not
> > > depend on any local state. Knowing this, I would not use the GPA to
> > > update the local state either. That is not its purpose. The consumer
> > > is responsible to update its local metadata cache either when it gets
> > > an unknown topic in the user-provided subscription or in the
> > > coordinator-provided assignment. The consumer has to do this anyway to
> > > get all topic metadata (e.g. leaders).
> > >
> > > 2. Interesting thought. Handling the translation is definitely
> > > possible. I wonder if there are cases where having the topic ids is
> > > beneficial. I am thinking about when a topic is deleted and recreated
> > > right away. In this case, the current member assignment would still
> > > have the old topic id and we would not be able to resolve it anymore.
> > > I suppose that we should directly remove it from the current
> > > assignment on the server side in this case and only add the new one.
> > > As I first thought, that seems OK. The major downside is that the user
> > > of the assignor would not know that the topic is different from what
> > > it may have cached locally.
> > >
> > > We could say that we don't expose topic ids externally for now in
> > > order to stay consistent with the other interfaces but it would be
> > > annoying to add it back if we want to expose topic ids in the future.
> > > I personally believe that we will have to expose them at some point.
> > > For instance, in the rebalance listener, it would be useful to know
> > > that a topic has been deleted and recreated. The only way to do this
> > > is to know the topic id. Let me think a little more about this and
> > > come back to you.
> > >
> > > 3. Sorry for this one. I was a typo. PartitionAssingor#assign should
> > > return GroupAssignment.
> > >
> > > Best,
> > > David
> > >
> > > On Thu, Sep 29, 2022 at 6:06 PM Guozhang Wang <wa...@gmail.com>
> wrote:
> > > >
> > > > Thanks David, I read through the updated wiki, and I have a couple
> > > > follow-up questions:
> > > >
> > > > 1. It seems we would have two potential sources to update the
> consumer's
> > > > internally maintained topic name -> topic ID mapping, which is used
> for
> > > > encoding subscribed topic Ids in the HB request, and offset commit /
> list
> > > > offset requests, etc. One is from metadata responses, and another
> one is
> > > > from the GPA (group prepare assignment). On the other hand, the
> assignor
> > > > APIs directly take parsed AssignmentMemberSpec and
> > > AssignmentTopicMetadata
> > > > from GPA responses directly instead of trying to consult the
> consumer's
> > > own
> > > > cached mapping. So just to clarify, consumers would also update its
> > > > internal mapping from the GPA responses as well, right?
> > > >
> > > > 2. I'm wondering if it's necessary to expose the topic IDs to the
> > > assignor
> > > > developers. What they need to do at the moment is to take the mapping
> > > from
> > > > AssignmentTopicMetadata, and translate their decided assignment into
> > > topic
> > > > IDs when returning the MemberAssignment. We can instead just let
> consumer
> > > > to do such translation for them internally and still let the assignor
> > > > developers to only deal with the topic names. Of course that means
> the
> > > > client side assignor APIs would be a bit further diverged from the
> > > > broker-side assignor such that:
> > > >
> > > > a) Topic metadata would be encoded just as String -> num.partitions;
> > > > b) AssignmentMemberSpec contains subscribedTopicNames and
> topicPartitions
> > > > as target assignments;
> > > > c) Returned MemberAssignment contains list of topicPartitions
> instead of
> > > > topicIdPartitions.
> > > >
> > > > Internally, consumers would convert the IDs to names when calling the
> > > > assign() function, and convert them back when sending the request;
> and
> > > > similarly it would convert the IDs to names when calling
> onAssignment()
> > > as
> > > > well.
> > > >
> > > > In short, I'm wondering if we should bite the bullet for the users,
> > > eating
> > > > more complexities internally than pushing that to users.
> > > >
> > > > 3. This is a nit one: the return type of `Assignment
> > > assign(AssignmentSpec
> > > > assignmentSpec);` seems not defined in the doc. I think it would just
> > > > include two fields, as:
> > > >
> > > > class Assignment {
> > > > GroupAssignment groupAssignment;
> > > >
> > > > ConsumerGroupMetadata consumerGroupMetadata;
> > > > }
> > > >
> > > > Is that right?
> > > >
> > > > Guozhang
> > > >
> > > > On Thu, Sep 29, 2022 at 1:26 AM David Jacot
> <djacot@confluent.io.invalid
> > > >
> > > > wrote:
> > > >
> > > > > Hi Guozhang,
> > > > >
> > > > > Thanks for your input.
> > > > >
> > > > > > 1) in HB request, when clients send the subscription, they are
> still
> > > > > > sending them in terms of topic names. That is, the consumer would
> > > not try
> > > > > > to keep all the mappings from topic names to topic ids for the
> > > subscribed
> > > > > > topics.
> > > > >
> > > > > This is a mistake of mine. The heartbeat request should use Topic
> Ids.
> > > > > I forgot to change it. The idea is that the client will get the
> > > > > metadata first to translate the topic names to topic ids and thus
> also
> > > > > have the mapping in its local metadata cache.
> > > > >
> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> > > already.
> > > > > And
> > > > > > fetch requests can directly rely on that since they just need
> topic
> > > IDs
> > > > > and
> > > > > > do not care what actual topic names are.
> > > > >
> > > > > That's right.
> > > > >
> > > > > > 3) in prepare-assignment responses, we return subscriptions as
> topic
> > > IDs
> > > > > as
> > > > > > well.
> > > > >
> > > > > That's right.
> > > > >
> > > > > > In a word, the consumer does not try to maintain the mapping of
> topic
> > > > > names
> > > > > > to IDs at all, and even if they do, they will not expose it in
> e.g.
> > > > > > assignor APIs. But for client-side assignors like Streams, if
> they
> > > do not
> > > > > > maintain the mapping from topic IDs to topic names, certain logic
> > > such as
> > > > > > co-partitioning indeed cannot be executed.
> > > > >
> > > > > As I said, the consumer will have the mapping in its local metadata
> > > > > cache. It needs to get the partition metadata anyway and the topic
> id
> > > > > is part of it. However, I do agree that this mapping is not
> exposed.
> > > > >
> > > > > > So I think we'd need to consider 1) use topic names in
> subscriptions
> > > in
> > > > > the
> > > > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > > > PartitionAssignor#TopicMetadata#topicName.
> > > > >
> > > > > 1) makes sense to me. I think that we could actually provide both
> > > > > names and ids. Regarding 2), I think that I am slightly in favor of
> > > > > using both names and ids as well. Let me try to update the protocol
> > > > > and the interface towards this.
> > > > >
> > > > > Thanks,
> > > > > David
> > > > >
> > > > > On Wed, Sep 28, 2022 at 9:35 PM Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > Hello Jun/David,
> > > > > >
> > > > > > Just to chime in here for Jun's comment #50, today Streams does
> not
> > > have
> > > > > a
> > > > > > way to map topic names to topic IDs, and in its topology
> description
> > > > > topics
> > > > > > are encoded with names.
> > > > > >
> > > > > > What I originally thought was that, on the consumer side, we'd
> need
> > > to
> > > > > > maintain this mapping from topic names to topic IDs anyways, and
> > > Streams
> > > > > > should leverage on the consumer's functionality to get the
> > > corresponding
> > > > > > topic names for the given topic IDs. But when I read through the
> RPC
> > > > > again
> > > > > > I realized it may not be the case, since:
> > > > > >
> > > > > > 1) in HB request, when clients send the subscription, they are
> still
> > > > > > sending them in terms of topic names. That is, the consumer would
> > > not try
> > > > > > to keep all the mappings from topic names to topic ids for the
> > > subscribed
> > > > > > topics.
> > > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> > > already.
> > > > > And
> > > > > > fetch requests can directly rely on that since they just need
> topic
> > > IDs
> > > > > and
> > > > > > do not care what actual topic names are.
> > > > > > 3) in prepare-assignment responses, we return subscriptions as
> topic
> > > IDs
> > > > > as
> > > > > > well.
> > > > > >
> > > > > > In a word, the consumer does not try to maintain the mapping of
> topic
> > > > > names
> > > > > > to IDs at all, and even if they do, they will not expose it in
> e.g.
> > > > > > assignor APIs. But for client-side assignors like Streams, if
> they
> > > do not
> > > > > > maintain the mapping from topic IDs to topic names, certain logic
> > > such as
> > > > > > co-partitioning indeed cannot be executed.
> > > > > >
> > > > > > So I think we'd need to consider 1) use topic names in
> subscriptions
> > > in
> > > > > the
> > > > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > > > PartitionAssignor#TopicMetadata#topicName.
> > > > > >
> > > > > > On Wed, Sep 28, 2022 at 1:26 AM David Jacot
> > > <djacot@confluent.io.invalid
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Jun,
> > > > > > >
> > > > > > > Thanks for your comments.
> > > > > > >
> > > > > > > 48. Okay. There is a typo in the text:
> > > > > > > > Note that the persisted target assignment of B already
> includes
> > > foo-2
> > > > > > > here
> > > > > > > but B does not get it yet because foo-2 is not free.
> > > > > > >
> > > > > > > It should be:
> > > > > > > > Note that the persisted member assignment of B already
> includes
> > > foo-2
> > > > > > > here
> > > > > > > but B does not get it yet because foo-2 is not free.
> > > > > > >
> > > > > > > The examples are showing the persisted state in the
> > > __consumer_offsets
> > > > > > > topics. When a member transitions to the next epoch, we
> directly
> > > > > > > persist the whole target assignment in order to reduce the
> number
> > > of
> > > > > > > writes and we manage dependencies in memory. So "B - epoch=2,
> > > > > > > partitions=[foo-2]" is actually correct from a persisted record
> > > > > > > perspective. However, the in-memory state would be different
> > > because
> > > > > > > foo-2 is not yet available as you pointed out. The in-memory
> state
> > > > > > > would like as follow:
> > > > > > >
> > > > > > > Group (epoch=2)
> > > > > > > A
> > > > > > > B
> > > > > > > Target Assignment (epoch=2)
> > > > > > > A - partitions=[foo-0, foo-1]
> > > > > > > B - partitions=[foo-2]
> > > > > > > Member Assignment
> > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2], pending
> > > partitions=[]
> > > > > > > B - epoch=2, partitions=[], pending partitions=[foo-2]
> > > > > > >
> > > > > > > I suppose that the confusion comes from this. I will go and
> convert
> > > > > > > all the examples to their in-memory representation in order to
> > > reduce
> > > > > > > the confusion.
> > > > > > >
> > > > > > > 50. Okay. I have to admit that I am not so familiar with the
> > > internals
> > > > > > > of KStreams. If I understand correctly, you're saying that
> KStreams
> > > > > > > may not have topic ids in its topology so topic names are
> missing
> > > here
> > > > > > > to match the topics. I was hoping that KStreams would have
> topic
> > > ids
> > > > > > > in the topology as well. If that is not the case, we can also
> add
> > > > > > > topic names on our side. I will check this.
> > > > > > >
> > > > > > > 53. It seems that I was not clear. Sorry for that. I did not
> mean
> > > to
> > > > > > > say that MemberAssignment.metadata is not used by the group
> > > > > > > coordinator. Let me re-explain the flow.
> > > > > > >
> > > > > > > PartitionAssignor.assign() returns an Assignment. Assignment
> > > contains
> > > > > > > MemberAssignments where each member has
> MemberAssignment.metadata.
> > > The
> > > > > > > Assignment is installed by the client with the
> > > > > > > ConsumerGroupInstallAssignment API and this overrides the
> current
> > > > > > > target assignment in the coordinator. Then the group
> coordinator
> > > uses
> > > > > > > it to reconcile members. Therefore the target assignment will
> > > > > > > eventually become the current member assignment and it will be
> > > > > > > delivered to the member, including the original
> > > > > > > MemberAssignment.metadata defines by the assignor.
> > > > > > >
> > > > > > > I hope that this helps to clarify how
> MemberAssignment.metadata is
> > > > > > > used afterwards.
> > > > > > >
> > > > > > > Best,
> > > > > > > David
> > > > > > >
> > > > > > > On Tue, Sep 27, 2022 at 10:31 PM Jun Rao
> <jun@confluent.io.invalid
> > > >
> > > > > wrote:
> > > > > > > > Hi, David,
> > > > > > > >
> > > > > > > > Thanks for the reply. A few more follow up questions.
> > > > > > > >
> > > > > > > > 48. Hmm, so member assignment is what the client sees. Then,
> I
> > > am not
> > > > > > > sure
> > > > > > > > about the following. The text says B cannot get foo-2 yet.
> But in
> > > > > Member
> > > > > > > > Assignment, B has foo-2.
> > > > > > > >
> > > > > > > > At this point B can transition to epoch 2 but cannot get
> foo-2
> > > until
> > > > > A
> > > > > > > > revokes it.
> > > > > > > >
> > > > > > > > Note that the persisted target assignment of B already
> includes
> > > foo-2
> > > > > > > here
> > > > > > > > but B does not get it yet because foo-2 is not free.
> > > > > > > >
> > > > > > > > Group (epoch=2)
> > > > > > > > A
> > > > > > > > B
> > > > > > > > Target Assignment (epoch=2)
> > > > > > > > A - partitions=[foo-0, foo-1]
> > > > > > > > B - partitions=[foo-2]
> > > > > > > > Member Assignment
> > > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2]
> > > > > > > > B - epoch=2, partitions=[foo-2]
> > > > > > > >
> > > > > > > >
> > > > > > > > 50. Hmm, I thought KStreams doesn't use range assignor all
> the
> > > time.
> > > > > It
> > > > > > > > does the assignment based on the topology. For example, if
> you
> > > are
> > > > > > > joining
> > > > > > > > two topics A and B and deriving the join result in a 3rd
> topic C,
> > > > > which
> > > > > > > > will be aggregated. You want to use range to co-locate the
> > > partitions
> > > > > > > from
> > > > > > > > topic A and B. However, for topic C, the partition could
> > > potentially
> > > > > be
> > > > > > > > assigned in round robin. Without the topic name, KStreams
> won't
> > > know
> > > > > > > which
> > > > > > > > topics need to be co-located.
> > > > > > > >
> > > > > > > > 53. If MemberAssignment.metadata is not used by the group
> > > > > coordinator,
> > > > > > > > where is it being used?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Tue, Sep 27, 2022 at 1:20 PM Jason Gustafson
> > > > > > > <ja...@confluent.io.invalid>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi David,
> > > > > > > > >
> > > > > > > > > Thanks for the response. Replied below:
> > > > > > > > >
> > > > > > > > > > I don't have a strong opinion on this one. However, it
> seems
> > > that
> > > > > > > this
> > > > > > > > > could be a pandora box if our subset turns out to not be
> > > sufficient
> > > > > > > > > for some existing users. We will have to add more to it
> and the
> > > > > > > > > compatibility story between versions could be a real
> > > nightmare. One
> > > > > > > > > thing that we could do for sure is disabling all unicode
> > > groups in
> > > > > the
> > > > > > > > > library. That seems possible and would already reduce the
> > > scope.
> > > > > > > > >
> > > > > > > > > Changes to the regex syntax seems not a major problem to
> me. We
> > > > > would
> > > > > > > bump
> > > > > > > > > the protocol version as we usually do. Actually that begs
> the
> > > same
> > > > > > > question
> > > > > > > > > about RE2. How stable is the syntax and does it differ
> > > depending
> > > > > on the
> > > > > > > > > version of the library? In general, I do agree that we'd be
> > > better
> > > > > off
> > > > > > > not
> > > > > > > > > having to manage the syntax ourselves if we can convince
> > > ourselves
> > > > > that
> > > > > > > > > compatibility won't be a problem and the client won't
> > > eventually
> > > > > need
> > > > > > > to
> > > > > > > > > have access to it.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Jason
> > > > > > > > >
> > > > > > > > > On Tue, Sep 27, 2022 at 6:21 AM David Jacot
> > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Luke,
> > > > > > > > > >
> > > > > > > > > > Thanks for your comments.
> > > > > > > > > >
> > > > > > > > > > Your suggestion makes sense. Jun's made the same one. We
> are
> > > also
> > > > > > > > > > discussing whether we should directly return the group
> state
> > > in
> > > > > the
> > > > > > > > > > response and drop the prepare assignment api. It seems
> that
> > > the
> > > > > > > > > > current consensus is to keep it though. If that stays
> like
> > > this,
> > > > > I
> > > > > > > > > > will apply your suggestion.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > David
> > > > > > > > > >
> > > > > > > > > > On Tue, Sep 27, 2022 at 10:40 AM Luke Chen <
> > > showuon@gmail.com>
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi David,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for your patience for answering all these
> questions!
> > > > > > > > > > >
> > > > > > > > > > > One more question from me:
> > > > > > > > > > > 1. When the selected consumer is going to be in
> client-side
> > > > > > > assigning,
> > > > > > > > > we
> > > > > > > > > > > will return COMPUTE_ASSIGNMENT in heartbeat response.
> I'm
> > > > > > > wondering if
> > > > > > > > > we
> > > > > > > > > > > could use a flag to indicate the assignment is
> required.
> > > The
> > > > > > > rationale
> > > > > > > > > is
> > > > > > > > > > > when returning error, we lose a chance to catch up
> with the
> > > > > latest
> > > > > > > > > target
> > > > > > > > > > > assignment or revoke partitions. Since we expected the
> > > record
> > > > > > > fetching
> > > > > > > > > is
> > > > > > > > > > > not blocked by the rebalance/assignment computing, we
> might
> > > > > want to
> > > > > > > > > allow
> > > > > > > > > > > the members to update the member assignment in each
> > > heartbeat.
> > > > > > > > > > >
> > > > > > > > > > > Luke
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Sep 27, 2022 at 9:43 AM Jason Gustafson
> > > > > > > > > > <ja...@confluent.io.invalid>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hey David,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for the response.
> > > > > > > > > > > >
> > > > > > > > > > > > > Do you think that we could have a case where two
> > > members
> > > > > with
> > > > > > > the
> > > > > > > > > > same
> > > > > > > > > > > > static id are active?
> > > > > > > > > > > >
> > > > > > > > > > > > I was just thinking about the handoff when we spin
> up a
> > > new
> > > > > node
> > > > > > > > > > before the
> > > > > > > > > > > > old one completes shutdown. It's probably ok to fence
> > > the old
> > > > > > > member
> > > > > > > > > > > > abruptly. I do recall we had a few issues with this
> > > handoff
> > > > > in
> > > > > > > the
> > > > > > > > > > past. It
> > > > > > > > > > > > was due to the fact that the replaced node might have
> > > been
> > > > > the
> > > > > > > > > leader.
> > > > > > > > > > > > Since we've removed the leader notion, perhaps this
> kind
> > > of
> > > > > issue
> > > > > > > > > won't
> > > > > > > > > > > > resurface.
> > > > > > > > > > > >
> > > > > > > > > > > > In regard to the RE2 library. I understand the
> intent to
> > > pass
> > > > > > > this
> > > > > > > > > > through
> > > > > > > > > > > > as a string without any validation so that we don't
> need
> > > the
> > > > > > > > > > dependence on
> > > > > > > > > > > > the client. I do wonder if that will be restrictive
> in
> > > > > practice.
> > > > > > > For
> > > > > > > > > > > > example, if a regex changes on the client side,
> would we
> > > > > need to
> > > > > > > > > > validate
> > > > > > > > > > > > against the current assignment to see which
> partitions to
> > > > > > > continue
> > > > > > > > > > > > fetching? Or would we instead invalidate the complete
> > > current
> > > > > > > > > > assignment?
> > > > > > > > > > > > Perhaps this is a rare case and we can do the latter.
> > > There
> > > > > > > could be
> > > > > > > > > > more
> > > > > > > > > > > > situations though.
> > > > > > > > > > > >
> > > > > > > > > > > > As an alternative, I wonder if we could define a
> > > minimalistic
> > > > > > > regex
> > > > > > > > > > syntax
> > > > > > > > > > > > (a subset of RE2 say) that could map more easily to
> regex
> > > > > support
> > > > > > > > > that
> > > > > > > > > > > > already exists in clients? For example, in this
> > > scenario, we
> > > > > are
> > > > > > > > > > matching
> > > > > > > > > > > > topics, so we only need to be able to represent topic
> > > > > characters
> > > > > > > and
> > > > > > > > > > not
> > > > > > > > > > > > the whole smorgasbord of whitespace and unicode
> options.
> > > > > Would it
> > > > > > > > > make
> > > > > > > > > > > > sense to have a reduced syntax which includes only
> > > standard
> > > > > topic
> > > > > > > > > > > > characters, wildcards, character classes, and
> > > repetitions?
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Jason
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 22, 2022 at 9:02 AM Guozhang Wang <
> > > > > > > wangguoz@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi David, thanks for all the detailed
> explanations. I
> > > think
> > > > > > > they
> > > > > > > > > all
> > > > > > > > > > make
> > > > > > > > > > > > > sense. Just want to have a couple follow-ups here:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > I don't really see the benefits here because
> server
> > > side
> > > > > > > > > assignors
> > > > > > > > > > > > > don't have metadata at all. They only assign
> > > > > topic-partitions.
> > > > > > > They
> > > > > > > > > > > > > are not supposed to generate metadata nor to
> receive
> > > > > metadata
> > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > > > > members.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Just to clarify I was asking about the `version`
> of the
> > > > > > > assignor
> > > > > > > > > > (i.e. up
> > > > > > > > > > > > > to what version that the client would support),
> and I
> > > do
> > > > > agree
> > > > > > > we
> > > > > > > > > > would
> > > > > > > > > > > > not
> > > > > > > > > > > > > need metadata. What I have in mind is that, for
> some
> > > > > specific
> > > > > > > > > > built-in
> > > > > > > > > > > > > broker-assignors, e.g. rack-aware assignors, if
> it's
> > > > > possible
> > > > > > > that
> > > > > > > > > > in a
> > > > > > > > > > > > > newer version we would have a hierarchical rack ID
> > > string
> > > > > > > format,
> > > > > > > > > > like
> > > > > > > > > > > > > "tier1-tier2" etc, but if some client has not
> upgraded
> > > > > their
> > > > > > > rack
> > > > > > > > > ID
> > > > > > > > > > > > > would still be in old format. In this case, the
> broker
> > > then
> > > > > > > needs
> > > > > > > > > to
> > > > > > > > > > > > choose
> > > > > > > > > > > > > the old versioned assignor. I'm probably making
> > > something
> > > > > up
> > > > > > > here
> > > > > > > > > for
> > > > > > > > > > > > rack
> > > > > > > > > > > > > aware assignors, but I'm wondering if in general
> such
> > > an
> > > > > > > > > > "auto-downgrade"
> > > > > > > > > > > > > behavior would be needed still for broker-side
> > > assignor,
> > > > > and
> > > > > > > if yes
> > > > > > > > > > would
> > > > > > > > > > > > > "version" still be useful.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Yeah, that's right. Within a rebalance,
> > > `onAssignment` is
> > > > > > > called
> > > > > > > > > > once
> > > > > > > > > > > > > when the member transitions to a new epoch. This
> one
> > > > > contains
> > > > > > > the
> > > > > > > > > > full
> > > > > > > > > > > > > metadata provided by the client side assignor.
> Then,
> > > > > > > `onAssignment`
> > > > > > > > > > > > > can be called max N times where N is the number of
> > > > > partitions
> > > > > > > > > pending
> > > > > > > > > > > > > revocation by other members. Let me try to clarify
> > > this in
> > > > > the
> > > > > > > KIP.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Okay, my understanding is that the calling
> ordering of
> > > > > these
> > > > > > > > > > callbacks
> > > > > > > > > > > > > would be like the following:
> > > > > > > > > > > > >
> > > > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > > > onPartitionsRevoked(); // just once, since we do
> not
> > > > > really
> > > > > > > need
> > > > > > > > > > > > > to revoke incrementally.
> > > > > > > > > > > > >
> > > > > > > > > > > > > onAssignment(); // the first call, with epoch
> > > > > incremented
> > > > > > > > > > > > > onPartitionsAssigned(); // paired with the
> > > onAssignment
> > > > > > > > > > > > >
> > > > > > > > > > > > > onAssignment(); // the first
> onAssignment
> > > > > would
> > > > > > > bump
> > > > > > > > > up
> > > > > > > > > > the
> > > > > > > > > > > > > epoch, and the metadata reflected.
> > > > > > > > > > > > > onPartitionsAssigned(); // each time we get an
> > > additional
> > > > > > > > > > assignment,
> > > > > > > > > > > > we
> > > > > > > > > > > > > call onAssignment and then paired with an
> > > > > onPartitionsAssigned
> > > > > > > > > > > > > ...
> > > > > > > > > > > > > onAssignment();
> > > > > > > > > > > > > onPartitionsAssigned(); // on each of the
> > > onAssignment
> > > > > > > calls, the
> > > > > > > > > > > > encoded
> > > > > > > > > > > > > metadata would not change, only the incrementally
> added
> > > > > > > partitions
> > > > > > > > > be
> > > > > > > > > > > > > reflected
> > > > > > > > > > > > >
> > > > > > > > > > > > > Is that the case?
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'm wondering if we would still call onAssignment
> just
> > > > > once,
> > > > > > > that
> > > > > > > > > > encodes
> > > > > > > > > > > > > all the assignment for this rebalance, including
> all
> > > the
> > > > > > > partitions
> > > > > > > > > > that
> > > > > > > > > > > > > should be assigned to the member but not yet
> assigned
> > > since
> > > > > > > they
> > > > > > > > > > have not
> > > > > > > > > > > > > been revoked by others. In that case the call
> ordering
> > > > > would
> > > > > > > be:
> > > > > > > > > > > > >
> > > > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > > > onPartitionsRevoked(); // just once
> > > > > > > > > > > > > onAssignment(); // just once, with epoch
> > > incremented,
> > > > > and
> > > > > > > > > metadata
> > > > > > > > > > > > > encoded changed, the "assignment" field also
> reflect
> > > the
> > > > > final
> > > > > > > > > target
> > > > > > > > > > > > > assignment
> > > > > > > > > > > > > onPartitionsAssigned(); // multiple times, which
> > > > > represent
> > > > > > > > > > > > incrementally
> > > > > > > > > > > > > added partitions
> > > > > > > > > > > > > ...
> > > > > > > > > > > > > onPartitionsAssigned();
> > > > > > > > > > > > >
> > > > > > > > > > > > > The motivation from this is that, most users would
> only
> > > > > > > implement
> > > > > > > > > the
> > > > > > > > > > > > > rebalance callback listeners and hence we'd
> definitely
> > > > > need to
> > > > > > > make
> > > > > > > > > > sure
> > > > > > > > > > > > > the semantics of that does not change much, and the
> > > time
> > > > > > > > > > > > > `onPartitionsAssigned` indicate the time when the
> > > > > partitions
> > > > > > > are
> > > > > > > > > > actually
> > > > > > > > > > > > > assigned to it; while for assignors, the
> > > `onAssignment` is
> > > > > > > used to
> > > > > > > > > > > > indicate
> > > > > > > > > > > > > what decision is made regarding for this member,
> i.e.
> > > when
> > > > > the
> > > > > > > > > > partitions
> > > > > > > > > > > > > are decided to be given to it, but not necessarily
> > > meant
> > > > > that
> > > > > > > it
> > > > > > > > > has
> > > > > > > > > > been
> > > > > > > > > > > > > given, since that time should be determined by the
> > > time of
> > > > > > > > > > > > > `onPartitionsAssigned`. The benefits though, would
> be
> > > that
> > > > > > > assignor
> > > > > > > > > > > > > implementers would not need to reason which
> > > `onAssignment`
> > > > > > > would be
> > > > > > > > > > the
> > > > > > > > > > > > > last one for this epoch.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Thu, Sep 22, 2022 at 2:20 AM David Jacot
> > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Guozhang,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 1) The client-side
> "PartitionAssignor#Assignment"
> > > has
> > > > > an
> > > > > > > Error
> > > > > > > > > > byte
> > > > > > > > > > > > > > field,
> > > > > > > > > > > > > > > while the broker-side
> > > "PartitionAssignor#Assignment"
> > > > > does
> > > > > > > not.
> > > > > > > > > > And
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > rationale seems to be assuming that we should
> > > always be
> > > > > > > able to
> > > > > > > > > > do
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > assignment at the broker-side assignor without
> > > errors.
> > > > > > > > > > Personally I
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > > it's still potentially beneficial to add the
> Error
> > > > > field
> > > > > > > even
> > > > > > > > > for
> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> > > where
> > > > > some
> > > > > > > > > > subscribed
> > > > > > > > > > > > > > > topics are not recognized with the current
> broker's
> > > > > > > metadata.
> > > > > > > > > > What do
> > > > > > > > > > > > > you
> > > > > > > > > > > > > > > think?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Yeah, that seems reasonable. However, I wonder
> if it
> > > > > would be
> > > > > > > > > > better
> > > > > > > > > > > > > > to use an exception on the server side. This is
> what
> > > we
> > > > > > > usually
> > > > > > > > > do
> > > > > > > > > > for
> > > > > > > > > > > > > > server side plugins. On the client side, we use a
> > > field
> > > > > > > because
> > > > > > > > > the
> > > > > > > > > > > > > > errors are not defined in advance.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Your comment also makes me think about what we
> > > should do
> > > > > > > when the
> > > > > > > > > > > > > > server side assignor fails. I suppose that we
> have to
> > > > > keep
> > > > > > > the
> > > > > > > > > > current
> > > > > > > > > > > > > > assignment until a new event occurs. For
> instance, in
> > > > > your
> > > > > > > > > example,
> > > > > > > > > > > > > > the coordinator would have to trigger a rebalance
> > > when
> > > > > > > > > unrecognized
> > > > > > > > > > > > > > topics are available. This would be part of the
> > > metadata
> > > > > > > > > > monitoring.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> > > additional
> > > > > > > fields
> > > > > > > > > > > > > > > reason/version/metadata compared with the
> > > broker-side
> > > > > > > > > > GroupMember. I
> > > > > > > > > > > > > > agree
> > > > > > > > > > > > > > > that broker-side assignor would not need
> > > > > reason/metadata
> > > > > > > since
> > > > > > > > > > they
> > > > > > > > > > > > are
> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
> what
> > > about
> > > > > > > version?
> > > > > > > > > > E.g.
> > > > > > > > > > > > is
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > possible that we evolve our broker-side
> built-in
> > > > > assignor
> > > > > > > but
> > > > > > > > > > the old
> > > > > > > > > > > > > > > versioned clients would not be able to work
> with
> > > the
> > > > > new
> > > > > > > > > > version, in
> > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > case we need to let the broker being aware of
> this
> > > and
> > > > > > > upgrade
> > > > > > > > > > its
> > > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I don't really see the benefits here because
> server
> > > side
> > > > > > > > > assignors
> > > > > > > > > > > > > > don't have metadata at all. They only assign
> > > > > > > topic-partitions.
> > > > > > > > > They
> > > > > > > > > > > > > > are not supposed to generate metadata nor to
> receive
> > > > > metadata
> > > > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > > > > > members.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
> client-side
> > > > > > > "GroupMember",
> > > > > > > > > > > > instead
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > including these three fields, what about just
> > > adding
> > > > > the
> > > > > > > > > > "Metadata"
> > > > > > > > > > > > > field
> > > > > > > > > > > > > > > class which has these three fields? Also, there
> > > are two
> > > > > > > > > > "Metadata"
> > > > > > > > > > > > > > > currently in the APIs, the first is a class
> that
> > > > > encodes
> > > > > > > > > > > > > > > reason/version/metadata, and the second is
> just the
> > > > > encoded
> > > > > > > > > > metadata
> > > > > > > > > > > > > > bytes.
> > > > > > > > > > > > > > > I'm wondering what about just naming the first
> as
> > > > > > > > > memberMetadata,
> > > > > > > > > > > > which
> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
> instead
> > > > > naming the
> > > > > > > > > > second
> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > That's a good point. Let me try to rationalize
> this
> > > > > interface
> > > > > > > > > > based on
> > > > > > > > > > > > > > your suggestions.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > David
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 9:21 PM Guozhang Wang <
> > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Just had a few more nit questions about the
> public
> > > > > APIs:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 1) The client-side
> "PartitionAssignor#Assignment"
> > > has
> > > > > an
> > > > > > > Error
> > > > > > > > > > byte
> > > > > > > > > > > > > > field,
> > > > > > > > > > > > > > > while the broker-side
> > > "PartitionAssignor#Assignment"
> > > > > does
> > > > > > > not.
> > > > > > > > > > And
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > rationale seems to be assuming that we should
> > > always be
> > > > > > > able to
> > > > > > > > > > do
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > assignment at the broker-side assignor without
> > > errors.
> > > > > > > > > > Personally I
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > > it's still potentially beneficial to add the
> Error
> > > > > field
> > > > > > > even
> > > > > > > > > for
> > > > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> > > where
> > > > > some
> > > > > > > > > > subscribed
> > > > > > > > > > > > > > > topics are not recognized with the current
> broker's
> > > > > > > metadata.
> > > > > > > > > > What do
> > > > > > > > > > > > > you
> > > > > > > > > > > > > > > think?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> > > additional
> > > > > > > fields
> > > > > > > > > > > > > > > reason/version/metadata compared with the
> > > broker-side
> > > > > > > > > > GroupMember. I
> > > > > > > > > > > > > > agree
> > > > > > > > > > > > > > > that broker-side assignor would not need
> > > > > reason/metadata
> > > > > > > since
> > > > > > > > > > they
> > > > > > > > > > > > are
> > > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but
> what
> > > about
> > > > > > > version?
> > > > > > > > > > E.g.
> > > > > > > > > > > > is
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > possible that we evolve our broker-side
> built-in
> > > > > assignor
> > > > > > > but
> > > > > > > > > > the old
> > > > > > > > > > > > > > > versioned clients would not be able to work
> with
> > > the
> > > > > new
> > > > > > > > > > version, in
> > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > case we need to let the broker being aware of
> this
> > > and
> > > > > > > upgrade
> > > > > > > > > > its
> > > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 3) Also related to 2) above, for the
> client-side
> > > > > > > "GroupMember",
> > > > > > > > > > > > instead
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > including these three fields, what about just
> > > adding
> > > > > the
> > > > > > > > > > "Metadata"
> > > > > > > > > > > > > field
> > > > > > > > > > > > > > > class which has these three fields? Also, there
> > > are two
> > > > > > > > > > "Metadata"
> > > > > > > > > > > > > > > currently in the APIs, the first is a class
> that
> > > > > encodes
> > > > > > > > > > > > > > > reason/version/metadata, and the second is
> just the
> > > > > encoded
> > > > > > > > > > metadata
> > > > > > > > > > > > > > bytes.
> > > > > > > > > > > > > > > I'm wondering what about just naming the first
> as
> > > > > > > > > memberMetadata,
> > > > > > > > > > > > which
> > > > > > > > > > > > > > > then has a bytebuffer field Metadata, or
> instead
> > > > > naming the
> > > > > > > > > > second
> > > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 12:08 PM Guozhang Wang
> <
> > > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks for bringing this question up. I
> think the
> > > > > main
> > > > > > > > > > benefits as
> > > > > > > > > > > > > you
> > > > > > > > > > > > > > > > listed is 2) above if it stays; just to
> clarify,
> > > we
> > > > > would
> > > > > > > > > only
> > > > > > > > > > be
> > > > > > > > > > > > > able
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > save one round trip if the rebalance is still
> > > > > triggered
> > > > > > > by
> > > > > > > > > the
> > > > > > > > > > > > > broker;
> > > > > > > > > > > > > > if
> > > > > > > > > > > > > > > > the rebalance is triggered by the client
> then the
> > > > > > > > > > num.round.trips
> > > > > > > > > > > > are
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > same:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 1) With GroupPrepareAssignment:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> > > suppose
> > > > > it has
> > > > > > > > > > already
> > > > > > > > > > > > > sent
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
> first
> > > since
> > > > > > > only
> > > > > > > > > one
> > > > > > > > > > > > > request
> > > > > > > > > > > > > > /
> > > > > > > > > > > > > > > > response can be inflight with the
> coordinator's
> > > > > socket.
> > > > > > > > > > > > > > > > T1: client receives the HB response, and then
> > > sends
> > > > > the
> > > > > > > > > > > > > > > > GroupPrepareAssignment request.
> > > > > > > > > > > > > > > > T2: the GroupPrepareAssignment response is
> > > returned.
> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
> sends a
> > > > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > > > request.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In total, two round trips.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2) Without GroupPrepareAssignment:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> > > suppose
> > > > > it has
> > > > > > > > > > already
> > > > > > > > > > > > > sent
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > HB and hence has to wait for it to return
> first
> > > since
> > > > > > > only
> > > > > > > > > one
> > > > > > > > > > > > > request
> > > > > > > > > > > > > > /
> > > > > > > > > > > > > > > > response can be inflight with the
> coordinator's
> > > > > socket.
> > > > > > > > > > > > > > > > T1: client receives the HB response, and then
> > > sends
> > > > > the
> > > > > > > new
> > > > > > > > > HB
> > > > > > > > > > > > > request
> > > > > > > > > > > > > > > > with the flag indicating a new rebalance
> needed..
> > > > > > > > > > > > > > > > T2: the HB response with the optional member
> > > metadata
> > > > > > > map is
> > > > > > > > > > > > > returned.
> > > > > > > > > > > > > > > > T3: it calculates the new assignment, and
> sends a
> > > > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > > > request.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In total, two round trips as well.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > -----------------------------
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > So to complete the full picture here, we'd
> need
> > > to
> > > > > modify
> > > > > > > > > both
> > > > > > > > > > HB
> > > > > > > > > > > > > > request
> > > > > > > > > > > > > > > > and response so that the client can also
> > > indicate a
> > > > > new
> > > > > > > > > > rebalance
> > > > > > > > > > > > via
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > HB request as well, right?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Assuming all above is true, I think it's
> okay to
> > > > > merge
> > > > > > > the
> > > > > > > > > > > > > > > > GroupPrepareAssignment into HB given that we
> can
> > > > > make the
> > > > > > > > > > > > additional
> > > > > > > > > > > > > > fields
> > > > > > > > > > > > > > > > encoding the full member (subscription)
> metadata
> > > and
> > > > > > > topic
> > > > > > > > > > metadata
> > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > optional fields.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Mon, Sep 12, 2022 at 5:22 AM David Jacot
> > > > > > > > > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >> Hi all,
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> During an offline conversation, someone
> asked
> > > why we
> > > > > > > need
> > > > > > > > > the
> > > > > > > > > > > > > > > >> ConsumerGroupPrepareAssignment API and
> suggested
> > > > > that we
> > > > > > > > > could
> > > > > > > > > > > > > instead
> > > > > > > > > > > > > > > >> provide the group state in the heartbeat
> > > response.
> > > > > This
> > > > > > > has
> > > > > > > > > a
> > > > > > > > > > few
> > > > > > > > > > > > > > > >> advantages: 1) it does not require using a
> > > special
> > > > > error
> > > > > > > > > code
> > > > > > > > > > to
> > > > > > > > > > > > > > > >> signal that a new assignment is required as
> the
> > > > > signal
> > > > > > > would
> > > > > > > > > > be
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> provided group state; 2) it removes one
> round
> > > trip
> > > > > when
> > > > > > > a
> > > > > > > > > > client
> > > > > > > > > > > > > side
> > > > > > > > > > > > > > > >> assignor is used. The downside is that it
> makes
> > > the
> > > > > > > > > heartbeat
> > > > > > > > > > > > > > > >> response's definition quite large. I recall
> > > that I
> > > > > went
> > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > > > > > > >> current approach due to this.
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> Providing the group state in the heartbeat
> > > response
> > > > > is
> > > > > > > > > > appealing.
> > > > > > > > > > > > > What
> > > > > > > > > > > > > > > >> do you guys think?
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> Best,
> > > > > > > > > > > > > > > >> David
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> On Mon, Sep 12, 2022 at 2:17 PM David Jacot
> <
> > > > > > > > > > djacot@confluent.io>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > Hi Guozhang,
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > 1. I have added a reference to the
> relevant
> > > > > chapter
> > > > > > > > > instead
> > > > > > > > > > of
> > > > > > > > > > > > > > > >> > repeating the whole thing. Does that work
> for
> > > you?
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > 2. The "Rebalance Triggers" section you
> are
> > > > > referring
> > > > > > > to
> > > > > > > > > is
> > > > > > > > > > > > about
> > > > > > > > > > > > > > when
> > > > > > > > > > > > > > > >> > a rebalance should be triggered for the
> > > > > non-upgraded
> > > > > > > > > members
> > > > > > > > > > > > using
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > old protocol. The section mentions that a
> > > > > rebalance
> > > > > > > must
> > > > > > > > > be
> > > > > > > > > > > > > > triggered
> > > > > > > > > > > > > > > >> > when a new assignment is installed. This
> > > implies
> > > > > that
> > > > > > > the
> > > > > > > > > > group
> > > > > > > > > > > > > > epoch
> > > > > > > > > > > > > > > >> > was updated either by a native member or a
> > > > > > > non-upgraded
> > > > > > > > > > member.
> > > > > > > > > > > > > For
> > > > > > > > > > > > > > > >> > the latter, the JoinGroup request would
> be the
> > > > > > > trigger. I
> > > > > > > > > > have
> > > > > > > > > > > > > > added a
> > > > > > > > > > > > > > > >> > reference to the relevant chapter in the
> > > > > "JoinGroup
> > > > > > > > > > Handling"
> > > > > > > > > > > > > > section
> > > > > > > > > > > > > > > >> > as well. Does that make sense?
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > Thanks,
> > > > > > > > > > > > > > > >> > David
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > On Fri, Sep 9, 2022 at 10:35 PM Guozhang
> Wang
> > > <
> > > > > > > > > > > > wangguoz@gmail.com
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > Hello David,
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > Alright I think that's sufficient. Just
> to
> > > make
> > > > > that
> > > > > > > > > > clear in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > doc,
> > > > > > > > > > > > > > > >> > > could we update:
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > 1) the heartbeat request handling
> section,
> > > > > stating
> > > > > > > when
> > > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > > >> will
> > > > > > > > > > > > > > > >> > > trigger rebalance based on the HB's
> member
> > > > > metadata
> > > > > > > /
> > > > > > > > > > reason?
> > > > > > > > > > > > > > > >> > > 2) the "Rebalance Triggers" section to
> > > include
> > > > > what
> > > > > > > we
> > > > > > > > > > > > described
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> "Group
> > > > > > > > > > > > > > > >> > > Epoch - Trigger a rebalance" section as
> > > well?
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > Guozhang
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > On Fri, Sep 9, 2022 at 1:28 AM David
> Jacot
> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > Hi Guozhang,
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > I thought that the assignor will
> always be
> > > > > > > consulted
> > > > > > > > > > when
> > > > > > > > > > > > the
> > > > > > > > > > > > > > next
> > > > > > > > > > > > > > > >> > > > heartbeat request is constructed. In
> other
> > > > > words,
> > > > > > > > > > > > > > > >> > > > `PartitionAssignor#metadata` will be
> > > called
> > > > > for
> > > > > > > every
> > > > > > > > > > > > > heartbeat.
> > > > > > > > > > > > > > > >> This
> > > > > > > > > > > > > > > >> > > > gives the opportunity for the
> assignor to
> > > > > enforce
> > > > > > > a
> > > > > > > > > > > > rebalance
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > > >> > > > setting the reason to a non-zero
> value or
> > > by
> > > > > > > changing
> > > > > > > > > > the
> > > > > > > > > > > > > > bytes. Do
> > > > > > > > > > > > > > > >> > > > you think that this is not sufficient?
> > > Are you
> > > > > > > > > > concerned by
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> delay?
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > Best,
> > > > > > > > > > > > > > > >> > > > David
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > On Fri, Sep 9, 2022 at 7:10 AM
> Guozhang
> > > Wang <
> > > > > > > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > Hello David,
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > One of Jun's comments make me
> thinking:
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > > > >> > > > > In this case, a new assignment is
> > > triggered
> > > > > by
> > > > > > > the
> > > > > > > > > > client
> > > > > > > > > > > > > side
> > > > > > > > > > > > > > > >> > > > > assignor. When constructing the HB,
> the
> > > > > consumer
> > > > > > > > > will
> > > > > > > > > > > > always
> > > > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > > > >> > > > > the client side assignor and
> propagate
> > > the
> > > > > > > > > > information to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > coordinator. In other words, we
> don't
> > > expect
> > > > > > > users
> > > > > > > > > to
> > > > > > > > > > call
> > > > > > > > > > > > > > > >> > > > > Consumer#enforceRebalance anymore.
> > > > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > As I looked at the current
> > > > > PartitionAssignor's
> > > > > > > > > > interface,
> > > > > > > > > > > > we
> > > > > > > > > > > > > > > >> actually do
> > > > > > > > > > > > > > > >> > > > > not have a way yet to instruct how
> to
> > > > > construct
> > > > > > > the
> > > > > > > > > > next
> > > > > > > > > > > > HB
> > > > > > > > > > > > > > > >> request, e.g.
> > > > > > > > > > > > > > > >> > > > > when the assignor wants to enforce
> a new
> > > > > > > rebalance
> > > > > > > > > > with a
> > > > > > > > > > > > > new
> > > > > > > > > > > > > > > >> assignment,
> > > > > > > > > > > > > > > >> > > > > we'd need some customizable APIs
> inside
> > > the
> > > > > > > > > > > > > PartitionAssignor
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> indicate
> > > > > > > > > > > > > > > >> > > > > the next HB telling broker about so.
> > > WDYT
> > > > > about
> > > > > > > > > adding
> > > > > > > > > > > > such
> > > > > > > > > > > > > an
> > > > > > > > > > > > > > > >> API on the
> > > > > > > > > > > > > > > >> > > > > PartitionAssignor?
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > Guozhang
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > On Tue, Sep 6, 2022 at 6:09 AM David
> > > Jacot
> > > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > Hi Jun,
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > I have updated the KIP to include
> your
> > > > > > > feedback. I
> > > > > > > > > > have
> > > > > > > > > > > > > also
> > > > > > > > > > > > > > > >> tried to
> > > > > > > > > > > > > > > >> > > > > > clarify the parts which were not
> > > cleared.
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > Best,
> > > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > On Fri, Sep 2, 2022 at 4:18 PM
> David
> > > > > Jacot <
> > > > > > > > > > > > > > djacot@confluent.io
> > > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Thanks for your feedback. Let me
> > > start
> > > > > by
> > > > > > > > > > answering
> > > > > > > > > > > > your
> > > > > > > > > > > > > > > >> questions
> > > > > > > > > > > > > > > >> > > > > > > inline and I will update the KIP
> > > next
> > > > > week.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
> the
> > > main
> > > > > > > benefits
> > > > > > > > > > of
> > > > > > > > > > > > the
> > > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > > > efficient
> > > > > > > > > > support of
> > > > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > I would also add that the KIP
> > > removes
> > > > > the
> > > > > > > global
> > > > > > > > > > sync
> > > > > > > > > > > > > > barrier
> > > > > > > > > > > > > > > >> in the
> > > > > > > > > > > > > > > >> > > > > > > protocol which is essential to
> > > improve
> > > > > group
> > > > > > > > > > stability
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> > > > > > > scalability, and the KIP also
> > > > > simplifies the
> > > > > > > > > > client by
> > > > > > > > > > > > > > moving
> > > > > > > > > > > > > > > >> most of
> > > > > > > > > > > > > > > >> > > > > > > the logic to the server side.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 30.
> ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> > > singleton.
> > > > > Do we
> > > > > > > plan
> > > > > > > > > > to
> > > > > > > > > > > > > > support
> > > > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
> the
> > > > > > > consumers?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Definitely. The group
> coordinator
> > > will
> > > > > use
> > > > > > > the
> > > > > > > > > > > > assignor
> > > > > > > > > > > > > > used
> > > > > > > > > > > > > > > >> by a
> > > > > > > > > > > > > > > >> > > > > > > majority of the members. This
> > > allows the
> > > > > > > group
> > > > > > > > > to
> > > > > > > > > > move
> > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > >> one
> > > > > > > > > > > > > > > >> > > > > > > assignor to another by a roll.
> This
> > > is
> > > > > > > explained
> > > > > > > > > > in
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> Assignor
> > > > > > > > > > > > > > > >> > > > > > > Selection chapter.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> > > explain
> > > > > > > whether
> > > > > > > > > > it's
> > > > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios when
> it
> > > > > needs to
> > > > > > > be
> > > > > > > > > > filled?
> > > > > > > > > > > > > For
> > > > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
> TopicPartitions
> > > > > needs to
> > > > > > > be
> > > > > > > > > > filled.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > The client is expected to set
> those
> > > > > fields
> > > > > > > in
> > > > > > > > > > case of
> > > > > > > > > > > > a
> > > > > > > > > > > > > > > >> connection
> > > > > > > > > > > > > > > >> > > > > > > issue (e.g. timeout) or when the
> > > fields
> > > > > have
> > > > > > > > > > changed
> > > > > > > > > > > > > since
> > > > > > > > > > > > > > > >> the last
> > > > > > > > > > > > > > > >> > > > > > > HB. The server populates those
> > > fields as
> > > > > > > long as
> > > > > > > > > > the
> > > > > > > > > > > > > > member
> > > > > > > > > > > > > > > >> is not
> > > > > > > > > > > > > > > >> > > > > > > fully reconciled - the member
> should
> > > > > > > acknowledge
> > > > > > > > > > that
> > > > > > > > > > > > it
> > > > > > > > > > > > > > has
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > expected epoch and assignment. I
> > > will
> > > > > > > clarify
> > > > > > > > > > this in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > KIP.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> > > protocol,
> > > > > the
> > > > > > > rack
> > > > > > > > > > > > > affinity
> > > > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
> considered
> > > > > during
> > > > > > > > > > fetching,
> > > > > > > > > > > > but
> > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > >> during
> > > > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> > > Sometimes,
> > > > > once
> > > > > > > the
> > > > > > > > > > > > > assignment
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
> affinity
> > > > > because
> > > > > > > no
> > > > > > > > > > replicas
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > > > wondering
> > > > > > > if we
> > > > > > > > > > should
> > > > > > > > > > > > > use
> > > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
> rack
> > > in
> > > > > > > > > > GroupMember.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > That's an interesting idea. I
> don't
> > > see
> > > > > any
> > > > > > > > > issue
> > > > > > > > > > with
> > > > > > > > > > > > > > adding
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > rack
> > > > > > > > > > > > > > > >> > > > > > > to the members. I will do so.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> > > it's
> > > > > > > useful to
> > > > > > > > > > know
> > > > > > > > > > > > how
> > > > > > > > > > > > > > busy
> > > > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> > > event
> > > > > loop
> > > > > > > > > model,
> > > > > > > > > > it
> > > > > > > > > > > > > seems
> > > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
> fraction
> > > of
> > > > > the
> > > > > > > time
> > > > > > > > > > the
> > > > > > > > > > > > > event
> > > > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > That's a great idea. I will add
> it.
> > > > > Thanks.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > > > coordinator
> > > > > > > > > > failover
> > > > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
> check
> > > if
> > > > > any
> > > > > > > group
> > > > > > > > > > with
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> > > matching
> > > > > topic?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Sure. When the new group
> coordinator
> > > > > takes
> > > > > > > over,
> > > > > > > > > > it
> > > > > > > > > > > > has
> > > > > > > > > > > > > > to:
> > > > > > > > > > > > > > > >> > > > > > > * Setup the session timeouts.
> > > > > > > > > > > > > > > >> > > > > > > * Trigger a new assignment if a
> > > client
> > > > > side
> > > > > > > > > > assignor
> > > > > > > > > > > > is
> > > > > > > > > > > > > > used.
> > > > > > > > > > > > > > > >> We
> > > > > > > > > > > > > > > >> > > > don't
> > > > > > > > > > > > > > > >> > > > > > > store the information about the
> > > member
> > > > > > > selected
> > > > > > > > > > to run
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > > >> > > > > > > so we have to start a new one.
> > > > > > > > > > > > > > > >> > > > > > > * Update the topics metadata,
> > > verify the
> > > > > > > > > wildcard
> > > > > > > > > > > > > > > >> subscriptions, and
> > > > > > > > > > > > > > > >> > > > > > > trigger a rebalance if needed.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 34.
> ConsumerGroupMetadataValue,
> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > > > >> > > > > > > >
> ConsumerGroupMemberMetadataValue:
> > > > > Could we
> > > > > > > > > > document
> > > > > > > > > > > > > what
> > > > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
> the
> > > epoch
> > > > > in
> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> > > about the
> > > > > > > one in
> > > > > > > > > > > > > > > >> > > > > > > >
> > > ConsumerGroupPartitionMetadataValue
> > > > > and
> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Sure. I will clarify that but
> it is
> > > > > always
> > > > > > > the
> > > > > > > > > > latest
> > > > > > > > > > > > > > group
> > > > > > > > > > > > > > > >> epoch.
> > > > > > > > > > > > > > > >> > > > > > > When the group state is
> updated, the
> > > > > group
> > > > > > > epoch
> > > > > > > > > > is
> > > > > > > > > > > > > > bumped so
> > > > > > > > > > > > > > > >> we use
> > > > > > > > > > > > > > > >> > > > > > > that one for all the change
> records
> > > > > related
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > > > > update.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
> will
> > > ensure
> > > > > > > that
> > > > > > > > > the
> > > > > > > > > > > > > > following
> > > > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members exists."
> It's
> > > > > > > possible
> > > > > > > > > for
> > > > > > > > > > a
> > > > > > > > > > > > > member
> > > > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > That's right. Here I meant that
> the
> > > > > members
> > > > > > > > > > provided
> > > > > > > > > > > > by
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > > >> > > > > > > in the assignment must exist in
> the
> > > > > group.
> > > > > > > The
> > > > > > > > > > > > assignor
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > >> not make
> > > > > > > > > > > > > > > >> > > > > > > up new member ids.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> > > with a
> > > > > > > member
> > > > > > > > > > epoch
> > > > > > > > > > > > > > equals to
> > > > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
> member
> > > id
> > > > > > > would be
> > > > > > > > > > used?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > A member is expected to abandon
> all
> > > its
> > > > > > > > > > partitions and
> > > > > > > > > > > > > > > >> rejoins when
> > > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > > >> > > > > > > receives the FENCED_MEMBER_EPOCH
> > > error.
> > > > > In
> > > > > > > this
> > > > > > > > > > case,
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > > coordinator will have removed
> the
> > > member
> > > > > > > from
> > > > > > > > > the
> > > > > > > > > > > > group.
> > > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> member
> > > > > > > > > > > > > > > >> > > > > > > can rejoin the group with the
> same
> > > > > member
> > > > > > > id but
> > > > > > > > > > with
> > > > > > > > > > > > 0
> > > > > > > > > > > > > as
> > > > > > > > > > > > > > > >> epoch. Let
> > > > > > > > > > > > > > > >> > > > > > > me see if I can clarify this in
> the
> > > KIP.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> > > have
> > > > > the
> > > > > > > > > ability
> > > > > > > > > > to
> > > > > > > > > > > > > > trigger a
> > > > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> > > reason
> > > > > or
> > > > > > > by
> > > > > > > > > > updating
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to
> be
> > > > > > > conflicting
> > > > > > > > > > with
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > In this case, a new assignment
> is
> > > > > triggered
> > > > > > > by
> > > > > > > > > the
> > > > > > > > > > > > > client
> > > > > > > > > > > > > > side
> > > > > > > > > > > > > > > >> > > > > > > assignor. When constructing the
> HB,
> > > the
> > > > > > > consumer
> > > > > > > > > > will
> > > > > > > > > > > > > > always
> > > > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > > > >> > > > > > > the client side assignor and
> > > propagate
> > > > > the
> > > > > > > > > > information
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > > coordinator. In other words, we
> > > don't
> > > > > expect
> > > > > > > > > > users to
> > > > > > > > > > > > > call
> > > > > > > > > > > > > > > >> > > > > > > Consumer#enforceRebalance
> anymore.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples
> are
> > > > > nice.
> > > > > > > But
> > > > > > > > > the
> > > > > > > > > > > > > section
> > > > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
> transitions to
> > > > > epoch
> > > > > > > 2, B
> > > > > > > > > > > > > > immediately
> > > > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> > > which
> > > > > seems
> > > > > > > > > > > > incorrect.
> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
> transitions to
> > > > > epoch
> > > > > > > 3, C
> > > > > > > > > > seems
> > > > > > > > > > > > to
> > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > >> into
> > > > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
> epoch
> > > 3, C
> > > > > > > still
> > > > > > > > > > has A -
> > > > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Sorry for that! I will revise
> them.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
> consumers:
> > > Do
> > > > > we
> > > > > > > > > support
> > > > > > > > > > the
> > > > > > > > > > > > > > upgrade
> > > > > > > > > > > > > > > >> from
> > > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > We will support upgrading from
> the
> > > > > consumer
> > > > > > > > > > protocol
> > > > > > > > > > > > > > version
> > > > > > > > > > > > > > > >> 3,
> > > > > > > > > > > > > > > >> > > > > > > introduced in KIP-792. KIP-792
> is
> > > not
> > > > > > > > > implemented
> > > > > > > > > > yet
> > > > > > > > > > > > so
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> earliest
> > > > > > > > > > > > > > > >> > > > > > > version is unknown at the
> moment.
> > > This
> > > > > is
> > > > > > > > > > explained in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> migration
> > > > > > > > > > > > > > > >> > > > > > > plan chapter.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Thanks again for your feedback,
> > > Jun. I
> > > > > will
> > > > > > > > > > update the
> > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > > >> based on
> > > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > > >> > > > > > > next week.
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > Best,
> > > > > > > > > > > > > > > >> > > > > > > David
> > > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > On Thu, Sep 1, 2022 at 9:07 PM
> Jun
> > > Rao
> > > > > > > > > > > > > > > >> <ju...@confluent.io.invalid>
> > > > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > Hi, David,
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall,
> the
> > > main
> > > > > > > benefits
> > > > > > > > > > of
> > > > > > > > > > > > the
> > > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > > > efficient
> > > > > > > > > > support of
> > > > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 30.
> ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> > > singleton.
> > > > > Do we
> > > > > > > plan
> > > > > > > > > > to
> > > > > > > > > > > > > > support
> > > > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in
> the
> > > > > > > consumers?
> > > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> > > explain
> > > > > > > whether
> > > > > > > > > > it's
> > > > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > > > >> > > > > > > > request or the scenarios when
> it
> > > > > needs to
> > > > > > > be
> > > > > > > > > > filled?
> > > > > > > > > > > > > For
> > > > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > > > >> > > > > > > > clear to me when
> TopicPartitions
> > > > > needs to
> > > > > > > be
> > > > > > > > > > filled.
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> > > protocol,
> > > > > the
> > > > > > > rack
> > > > > > > > > > > > > affinity
> > > > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > and the broker is only
> considered
> > > > > during
> > > > > > > > > > fetching,
> > > > > > > > > > > > but
> > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > >> during
> > > > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> > > Sometimes,
> > > > > once
> > > > > > > the
> > > > > > > > > > > > > assignment
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > > > >> > > > > > > > no opportunity for read
> affinity
> > > > > because
> > > > > > > no
> > > > > > > > > > replicas
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > > > wondering
> > > > > > > if we
> > > > > > > > > > should
> > > > > > > > > > > > > use
> > > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > > > >> > > > > > > > to address this by including
> rack
> > > in
> > > > > > > > > > GroupMember.
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> > > it's
> > > > > > > useful to
> > > > > > > > > > know
> > > > > > > > > > > > how
> > > > > > > > > > > > > > busy
> > > > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> > > event
> > > > > loop
> > > > > > > > > model,
> > > > > > > > > > it
> > > > > > > > > > > > > seems
> > > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the
> fraction
> > > of
> > > > > the
> > > > > > > time
> > > > > > > > > > the
> > > > > > > > > > > > > event
> > > > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > > > coordinator
> > > > > > > > > > failover
> > > > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the
> check
> > > if
> > > > > any
> > > > > > > group
> > > > > > > > > > with
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> > > matching
> > > > > topic?
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 34.
> ConsumerGroupMetadataValue,
> > > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > > > >> > > > > > > >
> ConsumerGroupMemberMetadataValue:
> > > > > Could we
> > > > > > > > > > document
> > > > > > > > > > > > > what
> > > > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does
> the
> > > epoch
> > > > > in
> > > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> > > about the
> > > > > > > one in
> > > > > > > > > > > > > > > >> > > > > > > >
> > > ConsumerGroupPartitionMetadataValue
> > > > > and
> > > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator
> will
> > > ensure
> > > > > > > that
> > > > > > > > > the
> > > > > > > > > > > > > > following
> > > > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > > > >> > > > > > > > met: ... All members exists."
> It's
> > > > > > > possible
> > > > > > > > > for
> > > > > > > > > > a
> > > > > > > > > > > > > member
> > > > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> > > with a
> > > > > > > member
> > > > > > > > > > epoch
> > > > > > > > > > > > > > equals to
> > > > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what
> member
> > > id
> > > > > > > would be
> > > > > > > > > > used?
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> > > have
> > > > > the
> > > > > > > > > ability
> > > > > > > > > > to
> > > > > > > > > > > > > > trigger a
> > > > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> > > reason
> > > > > or
> > > > > > > by
> > > > > > > > > > updating
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to
> be
> > > > > > > conflicting
> > > > > > > > > > with
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples
> are
> > > > > nice.
> > > > > > > But
> > > > > > > > > the
> > > > > > > > > > > > > section
> > > > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group
> transitions to
> > > > > epoch
> > > > > > > 2, B
> > > > > > > > > > > > > > immediately
> > > > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> > > which
> > > > > seems
> > > > > > > > > > > > incorrect.
> > > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group
> transitions to
> > > > > epoch
> > > > > > > 3, C
> > > > > > > > > > seems
> > > > > > > > > > > > to
> > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > >> into
> > > > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to
> epoch
> > > 3, C
> > > > > > > still
> > > > > > > > > > has A -
> > > > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of
> consumers:
> > > Do
> > > > > we
> > > > > > > > > support
> > > > > > > > > > the
> > > > > > > > > > > > > > upgrade
> > > > > > > > > > > > > > > >> from
> > > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > Thanks,
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > Jun
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > On Mon, Aug 29, 2022 at 9:20
> AM
> > > David
> > > > > > > Jacot
> > > > > > > > > > > > > > > >> > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > > >> > > > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > Hi all,
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > The KIP states that we will
> > > > > > > re-implement the
> > > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > > >> in
> > > > > > > > > > > > > > > >> > > > Java. I
> > > > > > > > > > > > > > > >> > > > > > > > > discussed this offline with
> a
> > > few
> > > > > folks
> > > > > > > and
> > > > > > > > > > folks
> > > > > > > > > > > > > are
> > > > > > > > > > > > > > > >> concerned
> > > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > we could introduce many
> > > regressions
> > > > > in
> > > > > > > the
> > > > > > > > > old
> > > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > > >> if we do
> > > > > > > > > > > > > > > >> > > > so.
> > > > > > > > > > > > > > > >> > > > > > > > > Therefore, I am going to
> remove
> > > this
> > > > > > > > > statement
> > > > > > > > > > > > from
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> KIP. It
> > > > > > > > > > > > > > > >> > > > is an
> > > > > > > > > > > > > > > >> > > > > > > > > implementation detail after
> all
> > > so
> > > > > it
> > > > > > > does
> > > > > > > > > not
> > > > > > > > > > > > have
> > > > > > > > > > > > > > to be
> > > > > > > > > > > > > > > >> > > > decided at
> > > > > > > > > > > > > > > >> > > > > > > > > this stage. We will likely
> > > start by
> > > > > > > trying
> > > > > > > > > to
> > > > > > > > > > > > > > refactor the
> > > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > > >> > > > > > > > > implementation as a first
> step.
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > Cheers,
> > > > > > > > > > > > > > > >> > > > > > > > > David
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > On Mon, Aug 29, 2022 at
> 3:52 PM
> > > > > David
> > > > > > > Jacot
> > > > > > > > > <
> > > > > > > > > > > > > > > >> djacot@confluent.io
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > Hi Luke,
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> > > machine
> > > > > are:
> > > > > > > > > > "Empty,
> > > > > > > > > > > > > > assigning,
> > > > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
> Consumer
> > > > > Group
> > > > > > > States
> > > > > > > > > > > > > section,
> > > > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > This sentence does not
> refer
> > > to
> > > > > those
> > > > > > > > > group
> > > > > > > > > > > > states
> > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > >> rather
> > > > > > > > > > > > > > > >> > > > to a
> > > > > > > > > > > > > > > >> > > > > > > > > > state machine replication
> > > (SMR).
> > > > > This
> > > > > > > > > > refers to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> entire
> > > > > > > > > > > > > > > >> > > > state of
> > > > > > > > > > > > > > > >> > > > > > > > > > group coordinator which is
> > > > > replicated
> > > > > > > via
> > > > > > > > > > the
> > > > > > > > > > > > log
> > > > > > > > > > > > > > > >> layer. I will
> > > > > > > > > > > > > > > >> > > > > > > > > > clarify this in the KIP.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
> "each
> > > > > state
> > > > > > > > > machine
> > > > > > > > > > is
> > > > > > > > > > > > > > modelled
> > > > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > The idea is to follow a
> model
> > > > > similar
> > > > > > > to
> > > > > > > > > > the new
> > > > > > > > > > > > > > quorum
> > > > > > > > > > > > > > > >> > > > > > controller. We
> > > > > > > > > > > > > > > >> > > > > > > > > > will have N threads to
> process
> > > > > events.
> > > > > > > > > Each
> > > > > > > > > > > > > > > >> __consumer_offsets
> > > > > > > > > > > > > > > >> > > > > > > > > > partition is assigned to a
> > > unique
> > > > > > > thread
> > > > > > > > > > and all
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> events
> > > > > > > > > > > > > > > >> > > > (e.g.
> > > > > > > > > > > > > > > >> > > > > > > > > > requests, callbacks,
> etc.) are
> > > > > > > processed
> > > > > > > > > by
> > > > > > > > > > this
> > > > > > > > > > > > > > > >> thread. This
> > > > > > > > > > > > > > > >> > > > > > simplify
> > > > > > > > > > > > > > > >> > > > > > > > > > concurrency and will
> enable
> > > us to
> > > > > do
> > > > > > > > > > simulation
> > > > > > > > > > > > > > testing
> > > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > > > >> > > > > > > > > > coordinator.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
> state
> > > > > machine
> > > > > > > per
> > > > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > > > consumer
> > > > > > > group"
> > > > > > > > > > owned
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > > > coordinator
> > > > > > > owns 2
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> > > will we
> > > > > > > have 1
> > > > > > > > > > state
> > > > > > > > > > > > > > machine
> > > > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.1. The confusion
> comes
> > > from
> > > > > > > there, I
> > > > > > > > > > > > think.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > > > *is
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> number
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
> machines.
> > > But
> > > > > I'm
> > > > > > > > > > wondering
> > > > > > > > > > > > if
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
> state of
> > > > > each
> > > > > > > > > consumer
> > > > > > > > > > > > group
> > > > > > > > > > > > > > (or
> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
> heavy
> > > > > > > computation,
> > > > > > > > > > why
> > > > > > > > > > > > > > should we
> > > > > > > > > > > > > > > >> need
> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > See 1.2. The idea is to
> have
> > > an
> > > > > > > ability to
> > > > > > > > > > shard
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > processing as
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > computation could be
> heavy.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
> session
> > > > > timeout,
> > > > > > > why
> > > > > > > > > > does
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
> (45s) and
> > > > > > > max(60s)? I
> > > > > > > > > > > > thought
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> > > lower/upper
> > > > > > > bound
> > > > > > > > > of
> > > > > > > > > > it,
> > > > > > > > > > > > > no?
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.session.timeout.ms
> > > > > > > int
> > > > > > > > > > 30s The
> > > > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> > > consumer
> > > > > > > group
> > > > > > > > > > > > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > group.consumer.min.session.timeout.ms
> > > > > > > > > > int 45s
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > group.consumer.max.session.timeout.ms
> > > > > > > > > > int 60s
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > This is indeed a mistake.
> The
> > > > > default
> > > > > > > > > > session
> > > > > > > > > > > > > > timeout
> > > > > > > > > > > > > > > >> should
> > > > > > > > > > > > > > > >> > > > be 45s
> > > > > > > > > > > > > > > >> > > > > > > > > > (the current default).
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server
> side
> > > > > > > assignor
> > > > > > > > > are
> > > > > > > > > > > > > [range,
> > > > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > > > assignor.
> > > > > > > I'd
> > > > > > > > > > like to
> > > > > > > > > > > > > > know
> > > > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
> will
> > > > > choose
> > > > > > > > > uniform
> > > > > > > > > > > > > assignor
> > > > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> > > distribution.
> > > > > Any
> > > > > > > > > other
> > > > > > > > > > > > reason
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> > > List
> > > > > range,
> > > > > > > > > > uniform
> > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > The order on the server
> side
> > > has
> > > > > no
> > > > > > > > > > influence
> > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > > > >> > > > > > must
> > > > > > > > > > > > > > > >> > > > > > > > > > chose the selector that he
> > > wants
> > > > > to
> > > > > > > use.
> > > > > > > > > > There
> > > > > > > > > > > > is
> > > > > > > > > > > > > no
> > > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > > >> > > > in the
> > > > > > > > > > > > > > > >> > > > > > > > > > current proposal. If the
> > > assignor
> > > > > is
> > > > > > > not
> > > > > > > > > > > > specified
> > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > client,
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > request is rejected. The
> > > default
> > > > > > > client
> > > > > > > > > > value
> > > > > > > > > > > > for
> > > > > > > > > > > > > > > >> > > > > > > > > > `group.remote.assignor` is
> > > > > `uniform`
> > > > > > > > > though.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for your very good
> > > > > comments,
> > > > > > > Luke.
> > > > > > > > > I
> > > > > > > > > > hope
> > > > > > > > > > > > > > that my
> > > > > > > > > > > > > > > >> > > > answers
> > > > > > > > > > > > > > > >> > > > > > help
> > > > > > > > > > > > > > > >> > > > > > > > > > to clarify things. I will
> > > update
> > > > > the
> > > > > > > KIP
> > > > > > > > > as
> > > > > > > > > > well
> > > > > > > > > > > > > > based
> > > > > > > > > > > > > > > >> on your
> > > > > > > > > > > > > > > >> > > > > > > > > > feedback.
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > Cheers,
> > > > > > > > > > > > > > > >> > > > > > > > > > David
> > > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > On Mon, Aug 22, 2022 at
> 9:29
> > > AM
> > > > > Luke
> > > > > > > Chen
> > > > > > > > > <
> > > > > > > > > > > > > > > >> showuon@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > Hi David,
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > Thanks for the update.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > Some more questions:
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1. In Group Coordinator
> > > > > section, you
> > > > > > > > > > > > mentioned:
> > > > > > > > > > > > > > > >> > > > > > > > > > > > The new group
> coordinator
> > > will
> > > > > > > have a
> > > > > > > > > > state
> > > > > > > > > > > > > > machine
> > > > > > > > > > > > > > > >> per
> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets*
> > > partitions,
> > > > > > > where
> > > > > > > > > > each
> > > > > > > > > > > > > state
> > > > > > > > > > > > > > > >> machine is
> > > > > > > > > > > > > > > >> > > > > > modelled
> > > > > > > > > > > > > > > >> > > > > > > > > as an
> > > > > > > > > > > > > > > >> > > > > > > > > > > event loop. Those state
> > > machines
> > > > > > > will be
> > > > > > > > > > > > > executed
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> *group.coordinator.threads*
> > > > > threads.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> > > machine
> > > > > are:
> > > > > > > > > > "Empty,
> > > > > > > > > > > > > > assigning,
> > > > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in
> Consumer
> > > > > Group
> > > > > > > States
> > > > > > > > > > > > > section,
> > > > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean
> "each
> > > > > state
> > > > > > > > > machine
> > > > > > > > > > is
> > > > > > > > > > > > > > modelled
> > > > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a
> state
> > > > > machine
> > > > > > > per
> > > > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > > > consumer
> > > > > > > group"
> > > > > > > > > > owned
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > > > coordinator
> > > > > > > owns 2
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> > > will we
> > > > > > > have 1
> > > > > > > > > > state
> > > > > > > > > > > > > > machine
> > > > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > > > *is
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> number
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > > > >> > > > > > > > > > > to run the state
> machines.
> > > But
> > > > > I'm
> > > > > > > > > > wondering
> > > > > > > > > > > > if
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the
> state of
> > > > > each
> > > > > > > > > consumer
> > > > > > > > > > > > group
> > > > > > > > > > > > > > (or
> > > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no
> heavy
> > > > > > > computation,
> > > > > > > > > > why
> > > > > > > > > > > > > > should we
> > > > > > > > > > > > > > > >> need
> > > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 2. For the default
> value in
> > > the
> > > > > new
> > > > > > > > > > configs:
> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer
> session
> > > > > timeout,
> > > > > > > why
> > > > > > > > > > does
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > > > >> > > > > > > > > > > locate between min
> (45s) and
> > > > > > > max(60s)? I
> > > > > > > > > > > > thought
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> > > lower/upper
> > > > > > > bound
> > > > > > > > > of
> > > > > > > > > > it,
> > > > > > > > > > > > > no?
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > group.consumer.session.timeout.ms
> > > > > > > int
> > > > > > > > > > 30s The
> > > > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> > > consumer
> > > > > > > group
> > > > > > > > > > > > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > group.consumer.min.session.timeout.ms
> > > > > > > > > > int 45s
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > group.consumer.max.session.timeout.ms
> > > > > > > > > > int 60s
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server
> side
> > > > > > > assignor
> > > > > > > > > are
> > > > > > > > > > > > > [range,
> > > > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > > > assignor.
> > > > > > > I'd
> > > > > > > > > > like to
> > > > > > > > > > > > > > know
> > > > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users
> will
> > > > > choose
> > > > > > > > > uniform
> > > > > > > > > > > > > assignor
> > > > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> > > distribution.
> > > > > Any
> > > > > > > > > other
> > > > > > > > > > > > reason
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> > > List
> > > > > range,
> > > > > > > > > > uniform
> > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > Thank you.
> > > > > > > > > > > > > > > >> > > > > > > > > > > Luke
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Aug 22, 2022 at
> > > 2:10 PM
> > > > > Luke
> > > > > > > > > Chen
> > > > > > > > > > <
> > > > > > > > > > > > > > > >> showuon@gmail.com
> > > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi Sagar,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > > I have some thoughts
> about
> > > > > Kafka
> > > > > > > > > Connect
> > > > > > > > > > > > > > > >> integrating with
> > > > > > > > > > > > > > > >> > > > > > KIP-848,
> > > > > > > > > > > > > > > >> > > > > > > > > but I
> > > > > > > > > > > > > > > >> > > > > > > > > > > > think we should have a
> > > > > separate
> > > > > > > > > > discussion
> > > > > > > > > > > > > > thread
> > > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > > >> > > > Kafka
> > > > > > > > > > > > > > > >> > > > > > > > > Connect
> > > > > > > > > > > > > > > >> > > > > > > > > > > > KIP: Integrating Kafka
> > > Connect
> > > > > > > With
> > > > > > > > > New
> > > > > > > > > > > > > Consumer
> > > > > > > > > > > > > > > >> Rebalance
> > > > > > > > > > > > > > > >> > > > > > Protocol
> > > > > > > > > > > > > > > >> > > > > > > > > [1],
> > > > > > > > > > > > > > > >> > > > > > > > > > > > and let this
> discussion
> > > thread
> > > > > > > focus
> > > > > > > > > on
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> rebalance
> > > > > > > > > > > > > > > >> > > > > > protocol,
> > > > > > > > > > > > > > > >> > > > > > > > > WDYT?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > > [1]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > > Thank you.
> > > > > > > > > > > > > > > >> > > > > > > > > > > > Luke
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Aug 12, 2022
> at
> > > 9:31
> > > > > PM
> > > > > > > Sagar
> > > > > > > > > <
> > > > > > > > > > > > > > > >> > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thank you
> Guozhang/David
> > > for
> > > > > the
> > > > > > > > > > feedback.
> > > > > > > > > > > > > > Looks
> > > > > > > > > > > > > > > >> like
> > > > > > > > > > > > > > > >> > > > there's
> > > > > > > > > > > > > > > >> > > > > > > > > agreement on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> using separate APIs
> for
> > > > > Connect.
> > > > > > > I
> > > > > > > > > > would
> > > > > > > > > > > > > > revisit
> > > > > > > > > > > > > > > >> the doc
> > > > > > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > > > >> > > > > > see
> > > > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> changes are to be
> made.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Thanks!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Sagar.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> On Tue, Aug 9, 2022
> at
> > > 7:11
> > > > > PM
> > > > > > > David
> > > > > > > > > > Jacot
> > > > > > > > > > > > > > > >> > > > > > > > > <djacot@confluent.io.invalid
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Hi Sagar,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Thanks for the
> feedback
> > > > > and the
> > > > > > > > > > document.
> > > > > > > > > > > > > > That's
> > > > > > > > > > > > > > > >> really
> > > > > > > > > > > > > > > >> > > > > > helpful. I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will take a look
> at it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Overall, it seems
> to me
> > > > > that
> > > > > > > both
> > > > > > > > > > Connect
> > > > > > > > > > > > > > and the
> > > > > > > > > > > > > > > >> > > > Consumer
> > > > > > > > > > > > > > > >> > > > > > could
> > > > > > > > > > > > > > > >> > > > > > > > > share
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the same underlying
> > > > > "engine".
> > > > > > > The
> > > > > > > > > > main
> > > > > > > > > > > > > > > >> difference is
> > > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > Consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigns
> > > topic-partitions to
> > > > > > > members
> > > > > > > > > > > > whereas
> > > > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > > > >> > > > assigns
> > > > > > > > > > > > > > > >> > > > > > tasks
> > > > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > workers. I see two
> > > ways to
> > > > > move
> > > > > > > > > > forward:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) We extend the
> new
> > > > > proposed
> > > > > > > APIs
> > > > > > > > > to
> > > > > > > > > > > > > support
> > > > > > > > > > > > > > > >> different
> > > > > > > > > > > > > > > >> > > > > > resource
> > > > > > > > > > > > > > > >> > > > > > > > > types
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g. partitions,
> > > tasks,
> > > > > > > etc.); or
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 2) We use new
> dedicated
> > > > > APIs
> > > > > > > for
> > > > > > > > > > Connect.
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> dedicated
> > > > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > > > >> > > > > > > > > would be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > similar to the new
> > > ones but
> > > > > > > > > > different on
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > content/resources and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they would rely on
> the
> > > same
> > > > > > > engine
> > > > > > > > > > on the
> > > > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > > > >> > > > side.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I personally lean
> > > towards
> > > > > 2)
> > > > > > > > > because
> > > > > > > > > > I am
> > > > > > > > > > > > > > not a
> > > > > > > > > > > > > > > >> fan of
> > > > > > > > > > > > > > > >> > > > > > > > > overcharging
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > APIs to serve
> different
> > > > > > > purposes.
> > > > > > > > > > That
> > > > > > > > > > > > > being
> > > > > > > > > > > > > > > >> said, I am
> > > > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > > > >> > > > > > > > > opposed to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) if we can find
> an
> > > > > elegant
> > > > > > > way to
> > > > > > > > > > do
> > > > > > > > > > > > it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I think that we can
> > > > > continue to
> > > > > > > > > > discuss
> > > > > > > > > > > > it
> > > > > > > > > > > > > > here
> > > > > > > > > > > > > > > >> for now
> > > > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > > > >> > > > > > order
> > > > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ensure that this
> KIP is
> > > > > > > compatible
> > > > > > > > > > with
> > > > > > > > > > > > > what
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > >> will do
> > > > > > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > > > >> > > > > > > > > Connect in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the future.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Best,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > On Mon, Aug 8,
> 2022 at
> > > > > 2:41 PM
> > > > > > > > > David
> > > > > > > > > > > > Jacot
> > > > > > > > > > > > > <
> > > > > > > > > > > > > > > >> > > > > > djacot@confluent.io>
> > > > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Hi all,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > I am back from
> > > vacation.
> > > > > I
> > > > > > > will
> > > > > > > > > go
> > > > > > > > > > > > > through
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> address
> > > > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > > > >> > > > > > > > > comments
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > in the coming
> days.
> > > > > Thanks
> > > > > > > for
> > > > > > > > > your
> > > > > > > > > > > > > > feedback.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Cheers,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Aug 3,
> 2022
> > > at
> > > > > 10:05
> > > > > > > PM
> > > > > > > > > > Gregory
> > > > > > > > > > > > > > Harris
> > > > > > > > > > > > > > > >> <
> > > > > > > > > > > > > > > >> > > > > > > > > gharris1727@gmail.com
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Hey All!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Thanks for the
> KIP,
> > > > > it's
> > > > > > > > > > wonderful to
> > > > > > > > > > > > > see
> > > > > > > > > > > > > > > >> > > > cooperative
> > > > > > > > > > > > > > > >> > > > > > > > > rebalancing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > making it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > down the stack!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > I had a few
> > > questions:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 1. The
> 'Rejected
> > > > > > > Alternatives'
> > > > > > > > > > > > section
> > > > > > > > > > > > > > > >> describes how
> > > > > > > > > > > > > > > >> > > > > > member
> > > > > > > > > > > > > > > >> > > > > > > > > epoch
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > advance in step
> > > with
> > > > > the
> > > > > > > group
> > > > > > > > > > epoch
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > > >> > > > > > epoch
> > > > > > > > > > > > > > > >> > > > > > > > > values. I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > that this is a
> good
> > > > > idea
> > > > > > > for
> > > > > > > > > the
> > > > > > > > > > > > > reasons
> > > > > > > > > > > > > > > >> described
> > > > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > KIP. When
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > protocol is
> > > > > incrementally
> > > > > > > > > > assigning
> > > > > > > > > > > > > > > >> partitions to a
> > > > > > > > > > > > > > > >> > > > > > worker,
> > > > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> member
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > epoch does each
> > > > > incremental
> > > > > > > > > > > > assignment
> > > > > > > > > > > > > > use?
> > > > > > > > > > > > > > > >> Are
> > > > > > > > > > > > > > > >> > > > member
> > > > > > > > > > > > > > > >> > > > > > epochs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> re-used,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > a single member
> > > epoch
> > > > > can
> > > > > > > > > > correspond
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> multiple
> > > > > > > > > > > > > > > >> > > > > > different
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (monotonically
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > larger)
> > > assignments?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 2. Is the
> > > Assignor's
> > > > > > > 'Reason'
> > > > > > > > > > field
> > > > > > > > > > > > > > opaque
> > > > > > > > > > > > > > > >> to the
> > > > > > > > > > > > > > > >> > > > group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> coordinator? If
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > not, should
> custom
> > > > > > > client-side
> > > > > > > > > > > > assignor
> > > > > > > > > > > > > > > >> > > > implementations
> > > > > > > > > > > > > > > >> > > > > > > > > interact
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Reason field,
> and
> > > how
> > > > > is
> > > > > > > its
> > > > > > > > > > common
> > > > > > > > > > > > > > meaning
> > > > > > > > > > > > > > > >> agreed
> > > > > > > > > > > > > > > >> > > > > > upon? If
> > > > > > > > > > > > > > > >> > > > > > > > > so, what
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > benefit of a
> > > distinct
> > > > > > > Reason
> > > > > > > > > > field
> > > > > > > > > > > > over
> > > > > > > > > > > > > > > >> including
> > > > > > > > > > > > > > > >> > > > such
> > > > > > > > > > > > > > > >> > > > > > > > > functionality
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > opaque
> metadata?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 3. The
> following is
> > > > > > > included in
> > > > > > > > > > the
> > > > > > > > > > > > > KIP:
> > > > > > > > > > > > > > > >> "Thanks to
> > > > > > > > > > > > > > > >> > > > > > this, the
> > > > > > > > > > > > > > > >> > > > > > > > > input
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client side
> > > assignor is
> > > > > > > > > entirely
> > > > > > > > > > > > driven
> > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > > > >> > > > > > > > > coordinator.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > consumer is no
> > > longer
> > > > > > > > > > responsible for
> > > > > > > > > > > > > > > >> maintaining
> > > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > > >> > > > > > state
> > > > > > > > > > > > > > > >> > > > > > > > > besides
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> its
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assigned
> > > partitions."
> > > > > Does
> > > > > > > this
> > > > > > > > > > mean
> > > > > > > > > > > > > > that the
> > > > > > > > > > > > > > > >> > > > > > client-side
> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> MAY
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > incorporate
> > > additional
> > > > > > > > > > non-Metadata
> > > > > > > > > > > > > state
> > > > > > > > > > > > > > > >> (such as
> > > > > > > > > > > > > > > >> > > > > > partition
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > throughput,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > cpu/memory
> metrics,
> > > > > config
> > > > > > > > > > topics,
> > > > > > > > > > > > > etc),
> > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > >> that
> > > > > > > > > > > > > > > >> > > > > > additional
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > non-Metadata
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > state SHOULD
> NOT be
> > > > > used?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 4. I see that
> > > there are
> > > > > > > > > separate
> > > > > > > > > > > > > classes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > org.apache.kafka.server.group.consumer.PartitionAssignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > > > >> > > >
> > > > > > > org.apache.kafka.clients.consumer.PartitionAssignor
> > > > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > seem to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > overlap
> > > significantly.
> > > > > Is
> > > > > > > it
> > > > > > > > > > possible
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > >> these two
> > > > > > > > > > > > > > > >> > > > > > > > > implementations
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > unified? This
> would
> > > > > serve
> > > > > > > to
> > > > > > > > > > promote
> > > > > > > > > > > > > > feature
> > > > > > > > > > > > > > > >> parity
> > > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > > >> > > > > > > > > server-side
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client-side
> > > assignors,
> > > > > and
> > > > > > > > > would
> > > > > > > > > > also
> > > > > > > > > > > > > > > >> facilitate
> > > > > > > > > > > > > > > >> > > > > > operational
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flexibility in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > certain
> > > situations. For
> > > > > > > > > example,
> > > > > > > > > > if a
> > > > > > > > > > > > > > > >> server-side
> > > > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > has some
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > poor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > behavior and
> needs
> > > a
> > > > > patch,
> > > > > > > > > > deploying
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> patched
> > > > > > > > > > > > > > > >> > > > > > assignor to
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and switching
> one
> > > > > consumer
> > > > > > > > > group
> > > > > > > > > > to a
> > > > > > > > > > > > > > > >> client-side
> > > > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > may be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > faster
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and less risky
> than
> > > > > > > patching
> > > > > > > > > all
> > > > > > > > > > of
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> brokers.
> > > > > > > > > > > > > > > >> > > > With
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > currently
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > proposed
> distinct
> > > > > APIs, a
> > > > > > > > > > non-trivial
> > > > > > > > > > > > > > > >> > > > reimplementation
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assembled, and
> if
> > > the
> > > > > two
> > > > > > > APIs
> > > > > > > > > > have
> > > > > > > > > > > > > > diverged
> > > > > > > > > > > > > > > >> > > > > > significantly,
> > > > > > > > > > > > > > > >> > > > > > > > > then it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > possible that a
> > > > > > > > > reimplementation
> > > > > > > > > > > > would
> > > > > > > > > > > > > > not be
> > > > > > > > > > > > > > > >> > > > possible.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > --
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Greg Harris
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > gharris1727@gmail.com
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > github.com/gharris1727
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Aug 3,
> > > 2022 at
> > > > > > > 8:39 AM
> > > > > > > > > > Sagar
> > > > > > > > > > > > <
> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Hi
> > > Guozhang/David,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > I created a
> > > > > confluence
> > > > > > > page
> > > > > > > > > to
> > > > > > > > > > > > > discuss
> > > > > > > > > > > > > > how
> > > > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > based on the
> new
> > > > > > > rebalance
> > > > > > > > > > > > protocol.
> > > > > > > > > > > > > > > >> Here's the
> > > > > > > > > > > > > > > >> > > > page:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > It's also
> pretty
> > > > > longish
> > > > > > > and
> > > > > > > > > I
> > > > > > > > > > have
> > > > > > > > > > > > > > tried
> > > > > > > > > > > > > > > >> to keep
> > > > > > > > > > > > > > > >> > > > a
> > > > > > > > > > > > > > > >> > > > > > format
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> similar to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > KIP-848. Let
> me
> > > know
> > > > > > > what you
> > > > > > > > > > > > think.
> > > > > > > > > > > > > > Also,
> > > > > > > > > > > > > > > >> do you
> > > > > > > > > > > > > > > >> > > > > > think this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > moved to a
> > > separate
> > > > > > > > > discussion
> > > > > > > > > > > > thread
> > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > >> is this
> > > > > > > > > > > > > > > >> > > > one
> > > > > > > > > > > > > > > >> > > > > > fine?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Sagar.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul
> 26,
> > > 2022
> > > > > at
> > > > > > > 7:37
> > > > > > > > > AM
> > > > > > > > > > > > > Sagar <
> > > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Hello
> Guozhang,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thank you
> so
> > > much
> > > > > for
> > > > > > > the
> > > > > > > > > > doc on
> > > > > > > > > > > > > > Kafka
> > > > > > > > > > > > > > > >> Streams.
> > > > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > > > >> > > > > > > > > would do
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > analysis
> and
> > > come
> > > > > up
> > > > > > > with
> > > > > > > > > > such a
> > > > > > > > > > > > > > > >> document.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Sagar.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue,
> Jul 26,
> > > > > 2022 at
> > > > > > > > > 4:47
> > > > > > > > > > AM
> > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > > >> Wang <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> wangguoz@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Hello
> Sagar,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> It would
> be
> > > great
> > > > > if
> > > > > > > you
> > > > > > > > > > could
> > > > > > > > > > > > > come
> > > > > > > > > > > > > > > >> back with
> > > > > > > > > > > > > > > >> > > > some
> > > > > > > > > > > > > > > >> > > > > > > > > analysis on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > how to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> implement
> the
> > > > > Connect
> > > > > > > side
> > > > > > > > > > > > > > integration
> > > > > > > > > > > > > > > >> with
> > > > > > > > > > > > > > > >> > > > the new
> > > > > > > > > > > > > > > >> > > > > > > > > protocol;
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> so
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > far
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> besides
> > > > > leveraging on
> > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > > "protocol
> > > > > > > > > > > > > > > >> type"
> > > > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > > > >> > > > > > did not
> > > > > > > > > > > > > > > >> > > > > > > > > yet
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > through
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> Connect
> > > side
> > > > > > > > > > > > implementations.
> > > > > > > > > > > > > > For
> > > > > > > > > > > > > > > >> Streams
> > > > > > > > > > > > > > > >> > > > > > here's a
> > > > > > > > > > > > > > > >> > > > > > > > > draft of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> integration
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> plan:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> just FYI
> for
> > > your
> > > > > > > analysis
> > > > > > > > > > on
> > > > > > > > > > > > > > Connect.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> On Tue,
> Jul
> > > 19,
> > > > > 2022
> > > > > > > at
> > > > > > > > > > 10:48 PM
> > > > > > > > > > > > > > Sagar <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > sagarmeansocean@gmail.com
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Hi
> David,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thank
> you
> > > for
> > > > > your
> > > > > > > > > > response.
> > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> reason I
> > > > > > > > > > > > > > > >> > > > thought
> > > > > > > > > > > > > > > >> > > > > > > > > connect can
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > also fit
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> into
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > this new
> > > scheme
> > > > > is
> > > > > > > that
> > > > > > > > > > even
> > > > > > > > > > > > > > today the
> > > > > > > > > > > > > > > >> > > > connect
> > > > > > > > > > > > > > > >> > > > > > uses a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> WorkerCoordinator
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> extending
> > > from
> > > > > > > > > > > > > > AbstractCoordinator to
> > > > > > > > > > > > > > > >> empower
> > > > > > > > > > > > > > > >> > > > > > > > > rebalances of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > tasks/connectors.
> > > > > > > The
> > > > > > > > > > > > > > > >> WorkerCoordinator sets
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocolType()
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> connect
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > and
> uses the
> > > > > > > metadata()
> > > > > > > > > > method
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > > >> plumbing
> > > > > > > > > > > > > > > >> > > > into
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > JoinGroupRequestProtocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > I think
> the
> > > > > changes
> > > > > > > to
> > > > > > > > > > support
> > > > > > > > > > > > > > > >> connect would
> > > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > > >> > > > > > > > > similar at a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > high
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > level
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> changes
> > > in
> > > > > > > streams
> > > > > > > > > > mainly
> > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > >> of the
> > > > > > > > > > > > > > > >> > > > > > Client
> > > > > > > > > > > > > > > >> > > > > > > > > side
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assignors
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > being
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > used in
> > > both.
> > > > > At an
> > > > > > > > > > > > > implementation
> > > > > > > > > > > > > > > >> level, we
> > > > > > > > > > > > > > > >> > > > > > might
> > > > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> make
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > a lot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > changes
> to
> > > get
> > > > > onto
> > > > > > > this
> > > > > > > > > > new
> > > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > > >> > > > protocol
> > > > > > > > > > > > > > > >> > > > > > like
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> enhancing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> JoinGroup
> > > > > > > > > > request/response and
> > > > > > > > > > > > > > > >> SyncGroup and
> > > > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > ConsumerGroupHeartbeat
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > API etc
> > > again on
> > > > > > > similar
> > > > > > > > > > lines
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> streams (or
> > > > > > > > > > > > > > > >> > > > > > there
> > > > > > > > > > > > > > > >> > > > > > > > > might be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> deviations). I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > would
> try to
> > > > > > > perform a
> > > > > > > > > > > > detailed
> > > > > > > > > > > > > > > >> analysis of
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > same
> > > > > > > > > > > > > > > >> > > > > > > > > and we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > separate
> > > > > discussion
> > > > > > > > > > thread for
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > >> as that
> > > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > derail this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > discussion
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > thread.
> Let
> > > me
> > > > > know
> > > > > > > if
> > > > > > > > > > that
> > > > > > > > > > > > > sounds
> > > > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > > > >> > > > you.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thanks!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Sagar.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > On Fri,
> Jul
> > > 15,
> > > > > > > 2022 at
> > > > > > > > > > 5:47
> > > > > > > > > > > > PM
> > > > > > > > > > > > > > David
> > > > > > > > > > > > > > > >> Jacot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Hi
> Sagar,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> Thanks for
> > > > > your
> > > > > > > > > > comments.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 1)
> Yes.
> > > That
> > > > > > > refers to
> > > > > > > > > > > > > > > >> `Assignment#error`.
> > > > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > > > >> > > > > > > > > can
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > mention it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 2) The
> > > idea
> > > > > is to
> > > > > > > > > > > > transition C
> > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > >> his
> > > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > > >> > > > > > > > > assignment
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > his
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > target
> > > > > assignment
> > > > > > > when
> > > > > > > > > > he
> > > > > > > > > > > > can
> > > > > > > > > > > > > > move
> > > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > epoch 3.
> > > > > > > > > > > > > > > >> > > > > > When
> > > > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > happens,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > > > assignment
> > > > > > > is
> > > > > > > > > > updated
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> persisted
> > > > > > > > > > > > > > > >> > > > > > with all
> > > > > > > > > > > > > > > >> > > > > > > > > its
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigned
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> partitions
> > > > > even if
> > > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > >> revoked
> > > > > > > > > > > > > > > >> > > > > > yet. In
> > > > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > words, the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > > > assignment
> > > > > > > > > > becomes
> > > > > > > > > > > > the
> > > > > > > > > > > > > > target
> > > > > > > > > > > > > > > >> > > > > > assignment.
> > > > > > > > > > > > > > > >> > > > > > > > > This is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > basically
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > optimization
> > > > > to
> > > > > > > avoid
> > > > > > > > > > having
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> write all
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > changes to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> examples
> > > are
> > > > > > > based on
> > > > > > > > > > the
> > > > > > > > > > > > > > persisted
> > > > > > > > > > > > > > > >> state
> > > > > > > > > > > > > > > >> > > > so I
> > > > > > > > > > > > > > > >> > > > > > > > > understand
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> confusion.
> > > > > Let me
> > > > > > > see
> > > > > > > > > > if I
> > > > > > > > > > > > can
> > > > > > > > > > > > > > > >> improve
> > > > > > > > > > > > > > > >> > > > this in
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > description.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 3)
> > > Regarding
> > > > > > > Connect,
> > > > > > > > > it
> > > > > > > > > > > > could
> > > > > > > > > > > > > > > >> reuse the
> > > > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > > > >> > > > > > > > > with a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client side
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> assignor
> > > if it
> > > > > > > fits in
> > > > > > > > > > the
> > > > > > > > > > > > > > > >> protocol. The
> > > > > > > > > > > > > > > >> > > > > > assignment
> > > > > > > > > > > > > > > >> > > > > > > > > is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > topicid-partitions +
> > > > > > > > > > > > metadata,
> > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > >> > > > Connect
> > > > > > > > > > > > > > > >> > > > > > fit
> > > > > > > > > > > > > > > >> > > > > > > > > into this?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Best,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > On
> Fri,
> > > Jul
> > > > > 15,
> > > > > > > 2022
> > > > > > > > > at
> > > > > > > > > > 1:55
> > > > > > > > > > > > > PM
> > > > > > > > > > > > > > > >> Sagar <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Hi
> > > David,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Thanks
> > > for
> > > > > the
> > > > > > > KIP.
> > > > > > > > > I
> > > > > > > > > > just
> > > > > > > > > > > > > had
> > > > > > > > > > > > > > > >> minor
> > > > > > > > > > > > > > > >> > > > > > observations:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 1)
> In
> > > the
> > > > > > > Assignment
> > > > > > > > > > Error
> > > > > > > > > > > > > > > >> section in
> > > > > > > > > > > > > > > >> > > > Client
> > > > > > > > > > > > > > > >> > > > > > Side
> > > > > > > > > > > > > > > >> > > > > > > > > mode
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Assignment
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> process,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > you
> > > > > mentioned
> > > > > > > => `In
> > > > > > > > > > this
> > > > > > > > > > > > > > case,
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > > > >> > > > > > side
> > > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> can
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > return
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> error
> > > to the
> > > > > > > group
> > > > > > > > > > > > > > coordinator`.
> > > > > > > > > > > > > > > >> In this
> > > > > > > > > > > > > > > >> > > > > > case are
> > > > > > > > > > > > > > > >> > > > > > > > > you
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > referring to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Assignor
> > > > > > > returning
> > > > > > > > > an
> > > > > > > > > > > > > > > >> AssignmentError
> > > > > > > > > > > > > > > >> > > > that's
> > > > > > > > > > > > > > > >> > > > > > > > > listed down
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > towards
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > end?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > If
> yes,
> > > do
> > > > > you
> > > > > > > think
> > > > > > > > > > it
> > > > > > > > > > > > > would
> > > > > > > > > > > > > > > >> make sense
> > > > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > mention this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > explicitly
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > here?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 2)
> In
> > > the
> > > > > Case
> > > > > > > > > Studies
> > > > > > > > > > > > > > section, I
> > > > > > > > > > > > > > > >> have a
> > > > > > > > > > > > > > > >> > > > > > slight
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> confusion,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > sure
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> if
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> others
> > > have
> > > > > the
> > > > > > > > > same.
> > > > > > > > > > > > > Consider
> > > > > > > > > > > > > > > >> this step:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> When B
> > > > > > > heartbeats,
> > > > > > > > > the
> > > > > > > > > > > > group
> > > > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > transitions him
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > epoch
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 3
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> because
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > B
> has no
> > > > > > > partitions
> > > > > > > > > to
> > > > > > > > > > > > > > revoke. It
> > > > > > > > > > > > > > > >> > > > persists
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > change and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > reply.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > > Group
> > > > > > > (epoch=3)
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > A
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > B
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > C
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > > Target
> > > > > > > > > Assignment
> > > > > > > > > > > > > > (epoch=3)
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > A -
> > > > > > > > > > > > > partitions=[foo-0]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > B -
> > > > > > > > > > > > > partitions=[foo-2]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > C -
> > > > > > > > > > > > > partitions=[foo-1]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > > Member
> > > > > > > > > Assignment
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > A -
> > > > > > > epoch=2,
> > > > > > > > > > > > > > > >> partitions=[foo-0,
> > > > > > > > > > > > > > > >> > > > > > foo-1]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > B -
> > > > > > > epoch=3,
> > > > > > > > > > > > > > > >> partitions=[foo-2]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> -
> > > C -
> > > > > > > epoch=3,
> > > > > > > > > > > > > > > >> partitions=[foo-1]
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> When C
> > > > > > > heartbeats,
> > > > > > > > > it
> > > > > > > > > > > > > > transitions
> > > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > epoch 3
> > > > > > > > > > > > > > > >> > > > > > but
> > > > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> get
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > foo-1
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> yet.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > Here,it's
> > > > > > > mentioned
> > > > > > > > > > that
> > > > > > > > > > > > > > member C
> > > > > > > > > > > > > > > >> can't
> > > > > > > > > > > > > > > >> > > > get
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > foo-1
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > partition
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > yet,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> based
> > > on the
> > > > > > > > > > description
> > > > > > > > > > > > > > above,
> > > > > > > > > > > > > > > >> it seems
> > > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > > >> > > > > > > > > already has
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Do you
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> would be
> > > > > better
> > > > > > > to
> > > > > > > > > > remove
> > > > > > > > > > > > it
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> > > > populate it
> > > > > > > > > > > > > > > >> > > > > > only
> > > > > > > > > > > > > > > >> > > > > > > > > when it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > actually
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> gets
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > I
> see
> > > this
> > > > > in a
> > > > > > > lot
> > > > > > > > > of
> > > > > > > > > > > > other
> > > > > > > > > > > > > > > >> places, so
> > > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > > >> > > > > > I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> understood it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> incorrectly
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > Regarding
> > > > > > > connect ,
> > > > > > > > > it
> > > > > > > > > > > > might
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> out of
> > > > > > > > > > > > > > > >> > > > scope
> > > > > > > > > > > > > > > >> > > > > > of
> > > > > > > > > > > > > > > >> > > > > > > > > this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > discussion,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > from
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> what I
> > > > > > > understood it
> > > > > > > > > > would
> > > > > > > > > > > > > > > >> probably be
> > > > > > > > > > > > > > > >> > > > > > running in
> > > > > > > > > > > > > > > >> > > > > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> assignor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > mode
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> even on
> > > the
> > > > > new
> > > > > > > > > > rebalance
> > > > > > > > > > > > > > > >> protocol as it
> > > > > > > > > > > > > > > >> > > > has
> > > > > > > > > > > > > > > >> > > > > > its
> > > > > > > > > > > > > > > >> > > > > > > > > own
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Custom
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > Assignors(Eager
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > and
> > > > > > > > > > > > IncrementalCooperative).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Thanks!
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> Sagar.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > On
> Fri,
> > > Jul
> > > > > 15,
> > > > > > > 2022
> > > > > > > > > > at
> > > > > > > > > > > > 5:00
> > > > > > > > > > > > > > PM
> > > > > > > > > > > > > > > >> David
> > > > > > > > > > > > > > > >> > > > Jacot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> Thanks
> > > > > > > Hector! Our
> > > > > > > > > > goal
> > > > > > > > > > > > is
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> move
> > > > > > > > > > > > > > > >> > > > forward
> > > > > > > > > > > > > > > >> > > > > > with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > specialized API
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > instead of
> > > > > > > relying
> > > > > > > > > > on
> > > > > > > > > > > > one
> > > > > > > > > > > > > > > >> generic API.
> > > > > > > > > > > > > > > >> > > > For
> > > > > > > > > > > > > > > >> > > > > > > > > Connect, we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > can apply
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> exact
> > > same
> > > > > > > pattern
> > > > > > > > > > and
> > > > > > > > > > > > > > > >> reuse/share the
> > > > > > > > > > > > > > > >> > > > core
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > implementation on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> server
> > > > > side.
> > > > > > > For
> > > > > > > > > the
> > > > > > > > > > > > > schema
> > > > > > > > > > > > > > > >> registry, I
> > > > > > > > > > > > > > > >> > > > > > think
> > > > > > > > > > > > > > > >> > > > > > > > > that we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consider
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > having a
> > > > > > > tailored
> > > > > > > > > > API to
> > > > > > > > > > > > > do
> > > > > > > > > > > > > > > >> simple
> > > > > > > > > > > > > > > >> > > > > > > > > membership/leader
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > election.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> Best,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > On
> > > Fri,
> > > > > Jul
> > > > > > > 15,
> > > > > > > > > > 2022 at
> > > > > > > > > > > > > > 10:22
> > > > > > > > > > > > > > > >> AM Ismael
> > > > > > > > > > > > > > > >> > > > > > Juma <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ismael@juma.me.uk
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > Three
> > > > > quick
> > > > > > > > > > comments:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> 1.
> > > > > Regarding
> > > > > > > > > > > > > > > >> java.util.regex.Pattern
> > > > > > > > > > > > > > > >> > > > vs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > com.google.re2j.Pattern,
> > > > > > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > document the
> > > > > > > > > > > > differences
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > > >> > > > detail
> > > > > > > > > > > > > > > >> > > > > > before
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deciding
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > way
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> or
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > another.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> That
> > > > > said,
> > > > > > > if
> > > > > > > > > > people
> > > > > > > > > > > > > pass
> > > > > > > > > > > > > > > >> > > > > > > > > java.util.regex.Pattern,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expect
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > their
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > semantics
> > > > > > > to be
> > > > > > > > > > > > honored.
> > > > > > > > > > > > > > If
> > > > > > > > > > > > > > > >> we are
> > > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > > >> > > > > > > > > something
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > different,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> then
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > should
> > > > > > > consider
> > > > > > > > > > adding
> > > > > > > > > > > > > an
> > > > > > > > > > > > > > > >> overload
> > > > > > > > > > > > > > > >> > > > with
> > > > > > > > > > > > > > > >> > > > > > our own
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Pattern
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > class
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > (I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > don't
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > think
> > > > > we'd
> > > > > > > want
> > > > > > > > > to
> > > > > > > > > > > > > expose
> > > > > > > > > > > > > > > >> re2j's at
> > > > > > > > > > > > > > > >> > > > this
> > > > > > > > > > > > > > > >> > > > > > > > > point).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> 2.
> > > > > Regarding
> > > > > > > > > topic
> > > > > > > > > > > > ids,
> > > > > > > > > > > > > > any
> > > > > > > > > > > > > > > >> major new
> > > > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > > > >> > > > > > > > > should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > integrate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> it
> > > and
> > > > > > > should
> > > > > > > > > > handle
> > > > > > > > > > > > the
> > > > > > > > > > > > > > topic
> > > > > > > > > > > > > > > >> > > > > > recreation case
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > correctly.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That's
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > main
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > part we
> > > > > > > need to
> > > > > > > > > > > > handle.
> > > > > > > > > > > > > I
> > > > > > > > > > > > > > > >> agree with
> > > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > > >> > > > > > > > > that we'd
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > want to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > add
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> to
> > > the
> > > > > > > relevant
> > > > > > > > > > > > > protocols
> > > > > > > > > > > > > > > >> that don't
> > > > > > > > > > > > > > > >> > > > > > have it
> > > > > > > > > > > > > > > >> > > > > > > > > yet and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > that we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > can
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> probably
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > focus
> > > > > on the
> > > > > > > > > > internals
> > > > > > > > > > > > > > versus
> > > > > > > > > > > > > > > >> adding
> > > > > > > > > > > > > > > >> > > > new
> > > > > > > > > > > > > > > >> > > > > > APIs
> > > > > > > > > > > > > > > >> > > > > > > > > to the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > (unless
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> we
> > > find
> > > > > that
> > > > > > > > > > adding
> > > > > > > > > > > > new
> > > > > > > > > > > > > > APIs
> > > > > > > > > > > > > > > >> is
> > > > > > > > > > > > > > > >> > > > required
> > > > > > > > > > > > > > > >> > > > > > for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> reasonable
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> semantics).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> 3.
> > > I am
> > > > > > > still
> > > > > > > > > not
> > > > > > > > > > sure
> > > > > > > > > > > > > > about
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > storing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> configs.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It's
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > powerful for
> > > > > > > > > > configs
> > > > > > > > > > > > to
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> > > > centralized
> > > > > > > > > > > > > > > >> > > > > > in the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> metadata
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > log for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> various
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > reasons
> > > > > > > > > > (auditability,
> > > > > > > > > > > > > > > >> visibility,
> > > > > > > > > > > > > > > >> > > > > > consistency,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> etc.).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> Similarly, I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > am
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> sure
> > > > > about
> > > > > > > > > > > > automatically
> > > > > > > > > > > > > > > >> deleting
> > > > > > > > > > > > > > > >> > > > > > configs in a
> > > > > > > > > > > > > > > >> > > > > > > > > way
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > they
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cannot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > recovered. A
> > > > > > > > > good
> > > > > > > > > > > > > property
> > > > > > > > > > > > > > > >> for modern
> > > > > > > > > > > > > > > >> > > > > > systems
> > > > > > > > > > > > > > > >> > > > > > > > > is to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > minimize
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > number
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > unrecoverable
> > > > > > > > > data
> > > > > > > > > > > > loss
> > > > > > > > > > > > > > > >> scenarios.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > Ismael
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> On
> > > Wed,
> > > > > Jul
> > > > > > > 13,
> > > > > > > > > > 2022
> > > > > > > > > > > > at
> > > > > > > > > > > > > > 3:47
> > > > > > > > > > > > > > > >> PM David
> > > > > > > > > > > > > > > >> > > > > > Jacot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > Thanks
> > > > > > > > > > Guozhang. My
> > > > > > > > > > > > > > answers
> > > > > > > > > > > > > > > >> are
> > > > > > > > > > > > > > > >> > > > below:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > 1)
> > > > > the
> > > > > > > > > > migration
> > > > > > > > > > > > > path,
> > > > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > flag
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > to
> > > > > > > enable
> > > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> both
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > protocols /
> > > > > > > > > > rpcs
> > > > > > > > > > > > and
> > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> same
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > group.
> > > > > > > How
> > > > > > > > > the
> > > > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> using
> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > new
> > > > > > > protocol
> > > > > > > > > > is
> > > > > > > > > > > > > > something
> > > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > Noted. I
> > > > > > > just
> > > > > > > > > > > > > published
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > >> new
> > > > > > > > > > > > > > > >> > > > version
> > > > > > > > > > > > > > > >> > > > > > of KIP
> > > > > > > > > > > > > > > >> > > > > > > > > which
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > includes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> more
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > details
> > > > > > > about
> > > > > > > > > > this.
> > > > > > > > > > > > > See
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > "Supporting
> > > > > > > > > > > > > > > >> > > > > > > > > Online
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> Upgrade"
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > and
> > > > > the
> > > > > > > > > > > > > "Compatibility,
> > > > > > > > > > > > > > > >> > > > Deprecation,
> > > > > > > > > > > > > > > >> > > > > > and
> > > > > > > > > > > > > > > >> > > > > > > > > Migration
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Plan". I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > I
> > > > > have to
> > > > > > > > > think
> > > > > > > > > > > > > through
> > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > >> few cases
> > > > > > > > > > > > > > > >> > > > > > now but
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overall idea
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > mechanism
> > > > > > > > > > should be
> > > > > > > > > > > > > > > >> understandable.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > 2)
> > > > > the
> > > > > > > usage
> > > > > > > > > > of
> > > > > > > > > > > > > topic
> > > > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> used
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > as
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > part of
> > > > > > > RPCs
> > > > > > > > > > and
> > > > > > > > > > > > > admin
> > > > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> public
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > APIs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > to
> > > > > > > consumers
> > > > > > > > > > yet.
> > > > > > > > > > > > I
> > > > > > > > > > > > > > think
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > client
> > > > > > > to be
> > > > > > > > > > > > > > maintaining
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > leverage
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > on
> > > > > all
> > > > > > > the
> > > > > > > > > > > > augmented
> > > > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> ids;
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > secondly,
> > > > > > > > > > should
> > > > > > > > > > > > we
> > > > > > > > > > > > > > ever
> > > > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > public
> > > > > > > APIs
> > > > > > > > > as
> > > > > > > > > > > > well
> > > > > > > > > > > > > > (both
> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > rebalance
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > listener for
> > > > > > > > > > cases
> > > > > > > > > > > > > > like
> > > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > deletion-and-recreation).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > a)
> > > > > > > Assuming
> > > > > > > > > > that we
> > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > >> include
> > > > > > > > > > > > > > > >> > > > > > converting
> > > > > > > > > > > > > > > >> > > > > > > > > all
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > offsets
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> related
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > RPCs
> > > > > to
> > > > > > > using
> > > > > > > > > > topic
> > > > > > > > > > > > > ids
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > > >> > > > KIP,
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> able
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > fully
> > > > > > > operate
> > > > > > > > > > with
> > > > > > > > > > > > > topic
> > > > > > > > > > > > > > > >> ids. That
> > > > > > > > > > > > > > > >> > > > > > being
> > > > > > > > > > > > > > > >> > > > > > > > > said, it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > still has
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> provide
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > the
> > > > > topics
> > > > > > > > > > names in
> > > > > > > > > > > > > > various
> > > > > > > > > > > > > > > >> APIs so
> > > > > > > > > > > > > > > >> > > > > > having a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> mapping
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > seems
> > > > > > > > > > inevitable to
> > > > > > > > > > > > > me.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > b) I
> > > > > don't
> > > > > > > > > have
> > > > > > > > > > a
> > > > > > > > > > > > > strong
> > > > > > > > > > > > > > > >> opinion on
> > > > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > > > >> > > > > > > > > Here I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > wonder if
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > goes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > beyond the
> > > > > > > > > > scope of
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > > >> KIP. I
> > > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > > >> > > > > > rather
> > > > > > > > > > > > > > > >> > > > > > > > > focus
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> internals
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > here
> > > > > and
> > > > > > > we
> > > > > > > > > can
> > > > > > > > > > > > > consider
> > > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > > >> > > > > > separately if
> > > > > > > > > > > > > > > >> > > > > > > > > we see
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > value in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> doing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > Coming
> > > > > > > back to
> > > > > > > > > > > > > Ismael's
> > > > > > > > > > > > > > > >> point about
> > > > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > > > >> > > > > > > > > topic ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> ConsumerGroupHeartbeatRequest, I
> > > > > > > > > > > > > > > >> > > > think
> > > > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > there
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> advantage
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > favour of
> > > > > > > it.
> > > > > > > > > > The
> > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> will
> > > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opportunity
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > the
> > > > > topics
> > > > > > > > > > exists
> > > > > > > > > > > > > before
> > > > > > > > > > > > > > > >> passing
> > > > > > > > > > > > > > > >> > > > them
> > > > > > > > > > > > > > > >> > > > > > into
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > rebalance
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > protocol.
> > > > > > > > > > Obviously,
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > coordinator
> > > > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > > > >> > > > > > > > > also
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> notice
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > does
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > not
> > > > > really
> > > > > > > > > have
> > > > > > > > > > a
> > > > > > > > > > > > way
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> reject an
> > > > > > > > > > > > > > > >> > > > > > invalid
> > > > > > > > > > > > > > > >> > > > > > > > > topic in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> response.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > I'm
> > > > > > > agreeing
> > > > > > > > > > with
> > > > > > > > > > > > > > David
> > > > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> necessary
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > deprecate
> > > > > > > > > the
> > > > > > > > > > > > > > subscribe
> > > > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > Pattern
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > while
> > > > > > > just
> > > > > > > > > > > > > documenting
> > > > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
> by
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > server.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > Since
> > > > > > > the
> > > > > > > > > > > > > incompatible
> > > > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > overloaded
> > > > > > > > > > > > `String`
> > > > > > > > > > > > > > based
> > > > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > various
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > invalid
> > > > > > > > > > regexes.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > That
> > > > > could
> > > > > > > > > > work. I
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> look at
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > differences
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > between
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > two
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > engines to
> > > > > > > > > > better
> > > > > > > > > > > > > > > >> understand the
> > > > > > > > > > > > > > > >> > > > > > potential
> > > > > > > > > > > > > > > >> > > > > > > > > issues.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> My
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > understanding
> > > > > is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > that
> > > > > would
> > > > > > > > > work
> > > > > > > > > > for
> > > > > > > > > > > > > all
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> basic
> > > > > > > > > > > > > > > >> > > > > > regular
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > expressions. The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > differences
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > between
> > > > > > > the
> > > > > > > > > two
> > > > > > > > > > are
> > > > > > > > > > > > > > mainly
> > > > > > > > > > > > > > > >> about
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > various
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> character
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> classes. I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > wonder
> > > > > > > what
> > > > > > > > > > other
> > > > > > > > > > > > > people
> > > > > > > > > > > > > > > >> think
> > > > > > > > > > > > > > > >> > > > about
> > > > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > Best,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > On
> > > > > Tue,
> > > > > > > Jul
> > > > > > > > > 12,
> > > > > > > > > > 2022
> > > > > > > > > > > > > at
> > > > > > > > > > > > > > > >> 11:28 PM
> > > > > > > > > > > > > > > >> > > > > > Guozhang
> > > > > > > > > > > > > > > >> > > > > > > > > Wang <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > wangguoz@gmail.com
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > Thanks
> > > > > > > > > David!
> > > > > > > > > > I
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > >> the high
> > > > > > > > > > > > > > > >> > > > > > level
> > > > > > > > > > > > > > > >> > > > > > > > > there are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > two meta
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> points we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> need
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > to
> > > > > > > > > concretize
> > > > > > > > > > a
> > > > > > > > > > > > bit
> > > > > > > > > > > > > > more:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > 1)
> > > > > the
> > > > > > > > > > migration
> > > > > > > > > > > > > path,
> > > > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > flag
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > to
> > > > > > > enable
> > > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> both
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > protocols /
> > > > > > > > > > rpcs
> > > > > > > > > > > > and
> > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> same
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > group.
> > > > > > > How
> > > > > > > > > the
> > > > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> using
> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > new
> > > > > > > protocol
> > > > > > > > > > is
> > > > > > > > > > > > > > something
> > > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > 2)
> > > > > the
> > > > > > > usage
> > > > > > > > > > of
> > > > > > > > > > > > > topic
> > > > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> used
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > as
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > part of
> > > > > > > RPCs
> > > > > > > > > > and
> > > > > > > > > > > > > admin
> > > > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> public
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > APIs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > to
> > > > > > > consumers
> > > > > > > > > > yet.
> > > > > > > > > > > > I
> > > > > > > > > > > > > > think
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > client
> > > > > > > to be
> > > > > > > > > > > > > > maintaining
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > leverage
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > on
> > > > > all
> > > > > > > the
> > > > > > > > > > > > augmented
> > > > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> ids;
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > secondly,
> > > > > > > > > > should
> > > > > > > > > > > > we
> > > > > > > > > > > > > > ever
> > > > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > public
> > > > > > > APIs
> > > > > > > > > as
> > > > > > > > > > > > well
> > > > > > > > > > > > > > (both
> > > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > rebalance
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > listener for
> > > > > > > > > > cases
> > > > > > > > > > > > > > like
> > > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > deletion-and-recreation).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > I'm
> > > > > > > agreeing
> > > > > > > > > > with
> > > > > > > > > > > > > > David
> > > > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> necessary
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > deprecate
> > > > > > > > > the
> > > > > > > > > > > > > > subscribe
> > > > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > Pattern
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > while
> > > > > > > just
> > > > > > > > > > > > > documenting
> > > > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected
> by
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > server.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > Since
> > > > > > > the
> > > > > > > > > > > > > incompatible
> > > > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > overloaded
> > > > > > > > > > > > `String`
> > > > > > > > > > > > > > based
> > > > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > various
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > invalid
> > > > > > > > > > regexes.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > Guozhang
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > On
> > > > > Tue,
> > > > > > > Jul
> > > > > > > > > > 12,
> > > > > > > > > > > > 2022
> > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > >> 5:23 AM
> > > > > > > > > > > > > > > >> > > > > > David Jacot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > Hi
> > > > > > > Ismael,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > Thanks for
> > > > > > > > > > your
> > > > > > > > > > > > > > > >> feedback. Let
> > > > > > > > > > > > > > > >> > > > me
> > > > > > > > > > > > > > > >> > > > > > answer
> > > > > > > > > > > > > > > >> > > > > > > > > your
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > questions
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > inline.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 1. I
> > > > > > > > > think
> > > > > > > > > > > > it's
> > > > > > > > > > > > > > > >> premature to
> > > > > > > > > > > > > > > >> > > > > > talk about
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> target
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > versions
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > deprecation
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > removal
> > > > > > > > > > of the
> > > > > > > > > > > > > > > >> existing group
> > > > > > > > > > > > > > > >> > > > > > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Unlike
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > KRaft,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > affects a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > core
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > client
> > > > > > > > > > > > protocol
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> hence
> > > > > > > > > > > > > > > >> > > > > > > > > deprecation/removal
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > heavily
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > dependent
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > how
> > > > > > > > > > quickly
> > > > > > > > > > > > > > > >> applications
> > > > > > > > > > > > > > > >> > > > migrate
> > > > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > That
> > > > > > > makes
> > > > > > > > > > > > sense.
> > > > > > > > > > > > > I
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > >> > > > remove it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 2.
> > > > > > > The
> > > > > > > > > KIP
> > > > > > > > > > > > says
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > >> intend to
> > > > > > > > > > > > > > > >> > > > > > release
> > > > > > > > > > > > > > > >> > > > > > > > > this in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > 4.x, but
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wasn't
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> made
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > clear
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > why. If
> > > > > > > > > we
> > > > > > > > > > > > added
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > >> as a
> > > > > > > > > > > > > > > >> > > > way to
> > > > > > > > > > > > > > > >> > > > > > > > > estimate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we'd
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> deprecate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > remove
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > the
> > > > > > > > > group
> > > > > > > > > > > > > > protocol, I
> > > > > > > > > > > > > > > >> also
> > > > > > > > > > > > > > > >> > > > > > suggest
> > > > > > > > > > > > > > > >> > > > > > > > > removing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this part.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Let me
> > > > > > > > > > explain
> > > > > > > > > > > > my
> > > > > > > > > > > > > > > >> reasoning. As
> > > > > > > > > > > > > > > >> > > > > > > > > explained, I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> plan
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rewrite
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > coordinator
> > > > > > > > > > in
> > > > > > > > > > > > > Java
> > > > > > > > > > > > > > > >> while we
> > > > > > > > > > > > > > > >> > > > > > implement
> > > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> This
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > that
> > > > > > > the
> > > > > > > > > > > > internals
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > >> be
> > > > > > > > > > > > > > > >> > > > slightly
> > > > > > > > > > > > > > > >> > > > > > > > > different
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> threading
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > model).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > Therefore, I
> > > > > > > > > > > > > wanted
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> tighten
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > switch from
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the old
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > coordinator
> > > > > > > > > > to
> > > > > > > > > > > > the
> > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > to a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > major
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> release.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > alternative
> > > > > > > > > > > > would
> > > > > > > > > > > > > > be to
> > > > > > > > > > > > > > > >> use a
> > > > > > > > > > > > > > > >> > > > flag
> > > > > > > > > > > > > > > >> > > > > > to do
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instead
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > relying
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > on the
> > > > > > > > > > software
> > > > > > > > > > > > > > upgrade.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 3.
> > > > > > > We
> > > > > > > > > > need to
> > > > > > > > > > > > > > flesh
> > > > > > > > > > > > > > > >> out the
> > > > > > > > > > > > > > > >> > > > > > details of
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > migration
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> story.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> sounds
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > like
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > we're
> > > > > > > > > > saying
> > > > > > > > > > > > we
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > >> support
> > > > > > > > > > > > > > > >> > > > > > online
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> migrations.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> correct?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > We
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > explain
> > > > > > > > > > this
> > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> detail. It
> > > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > > >> > > > > > also
> > > > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> separate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > KIP,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > if
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > it's
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > easier.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Yes,
> > > > > > > we
> > > > > > > > > will
> > > > > > > > > > > > > support
> > > > > > > > > > > > > > > >> online
> > > > > > > > > > > > > > > >> > > > > > migrations
> > > > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > a
> > > > > > > group
> > > > > > > > > > using
> > > > > > > > > > > > the
> > > > > > > > > > > > > > old
> > > > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > > > >> > > > > > > > > able to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Let me
> > > > > > > > > > briefly
> > > > > > > > > > > > > > explain
> > > > > > > > > > > > > > > >> how that
> > > > > > > > > > > > > > > >> > > > > > will work
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> though.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > It is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> basically
> > > a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > four
> > > > > > > step
> > > > > > > > > > > > process:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > 1. The
> > > > > > > > > > cluster
> > > > > > > > > > > > > must
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> > > > upgraded or
> > > > > > > > > > > > > > > >> > > > > > > > > rolled to a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > software
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> supporting
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > new
> > > > > > > group
> > > > > > > > > > > > > > coordinator.
> > > > > > > > > > > > > > > >> Both the
> > > > > > > > > > > > > > > >> > > > > > old and
> > > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > will
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > support
> > > > > > > > > the
> > > > > > > > > > old
> > > > > > > > > > > > > > > >> protocol and
> > > > > > > > > > > > > > > >> > > > rely
> > > > > > > > > > > > > > > >> > > > > > on the
> > > > > > > > > > > > > > > >> > > > > > > > > same
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > persisted
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> metadata
> > > so
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > they
> > > > > > > can
> > > > > > > > > > work
> > > > > > > > > > > > > > together.
> > > > > > > > > > > > > > > >> This
> > > > > > > > > > > > > > > >> > > > point
> > > > > > > > > > > > > > > >> > > > > > is an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> offline
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> migration.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > We
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> cannot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > do
> > > > > > > this
> > > > > > > > > one
> > > > > > > > > > live
> > > > > > > > > > > > > > > >> because it
> > > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > > >> > > > > > require
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > shutting down
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> current
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > coordinator
> > > > > > > > > > and
> > > > > > > > > > > > > > > >> starting up the
> > > > > > > > > > > > > > > >> > > > > > new one
> > > > > > > > > > > > > > > >> > > > > > > > > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cause
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > unavailabilities.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > 2. The
> > > > > > > > > > cluster's
> > > > > > > > > > > > > > > >> metadata
> > > > > > > > > > > > > > > >> > > > > > version/IBP
> > > > > > > > > > > > > > > >> > > > > > > > > must be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> X
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> order
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > to
> > > > > > > enable
> > > > > > > > > > the
> > > > > > > > > > > > new
> > > > > > > > > > > > > > > >> protocol.
> > > > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > > > >> > > > > > cannot
> > > > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 1)
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > terminated
> > > > > > > > > > > > because
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> old
> > > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > doesn't
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > support
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > protocol.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > 3. The
> > > > > > > > > > consumers
> > > > > > > > > > > > > > must be
> > > > > > > > > > > > > > > >> > > > upgraded
> > > > > > > > > > > > > > > >> > > > > > to a
> > > > > > > > > > > > > > > >> > > > > > > > > version
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > supporting
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> online
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > migration
> > > > > > > > > > (must
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > > >> KIP-792).
> > > > > > > > > > > > > > > >> > > > If
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > already
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > there.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > Nothing
> > > > > > > > > > must be
> > > > > > > > > > > > > > done at
> > > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > > >> > > > point.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > 4. The
> > > > > > > > > > consumers
> > > > > > > > > > > > > > must
> > > > > > > > > > > > > > > >> be rolled
> > > > > > > > > > > > > > > >> > > > > > with the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> turned
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > on.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > consumer
> > > > > > > > > > group
> > > > > > > > > > > > is
> > > > > > > > > > > > > > > >> automatically
> > > > > > > > > > > > > > > >> > > > > > > > > converted when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > using
> > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > > >> joins
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > group.
> > > > > > > > > > > > > > > >> > > > > > > > > While
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > members
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > using
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > old
> > > > > > > > > > protocol are
> > > > > > > > > > > > > > being
> > > > > > > > > > > > > > > >> > > > upgraded,
> > > > > > > > > > > > > > > >> > > > > > the old
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> proxied
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > into
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > new
> > > > > > > one.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Let me
> > > > > > > > > > clarify
> > > > > > > > > > > > all
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > >> this in
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > KIP.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 4.
> > > > > > > I am
> > > > > > > > > > happy
> > > > > > > > > > > > > > that we
> > > > > > > > > > > > > > > >> are
> > > > > > > > > > > > > > > >> > > > > > pushing the
> > > > > > > > > > > > > > > >> > > > > > > > > pattern
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> subscriptions
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > server,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > but
> > > > > > > it
> > > > > > > > > > seems
> > > > > > > > > > > > > like
> > > > > > > > > > > > > > > >> there
> > > > > > > > > > > > > > > >> > > > could be
> > > > > > > > > > > > > > > >> > > > > > some
> > > > > > > > > > > > > > > >> > > > > > > > > tricky
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> compatibility
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > issues.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > Will
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > have a
> > > > > > > > > > > > mechanism
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > >> users to
> > > > > > > > > > > > > > > >> > > > > > detect
> > > > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > update
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> their
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > regex
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > before
> > > > > > > > > > > > switching
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> the new
> > > > > > > > > > > > > > > >> > > > > > protocol?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > I
> > > > > > > think
> > > > > > > > > > that I
> > > > > > > > > > > > am
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > bit
> > > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > > >> > > > > > optimistic
> > > > > > > > > > > > > > > >> > > > > > > > > than
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> you
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> point. I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > believe
> > > > > > > > > > that the
> > > > > > > > > > > > > > > >> majority of
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > cases
> > > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> simple
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > regexes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > which
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > work
> > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > > > new
> > > > > > > > > > > > > > > >> engine. The
> > > > > > > > > > > > > > > >> > > > > > > > > coordinator will
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > verify
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > regex
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> anyway
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > and
> > > > > > > reject
> > > > > > > > > > the
> > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> if the
> > > > > > > > > > > > > > > >> > > > > > regex is
> > > > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> valid.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Coming
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > back
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > migration
> > > > > > > > > > path,
> > > > > > > > > > > > in
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> worst
> > > > > > > > > > > > > > > >> > > > case,
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > joining
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > the
> > > > > > > group
> > > > > > > > > > will
> > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> rejected.
> > > > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > > > >> > > > > > should
> > > > > > > > > > > > > > > >> > > > > > > > > be used
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > as the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> last
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > defence, I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > would
> > > > > > > say.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > One
> > > > > > > way
> > > > > > > > > for
> > > > > > > > > > > > > > customers to
> > > > > > > > > > > > > > > >> > > > validate
> > > > > > > > > > > > > > > >> > > > > > their
> > > > > > > > > > > > > > > >> > > > > > > > > regex
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> upgrading
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> their
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > prod
> > > > > > > would
> > > > > > > > > > be to
> > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > >> them with
> > > > > > > > > > > > > > > >> > > > > > another
> > > > > > > > > > > > > > > >> > > > > > > > > group.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> For
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instance,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > could
> > > > > > > be
> > > > > > > > > > done
> > > > > > > > > > > > in a
> > > > > > > > > > > > > > > >> pre-prod
> > > > > > > > > > > > > > > >> > > > > > environment.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Another
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > way
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> would be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > extend the
> > > > > > > > > > > > > > > >> consumer-group tool
> > > > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > provide a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> regex
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validation
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > mechanism.
> > > > > > > > > > Would
> > > > > > > > > > > > > > this be
> > > > > > > > > > > > > > > >> > > > enough in
> > > > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> opinion?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 5.
> > > > > > > > > > Related to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > last
> > > > > > > > > > > > > > > >> > > > question,
> > > > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> Java
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> allow
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > users
> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > stick
> > > > > > > > > > with the
> > > > > > > > > > > > > > > >> current regex
> > > > > > > > > > > > > > > >> > > > > > engine for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > compatibility
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> reasons?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> For
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > example,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > it
> > > > > > > may
> > > > > > > > > be
> > > > > > > > > > > > handy
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> keep using
> > > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > > >> > > > > > > > > based
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > regex at
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > keep
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > migrations
> > > > > > > > > > > > > simple
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > >> then
> > > > > > > > > > > > > > > >> > > > > > migrate to
> > > > > > > > > > > > > > > >> > > > > > > > > server
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > based
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regexes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > as a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > second
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > step.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > I
> > > > > > > > > understand
> > > > > > > > > > > > your
> > > > > > > > > > > > > > point
> > > > > > > > > > > > > > > >> but I
> > > > > > > > > > > > > > > >> > > > am
> > > > > > > > > > > > > > > >> > > > > > > > > concerned that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > allow
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> users
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > to
> > > > > > > > > actually
> > > > > > > > > > stay
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> this mode.
> > > > > > > > > > > > > > > >> > > > That
> > > > > > > > > > > > > > > >> > > > > > > > > would go
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > against our
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> goal
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > simplifying
> > > > > > > > > > the
> > > > > > > > > > > > > > client
> > > > > > > > > > > > > > > >> because
> > > > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > continue
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> monitoring
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > the
> > > > > > > > > > metadata on
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> client
> > > > > > > > > > > > > > > >> > > > side. I
> > > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > > >> > > > > > > > > rather
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > not do
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 6.
> > > > > > > When
> > > > > > > > > > we say
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > > > >> > > > > > > > > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> will
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > responsible
> > > > > for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > storing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > the
> > > > > > > > > > > > > configurations
> > > > > > > > > > > > > > > >> and that
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> configurations
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> deleted
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > group is
> > > > > > > > > > > > > deleted.
> > > > > > > > > > > > > > > >> Will a
> > > > > > > > > > > > > > > >> > > > > > transition to
> > > > > > > > > > > > > > > >> > > > > > > > > DEAD
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > trigger
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > deletion
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > configurations?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > That's
> > > > > > > > > > right.
> > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> > > > configurations
> > > > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> deleted
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
> is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > deleted.
> > > > > > > > > > They go
> > > > > > > > > > > > > > > >> together.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 7.
> > > > > > > Will
> > > > > > > > > > the
> > > > > > > > > > > > > > choice to
> > > > > > > > > > > > > > > >> store
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > configs in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > make
> > > > > it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > harder
> > > > > > > > > to
> > > > > > > > > > list
> > > > > > > > > > > > > all
> > > > > > > > > > > > > > > >> cluster
> > > > > > > > > > > > > > > >> > > > > > configs and
> > > > > > > > > > > > > > > >> > > > > > > > > their
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > values?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > I
> > > > > > > don't
> > > > > > > > > > think
> > > > > > > > > > > > so.
> > > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > > >> > > > > > > > > configurations are
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > overrides
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> cluster
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > configs.
> > > > > > > > > If
> > > > > > > > > > you
> > > > > > > > > > > > > > want to
> > > > > > > > > > > > > > > >> know
> > > > > > > > > > > > > > > >> > > > all
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > overrides
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > you
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > to
> > > > > > > ask all
> > > > > > > > > > the
> > > > > > > > > > > > > group
> > > > > > > > > > > > > > > >> > > > coordinators.
> > > > > > > > > > > > > > > >> > > > > > You
> > > > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > rely on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> metadata
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> log
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > for
> > > > > > > > > > instance.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 8.
> > > > > > > How
> > > > > > > > > > would
> > > > > > > > > > > > > > someone
> > > > > > > > > > > > > > > >> > > > configure a
> > > > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > > > >> > > > > > > > > before
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > starting
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > consumers?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > we
> > > > > > > > > > considered
> > > > > > > > > > > > > > > >> allowing the
> > > > > > > > > > > > > > > >> > > > > > explicit
> > > > > > > > > > > > > > > >> > > > > > > > > creation
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > groups?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > > > > > Alternatively,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > configs
> > > > > > > > > > could
> > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> decoupled
> > > > > > > > > > > > > > > >> > > > from
> > > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > lifecycle.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Yes.
> > > > > > > The
> > > > > > > > > > group
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> > > > > > automatically
> > > > > > > > > > > > > > > >> > > > > > > > > created in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> However,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > the
> > > > > > > > > configs
> > > > > > > > > > will
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > >> lost after
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > retention
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > period of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > passes.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 9.
> > > > > > > Will
> > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > Consumer.subscribe
> > > > > > > > > > > > > > > >> > > > > > method
> > > > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> client
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > still
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> take
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` of
> > > > > > > > > > > > > > > >> > > > do
> > > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > introduce
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> overload?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > That's a
> > > > > > > > > > very
> > > > > > > > > > > > > group
> > > > > > > > > > > > > > > >> question. I
> > > > > > > > > > > > > > > >> > > > > > forgot
> > > > > > > > > > > > > > > >> > > > > > > > > about
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > one.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > As
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> `java.util.regex.Pattern` is
> > > > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > > > >> > > > > > fully
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> compatible
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > engine
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > we
> > > > > > > plan to
> > > > > > > > > > use,
> > > > > > > > > > > > it
> > > > > > > > > > > > > > > >> might be
> > > > > > > > > > > > > > > >> > > > better
> > > > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > deprecate
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > it and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> use an
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > overload
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > which
> > > > > > > > > takes
> > > > > > > > > > a
> > > > > > > > > > > > > > string.
> > > > > > > > > > > > > > > >> We would
> > > > > > > > > > > > > > > >> > > > > > rely on
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> server
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > validation.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > During the
> > > > > > > > > > > > > > migration, I
> > > > > > > > > > > > > > > >> think
> > > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> still
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > try to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> toString
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > regex
> > > > > > > and
> > > > > > > > > > use
> > > > > > > > > > > > it.
> > > > > > > > > > > > > > That
> > > > > > > > > > > > > > > >> should
> > > > > > > > > > > > > > > >> > > > > > work, I
> > > > > > > > > > > > > > > >> > > > > > > > > think, in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> majority
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > cases.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 10.
> > > > > > > I
> > > > > > > > > > agree
> > > > > > > > > > > > with
> > > > > > > > > > > > > > > >> Justine
> > > > > > > > > > > > > > > >> > > > that we
> > > > > > > > > > > > > > > >> > > > > > > > > should be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > clearer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > about
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> reason
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > switch
> > > > > > > > > to
> > > > > > > > > > > > > > > >> > > > IBP/metadata.version
> > > > > > > > > > > > > > > >> > > > > > from the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Maybe
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> mean
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > can
> > > > > > > > > > switch the
> > > > > > > > > > > > > > > >> default for
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > feature
> > > > > > > > > > > > > > > >> > > > > > > > > flag
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > true
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> based
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > metadata.version
> > > > > > > > > > > > > > once
> > > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > > >> > > > want to
> > > > > > > > > > > > > > > >> > > > > > make
> > > > > > > > > > > > > > > >> > > > > > > > > it the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > default.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > My
> > > > > > > plan
> > > > > > > > > was
> > > > > > > > > > to
> > > > > > > > > > > > use
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > >> feature
> > > > > > > > > > > > > > > >> > > > > > flag
> > > > > > > > > > > > > > > >> > > > > > > > > mainly
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > during the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > development
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > phase. I
> > > > > > > > > > should
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > >> > > > mentioned
> > > > > > > > > > > > > > > >> > > > > > it, I
> > > > > > > > > > > > > > > >> > > > > > > > > think,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > because
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > could
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> use
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > an
> > > > > > > > > internal
> > > > > > > > > > > > config
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > >> it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 11.
> > > > > > > Some
> > > > > > > > > > of
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> protocol APIs
> > > > > > > > > > > > > > > >> > > > > > don't
> > > > > > > > > > > > > > > >> > > > > > > > > mention
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > required
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ACLs,
> it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > would
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > good to
> > > > > > > > > > add
> > > > > > > > > > > > that
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > >> > > > consistency.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Noted.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 12.
> > > > > > > It
> > > > > > > > > is
> > > > > > > > > > a
> > > > > > > > > > > > bit
> > > > > > > > > > > > > > odd
> > > > > > > > > > > > > > > >> that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> ConsumerGroupHeartbeat
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > requires
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > "Read
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> Group"
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > even
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > though
> > > > > > > > > it
> > > > > > > > > > > > seems
> > > > > > > > > > > > > > to do
> > > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > > >> > > > than
> > > > > > > > > > > > > > > >> > > > > > > > > reading.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > I
> > > > > > > agree.
> > > > > > > > > > This is
> > > > > > > > > > > > > > how the
> > > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > > >> > > > > > > > > protocol works
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > though.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > We
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > only
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > require
> > > > > > > > > > "Read
> > > > > > > > > > > > > > Group" to
> > > > > > > > > > > > > > > >> join a
> > > > > > > > > > > > > > > >> > > > > > group. We
> > > > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consider
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> changing
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> this
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > but I
> > > > > > > am
> > > > > > > > > not
> > > > > > > > > > > > sure
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > >> it is
> > > > > > > > > > > > > > > >> > > > worth
> > > > > > > > > > > > > > > >> > > > > > it.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 13.
> > > > > > > How
> > > > > > > > > is
> > > > > > > > > > > > topic
> > > > > > > > > > > > > > > >> recreation
> > > > > > > > > > > > > > > >> > > > > > handled by
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > consumer
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > group
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > protocol?
> > > > > > > > > > It
> > > > > > > > > > > > > > would be
> > > > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > > > >> > > > > > have a
> > > > > > > > > > > > > > > >> > > > > > > > > section
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > this.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Noted.
> > > > > > > > > From
> > > > > > > > > > a
> > > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > > >> > > > > > perspective, the
> > > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > will
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> have a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > topic
> > > > > > > id
> > > > > > > > > so
> > > > > > > > > > it
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > > >> treat it
> > > > > > > > > > > > > > > >> > > > like a
> > > > > > > > > > > > > > > >> > > > > > > > > topic with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > different
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > name.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > only
> > > > > > > issue
> > > > > > > > > > is
> > > > > > > > > > > > that
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > fetch/commit
> > > > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> APIs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > do not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > support
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > IDs
> > > > > > > so the
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > >> would
> > > > > > > > > > > > > > > >> > > > reuse the
> > > > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> based
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > same.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > that
> > > > > > > we
> > > > > > > > > > should
> > > > > > > > > > > > > > update
> > > > > > > > > > > > > > > >> those
> > > > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > > > >> > > > > > as well
> > > > > > > > > > > > > > > >> > > > > > > > > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> order
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> consistent
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> end
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > to
> > > > > > > end.
> > > > > > > > > That
> > > > > > > > > > > > would
> > > > > > > > > > > > > > > >> strengthen
> > > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > semantics of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consumer.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 14.
> > > > > > > The
> > > > > > > > > > KIP
> > > > > > > > > > > > > > mentions
> > > > > > > > > > > > > > > >> we will
> > > > > > > > > > > > > > > >> > > > > > write the
> > > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > coordinator
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Java.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> Even
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > though
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > this is
> > > > > > > > > an
> > > > > > > > > > > > > > > >> implementation
> > > > > > > > > > > > > > > >> > > > > > detail, do
> > > > > > > > > > > > > > > >> > > > > > > > > we plan
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > have a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > gradle
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > module
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > for
> > > > > > > it?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Yes.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 15.
> > > > > > > Do
> > > > > > > > > we
> > > > > > > > > > > > have a
> > > > > > > > > > > > > > > >> scalability
> > > > > > > > > > > > > > > >> > > > > > goal when
> > > > > > > > > > > > > > > >> > > > > > > > > it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> comes
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > to how
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> many
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > members
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > new
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > group
> > > > > > > > > > protocol
> > > > > > > > > > > > > can
> > > > > > > > > > > > > > > >> support?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > We
> > > > > > > don't
> > > > > > > > > > have
> > > > > > > > > > > > > > numbers
> > > > > > > > > > > > > > > >> at the
> > > > > > > > > > > > > > > >> > > > > > moment. The
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> support
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> 1000s
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > of
> > > > > > > members
> > > > > > > > > > per
> > > > > > > > > > > > > > group.
> > > > > > > > > > > > > > > >> We will
> > > > > > > > > > > > > > > >> > > > > > measure
> > > > > > > > > > > > > > > >> > > > > > > > > this when
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > we have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > first
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > implementation.
> > > > > > > > > > > > > Note
> > > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > > >> > > > might
> > > > > > > > > > > > > > > >> > > > > > have
> > > > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > bottlenecks
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > down
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > road
> > > > > > > (e.g.
> > > > > > > > > > > > offset
> > > > > > > > > > > > > > > >> commits).
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > 16.
> > > > > > > Did
> > > > > > > > > we
> > > > > > > > > > > > > > consider
> > > > > > > > > > > > > > > >> having
> > > > > > > > > > > > > > > >> > > > > > > > > SubscribedTopidIds
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > > > SubscribedTopicNames
> > > > > > > > > > > > > > > >> in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > ConsumerGroupHeartbeatRequest?
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> idea
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > since we
> > > > > > > > > > have
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > >> resolve the
> > > > > > > > > > > > > > > >> > > > > > regex on
> > > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > server, we
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> can do
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> same
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > the
> > > > > > > > > topic
> > > > > > > > > > > > name?
> > > > > > > > > > > > > > The
> > > > > > > > > > > > > > > >> > > > difference
> > > > > > > > > > > > > > > >> > > > > > is that
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> sending
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regex is
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > more
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > efficient
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > whereas
> > > > > > > > > > > > sending
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > > >> > > > > > is less
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > efficient.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > Furthermore,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > delete
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > recreation is
> > > > > > > > > > > > > > easier
> > > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > > >> > > > handle
> > > > > > > > > > > > > > > >> > > > > > if we
> > > > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > ids.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > The
> > > > > > > idea
> > > > > > > > > > was to
> > > > > > > > > > > > > > > >> consolidate the
> > > > > > > > > > > > > > > >> > > > > > metadata
> > > > > > > > > > > > > > > >> > > > > > > > > lookup
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> server
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > for
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> both
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > paths
> > > > > > > but
> > > > > > > > > I
> > > > > > > > > > do
> > > > > > > > > > > > > agree
> > > > > > > > > > > > > > > >> with your
> > > > > > > > > > > > > > > >> > > > > > point. As
> > > > > > > > > > > > > > > >> > > > > > > > > a
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> second
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > though,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > ids
> > > > > > > may be
> > > > > > > > > > > > better
> > > > > > > > > > > > > > here
> > > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > > >> > > > > > delete and
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > recreation
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Also,
> I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > suppose
> > > > > > > > > > that we
> > > > > > > > > > > > > may
> > > > > > > > > > > > > > > >> allow
> > > > > > > > > > > > > > > >> > > > users to
> > > > > > > > > > > > > > > >> > > > > > > > > subscribe
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> ids
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > future
> > > > > > > > > > because
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > >> the only
> > > > > > > > > > > > > > > >> > > > > > way to be
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> really
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > robust
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > re-creation.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > Best,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > David
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > On
> > > > > > > Tue,
> > > > > > > > > Jul
> > > > > > > > > > 12,
> > > > > > > > > > > > > > 2022 at
> > > > > > > > > > > > > > > >> 1:38 PM
> > > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > > >> > > > > > > > > Jacot <
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > djacot@confluent.io>
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> >
> > > wrote:
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > Hi
> > > > > > > > > > Justine,
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > Thanks
> > > > > > > > > for
> > > > > > > > > > > > your
> > > > > > > > > > > > > > > >> comments.
> > > > > > > > > > > > > > > >> > > > Please
> > > > > > > > > > > > > > > >> > > > > > find
> > > > > > > > > > > > > > > >> > > > > > > > > my
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> answers
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > below.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > -
> > > > > > > Yes,
> > > > > > > > > > the new
> > > > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > > >> > > > relies on
> > > > > > > > > > > > > > > >> > > > > > > > > topic IDs
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > with the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> exception
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> of the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > > > topic
> > > > > > > > > > names
> > > > > > > > > > > > > based
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > ConsumerGroupHeartbeatRequest.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > am
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> sure
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > if
> > > > > > > using
> > > > > > > > > > topic
> > > > > > > > > > > > > > names
> > > > > > > > > > > > > > > >> is the
> > > > > > > > > > > > > > > >> > > > > > right call
> > > > > > > > > > > > > > > >> > > > > > > > > here.
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> I
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> about
> > > it
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > >
> > > > >
> > > > > a
> > > > > > > little
> > > > > > > > > > more.
> > > > > > > > > > > > > > > >> Obviously, the
> > > > > > > > > > > > > > > >> > > > > > KIP does
> > > > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > > > >> > > > > > > > > > > >> > change the
> > > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > -- Guozhang
> > > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > >
> >
> >
> > --
> > -- Guozhang
>
--
-- Guozhang
Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol
Posted by David Jacot <dj...@confluent.io.INVALID>.
3. ConsumerGroupMetadata is already defined in the consumer [1]. I
realized that we pass it in the current assignor so I thought that we
could reuse it. The alternative would be to add the same information
in a class which includes the error and the member assignment as well.
[1] https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerGroupMetadata.java
Best,
David
On Mon, Oct 3, 2022 at 6:15 AM Guozhang Wang <wa...@gmail.com> wrote:
>
> 1. Yeah that makes sense. I originally concerned if there's any consistency
> issues if the HB request with encoded topic IDs and the GPA/GIA RPCs with
> encoded topic IDs use mappings from two different brokers, and one of them
> is stale, but after thinking twice about it I think as long as we handle
> e.g. `unknown topic ids` from HB responses then we should be fine.
>
> 3. Ack, in that case I think we also need to define "ConsumerGroupMetadata"
> which is passed in `onAssignment`. We have a schema defined in the
> persistent storage for different purposes, but not in the consumer APIs
> yet. I think that would just include a single "Map<String, Metadata>"?
>
> On Fri, Sep 30, 2022 at 5:51 AM David Jacot <dj...@confluent.io.invalid>
> wrote:
>
> > Hey Guozhang,
> >
> > Thanks for your comments.
> >
> > 1. That's a very good question. I think that it is better to separate
> > the two concerns here. In my opinion, the GPA should only be used to
> > feed the assignor. I see the GPA as a way to call a remote function so
> > the GPA provides all its inputs to ensure that the assignor does not
> > depend on any local state. Knowing this, I would not use the GPA to
> > update the local state either. That is not its purpose. The consumer
> > is responsible to update its local metadata cache either when it gets
> > an unknown topic in the user-provided subscription or in the
> > coordinator-provided assignment. The consumer has to do this anyway to
> > get all topic metadata (e.g. leaders).
> >
> > 2. Interesting thought. Handling the translation is definitely
> > possible. I wonder if there are cases where having the topic ids is
> > beneficial. I am thinking about when a topic is deleted and recreated
> > right away. In this case, the current member assignment would still
> > have the old topic id and we would not be able to resolve it anymore.
> > I suppose that we should directly remove it from the current
> > assignment on the server side in this case and only add the new one.
> > As I first thought, that seems OK. The major downside is that the user
> > of the assignor would not know that the topic is different from what
> > it may have cached locally.
> >
> > We could say that we don't expose topic ids externally for now in
> > order to stay consistent with the other interfaces but it would be
> > annoying to add it back if we want to expose topic ids in the future.
> > I personally believe that we will have to expose them at some point.
> > For instance, in the rebalance listener, it would be useful to know
> > that a topic has been deleted and recreated. The only way to do this
> > is to know the topic id. Let me think a little more about this and
> > come back to you.
> >
> > 3. Sorry for this one. I was a typo. PartitionAssingor#assign should
> > return GroupAssignment.
> >
> > Best,
> > David
> >
> > On Thu, Sep 29, 2022 at 6:06 PM Guozhang Wang <wa...@gmail.com> wrote:
> > >
> > > Thanks David, I read through the updated wiki, and I have a couple
> > > follow-up questions:
> > >
> > > 1. It seems we would have two potential sources to update the consumer's
> > > internally maintained topic name -> topic ID mapping, which is used for
> > > encoding subscribed topic Ids in the HB request, and offset commit / list
> > > offset requests, etc. One is from metadata responses, and another one is
> > > from the GPA (group prepare assignment). On the other hand, the assignor
> > > APIs directly take parsed AssignmentMemberSpec and
> > AssignmentTopicMetadata
> > > from GPA responses directly instead of trying to consult the consumer's
> > own
> > > cached mapping. So just to clarify, consumers would also update its
> > > internal mapping from the GPA responses as well, right?
> > >
> > > 2. I'm wondering if it's necessary to expose the topic IDs to the
> > assignor
> > > developers. What they need to do at the moment is to take the mapping
> > from
> > > AssignmentTopicMetadata, and translate their decided assignment into
> > topic
> > > IDs when returning the MemberAssignment. We can instead just let consumer
> > > to do such translation for them internally and still let the assignor
> > > developers to only deal with the topic names. Of course that means the
> > > client side assignor APIs would be a bit further diverged from the
> > > broker-side assignor such that:
> > >
> > > a) Topic metadata would be encoded just as String -> num.partitions;
> > > b) AssignmentMemberSpec contains subscribedTopicNames and topicPartitions
> > > as target assignments;
> > > c) Returned MemberAssignment contains list of topicPartitions instead of
> > > topicIdPartitions.
> > >
> > > Internally, consumers would convert the IDs to names when calling the
> > > assign() function, and convert them back when sending the request; and
> > > similarly it would convert the IDs to names when calling onAssignment()
> > as
> > > well.
> > >
> > > In short, I'm wondering if we should bite the bullet for the users,
> > eating
> > > more complexities internally than pushing that to users.
> > >
> > > 3. This is a nit one: the return type of `Assignment
> > assign(AssignmentSpec
> > > assignmentSpec);` seems not defined in the doc. I think it would just
> > > include two fields, as:
> > >
> > > class Assignment {
> > > GroupAssignment groupAssignment;
> > >
> > > ConsumerGroupMetadata consumerGroupMetadata;
> > > }
> > >
> > > Is that right?
> > >
> > > Guozhang
> > >
> > > On Thu, Sep 29, 2022 at 1:26 AM David Jacot <djacot@confluent.io.invalid
> > >
> > > wrote:
> > >
> > > > Hi Guozhang,
> > > >
> > > > Thanks for your input.
> > > >
> > > > > 1) in HB request, when clients send the subscription, they are still
> > > > > sending them in terms of topic names. That is, the consumer would
> > not try
> > > > > to keep all the mappings from topic names to topic ids for the
> > subscribed
> > > > > topics.
> > > >
> > > > This is a mistake of mine. The heartbeat request should use Topic Ids.
> > > > I forgot to change it. The idea is that the client will get the
> > > > metadata first to translate the topic names to topic ids and thus also
> > > > have the mapping in its local metadata cache.
> > > >
> > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> > already.
> > > > And
> > > > > fetch requests can directly rely on that since they just need topic
> > IDs
> > > > and
> > > > > do not care what actual topic names are.
> > > >
> > > > That's right.
> > > >
> > > > > 3) in prepare-assignment responses, we return subscriptions as topic
> > IDs
> > > > as
> > > > > well.
> > > >
> > > > That's right.
> > > >
> > > > > In a word, the consumer does not try to maintain the mapping of topic
> > > > names
> > > > > to IDs at all, and even if they do, they will not expose it in e.g.
> > > > > assignor APIs. But for client-side assignors like Streams, if they
> > do not
> > > > > maintain the mapping from topic IDs to topic names, certain logic
> > such as
> > > > > co-partitioning indeed cannot be executed.
> > > >
> > > > As I said, the consumer will have the mapping in its local metadata
> > > > cache. It needs to get the partition metadata anyway and the topic id
> > > > is part of it. However, I do agree that this mapping is not exposed.
> > > >
> > > > > So I think we'd need to consider 1) use topic names in subscriptions
> > in
> > > > the
> > > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > > PartitionAssignor#TopicMetadata#topicName.
> > > >
> > > > 1) makes sense to me. I think that we could actually provide both
> > > > names and ids. Regarding 2), I think that I am slightly in favor of
> > > > using both names and ids as well. Let me try to update the protocol
> > > > and the interface towards this.
> > > >
> > > > Thanks,
> > > > David
> > > >
> > > > On Wed, Sep 28, 2022 at 9:35 PM Guozhang Wang <wa...@gmail.com>
> > wrote:
> > > > >
> > > > > Hello Jun/David,
> > > > >
> > > > > Just to chime in here for Jun's comment #50, today Streams does not
> > have
> > > > a
> > > > > way to map topic names to topic IDs, and in its topology description
> > > > topics
> > > > > are encoded with names.
> > > > >
> > > > > What I originally thought was that, on the consumer side, we'd need
> > to
> > > > > maintain this mapping from topic names to topic IDs anyways, and
> > Streams
> > > > > should leverage on the consumer's functionality to get the
> > corresponding
> > > > > topic names for the given topic IDs. But when I read through the RPC
> > > > again
> > > > > I realized it may not be the case, since:
> > > > >
> > > > > 1) in HB request, when clients send the subscription, they are still
> > > > > sending them in terms of topic names. That is, the consumer would
> > not try
> > > > > to keep all the mappings from topic names to topic ids for the
> > subscribed
> > > > > topics.
> > > > > 2) in HB responses, the assignment's topics are encoded as IDs
> > already.
> > > > And
> > > > > fetch requests can directly rely on that since they just need topic
> > IDs
> > > > and
> > > > > do not care what actual topic names are.
> > > > > 3) in prepare-assignment responses, we return subscriptions as topic
> > IDs
> > > > as
> > > > > well.
> > > > >
> > > > > In a word, the consumer does not try to maintain the mapping of topic
> > > > names
> > > > > to IDs at all, and even if they do, they will not expose it in e.g.
> > > > > assignor APIs. But for client-side assignors like Streams, if they
> > do not
> > > > > maintain the mapping from topic IDs to topic names, certain logic
> > such as
> > > > > co-partitioning indeed cannot be executed.
> > > > >
> > > > > So I think we'd need to consider 1) use topic names in subscriptions
> > in
> > > > the
> > > > > prepare-assignment responses, 2) use topic Names in client-side
> > > > > PartitionAssignor#GroupMember#subscribedTopics and
> > > > > PartitionAssignor#TopicMetadata#topicName.
> > > > >
> > > > > On Wed, Sep 28, 2022 at 1:26 AM David Jacot
> > <djacot@confluent.io.invalid
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Jun,
> > > > > >
> > > > > > Thanks for your comments.
> > > > > >
> > > > > > 48. Okay. There is a typo in the text:
> > > > > > > Note that the persisted target assignment of B already includes
> > foo-2
> > > > > > here
> > > > > > but B does not get it yet because foo-2 is not free.
> > > > > >
> > > > > > It should be:
> > > > > > > Note that the persisted member assignment of B already includes
> > foo-2
> > > > > > here
> > > > > > but B does not get it yet because foo-2 is not free.
> > > > > >
> > > > > > The examples are showing the persisted state in the
> > __consumer_offsets
> > > > > > topics. When a member transitions to the next epoch, we directly
> > > > > > persist the whole target assignment in order to reduce the number
> > of
> > > > > > writes and we manage dependencies in memory. So "B - epoch=2,
> > > > > > partitions=[foo-2]" is actually correct from a persisted record
> > > > > > perspective. However, the in-memory state would be different
> > because
> > > > > > foo-2 is not yet available as you pointed out. The in-memory state
> > > > > > would like as follow:
> > > > > >
> > > > > > Group (epoch=2)
> > > > > > A
> > > > > > B
> > > > > > Target Assignment (epoch=2)
> > > > > > A - partitions=[foo-0, foo-1]
> > > > > > B - partitions=[foo-2]
> > > > > > Member Assignment
> > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2], pending
> > partitions=[]
> > > > > > B - epoch=2, partitions=[], pending partitions=[foo-2]
> > > > > >
> > > > > > I suppose that the confusion comes from this. I will go and convert
> > > > > > all the examples to their in-memory representation in order to
> > reduce
> > > > > > the confusion.
> > > > > >
> > > > > > 50. Okay. I have to admit that I am not so familiar with the
> > internals
> > > > > > of KStreams. If I understand correctly, you're saying that KStreams
> > > > > > may not have topic ids in its topology so topic names are missing
> > here
> > > > > > to match the topics. I was hoping that KStreams would have topic
> > ids
> > > > > > in the topology as well. If that is not the case, we can also add
> > > > > > topic names on our side. I will check this.
> > > > > >
> > > > > > 53. It seems that I was not clear. Sorry for that. I did not mean
> > to
> > > > > > say that MemberAssignment.metadata is not used by the group
> > > > > > coordinator. Let me re-explain the flow.
> > > > > >
> > > > > > PartitionAssignor.assign() returns an Assignment. Assignment
> > contains
> > > > > > MemberAssignments where each member has MemberAssignment.metadata.
> > The
> > > > > > Assignment is installed by the client with the
> > > > > > ConsumerGroupInstallAssignment API and this overrides the current
> > > > > > target assignment in the coordinator. Then the group coordinator
> > uses
> > > > > > it to reconcile members. Therefore the target assignment will
> > > > > > eventually become the current member assignment and it will be
> > > > > > delivered to the member, including the original
> > > > > > MemberAssignment.metadata defines by the assignor.
> > > > > >
> > > > > > I hope that this helps to clarify how MemberAssignment.metadata is
> > > > > > used afterwards.
> > > > > >
> > > > > > Best,
> > > > > > David
> > > > > >
> > > > > > On Tue, Sep 27, 2022 at 10:31 PM Jun Rao <jun@confluent.io.invalid
> > >
> > > > wrote:
> > > > > > > Hi, David,
> > > > > > >
> > > > > > > Thanks for the reply. A few more follow up questions.
> > > > > > >
> > > > > > > 48. Hmm, so member assignment is what the client sees. Then, I
> > am not
> > > > > > sure
> > > > > > > about the following. The text says B cannot get foo-2 yet. But in
> > > > Member
> > > > > > > Assignment, B has foo-2.
> > > > > > >
> > > > > > > At this point B can transition to epoch 2 but cannot get foo-2
> > until
> > > > A
> > > > > > > revokes it.
> > > > > > >
> > > > > > > Note that the persisted target assignment of B already includes
> > foo-2
> > > > > > here
> > > > > > > but B does not get it yet because foo-2 is not free.
> > > > > > >
> > > > > > > Group (epoch=2)
> > > > > > > A
> > > > > > > B
> > > > > > > Target Assignment (epoch=2)
> > > > > > > A - partitions=[foo-0, foo-1]
> > > > > > > B - partitions=[foo-2]
> > > > > > > Member Assignment
> > > > > > > A - epoch=1, partitions=[foo-0, foo-1, foo-2]
> > > > > > > B - epoch=2, partitions=[foo-2]
> > > > > > >
> > > > > > >
> > > > > > > 50. Hmm, I thought KStreams doesn't use range assignor all the
> > time.
> > > > It
> > > > > > > does the assignment based on the topology. For example, if you
> > are
> > > > > > joining
> > > > > > > two topics A and B and deriving the join result in a 3rd topic C,
> > > > which
> > > > > > > will be aggregated. You want to use range to co-locate the
> > partitions
> > > > > > from
> > > > > > > topic A and B. However, for topic C, the partition could
> > potentially
> > > > be
> > > > > > > assigned in round robin. Without the topic name, KStreams won't
> > know
> > > > > > which
> > > > > > > topics need to be co-located.
> > > > > > >
> > > > > > > 53. If MemberAssignment.metadata is not used by the group
> > > > coordinator,
> > > > > > > where is it being used?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Tue, Sep 27, 2022 at 1:20 PM Jason Gustafson
> > > > > > <ja...@confluent.io.invalid>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi David,
> > > > > > > >
> > > > > > > > Thanks for the response. Replied below:
> > > > > > > >
> > > > > > > > > I don't have a strong opinion on this one. However, it seems
> > that
> > > > > > this
> > > > > > > > could be a pandora box if our subset turns out to not be
> > sufficient
> > > > > > > > for some existing users. We will have to add more to it and the
> > > > > > > > compatibility story between versions could be a real
> > nightmare. One
> > > > > > > > thing that we could do for sure is disabling all unicode
> > groups in
> > > > the
> > > > > > > > library. That seems possible and would already reduce the
> > scope.
> > > > > > > >
> > > > > > > > Changes to the regex syntax seems not a major problem to me. We
> > > > would
> > > > > > bump
> > > > > > > > the protocol version as we usually do. Actually that begs the
> > same
> > > > > > question
> > > > > > > > about RE2. How stable is the syntax and does it differ
> > depending
> > > > on the
> > > > > > > > version of the library? In general, I do agree that we'd be
> > better
> > > > off
> > > > > > not
> > > > > > > > having to manage the syntax ourselves if we can convince
> > ourselves
> > > > that
> > > > > > > > compatibility won't be a problem and the client won't
> > eventually
> > > > need
> > > > > > to
> > > > > > > > have access to it.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Jason
> > > > > > > >
> > > > > > > > On Tue, Sep 27, 2022 at 6:21 AM David Jacot
> > > > > > <dj...@confluent.io.invalid>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Luke,
> > > > > > > > >
> > > > > > > > > Thanks for your comments.
> > > > > > > > >
> > > > > > > > > Your suggestion makes sense. Jun's made the same one. We are
> > also
> > > > > > > > > discussing whether we should directly return the group state
> > in
> > > > the
> > > > > > > > > response and drop the prepare assignment api. It seems that
> > the
> > > > > > > > > current consensus is to keep it though. If that stays like
> > this,
> > > > I
> > > > > > > > > will apply your suggestion.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > David
> > > > > > > > >
> > > > > > > > > On Tue, Sep 27, 2022 at 10:40 AM Luke Chen <
> > showuon@gmail.com>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi David,
> > > > > > > > > >
> > > > > > > > > > Thanks for your patience for answering all these questions!
> > > > > > > > > >
> > > > > > > > > > One more question from me:
> > > > > > > > > > 1. When the selected consumer is going to be in client-side
> > > > > > assigning,
> > > > > > > > we
> > > > > > > > > > will return COMPUTE_ASSIGNMENT in heartbeat response. I'm
> > > > > > wondering if
> > > > > > > > we
> > > > > > > > > > could use a flag to indicate the assignment is required.
> > The
> > > > > > rationale
> > > > > > > > is
> > > > > > > > > > when returning error, we lose a chance to catch up with the
> > > > latest
> > > > > > > > target
> > > > > > > > > > assignment or revoke partitions. Since we expected the
> > record
> > > > > > fetching
> > > > > > > > is
> > > > > > > > > > not blocked by the rebalance/assignment computing, we might
> > > > want to
> > > > > > > > allow
> > > > > > > > > > the members to update the member assignment in each
> > heartbeat.
> > > > > > > > > >
> > > > > > > > > > Luke
> > > > > > > > > >
> > > > > > > > > > On Tue, Sep 27, 2022 at 9:43 AM Jason Gustafson
> > > > > > > > > <ja...@confluent.io.invalid>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hey David,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for the response.
> > > > > > > > > > >
> > > > > > > > > > > > Do you think that we could have a case where two
> > members
> > > > with
> > > > > > the
> > > > > > > > > same
> > > > > > > > > > > static id are active?
> > > > > > > > > > >
> > > > > > > > > > > I was just thinking about the handoff when we spin up a
> > new
> > > > node
> > > > > > > > > before the
> > > > > > > > > > > old one completes shutdown. It's probably ok to fence
> > the old
> > > > > > member
> > > > > > > > > > > abruptly. I do recall we had a few issues with this
> > handoff
> > > > in
> > > > > > the
> > > > > > > > > past. It
> > > > > > > > > > > was due to the fact that the replaced node might have
> > been
> > > > the
> > > > > > > > leader.
> > > > > > > > > > > Since we've removed the leader notion, perhaps this kind
> > of
> > > > issue
> > > > > > > > won't
> > > > > > > > > > > resurface.
> > > > > > > > > > >
> > > > > > > > > > > In regard to the RE2 library. I understand the intent to
> > pass
> > > > > > this
> > > > > > > > > through
> > > > > > > > > > > as a string without any validation so that we don't need
> > the
> > > > > > > > > dependence on
> > > > > > > > > > > the client. I do wonder if that will be restrictive in
> > > > practice.
> > > > > > For
> > > > > > > > > > > example, if a regex changes on the client side, would we
> > > > need to
> > > > > > > > > validate
> > > > > > > > > > > against the current assignment to see which partitions to
> > > > > > continue
> > > > > > > > > > > fetching? Or would we instead invalidate the complete
> > current
> > > > > > > > > assignment?
> > > > > > > > > > > Perhaps this is a rare case and we can do the latter.
> > There
> > > > > > could be
> > > > > > > > > more
> > > > > > > > > > > situations though.
> > > > > > > > > > >
> > > > > > > > > > > As an alternative, I wonder if we could define a
> > minimalistic
> > > > > > regex
> > > > > > > > > syntax
> > > > > > > > > > > (a subset of RE2 say) that could map more easily to regex
> > > > support
> > > > > > > > that
> > > > > > > > > > > already exists in clients? For example, in this
> > scenario, we
> > > > are
> > > > > > > > > matching
> > > > > > > > > > > topics, so we only need to be able to represent topic
> > > > characters
> > > > > > and
> > > > > > > > > not
> > > > > > > > > > > the whole smorgasbord of whitespace and unicode options.
> > > > Would it
> > > > > > > > make
> > > > > > > > > > > sense to have a reduced syntax which includes only
> > standard
> > > > topic
> > > > > > > > > > > characters, wildcards, character classes, and
> > repetitions?
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Jason
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Sep 22, 2022 at 9:02 AM Guozhang Wang <
> > > > > > wangguoz@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi David, thanks for all the detailed explanations. I
> > think
> > > > > > they
> > > > > > > > all
> > > > > > > > > make
> > > > > > > > > > > > sense. Just want to have a couple follow-ups here:
> > > > > > > > > > > >
> > > > > > > > > > > > > I don't really see the benefits here because server
> > side
> > > > > > > > assignors
> > > > > > > > > > > > don't have metadata at all. They only assign
> > > > topic-partitions.
> > > > > > They
> > > > > > > > > > > > are not supposed to generate metadata nor to receive
> > > > metadata
> > > > > > from
> > > > > > > > > the
> > > > > > > > > > > > members.
> > > > > > > > > > > >
> > > > > > > > > > > > Just to clarify I was asking about the `version` of the
> > > > > > assignor
> > > > > > > > > (i.e. up
> > > > > > > > > > > > to what version that the client would support), and I
> > do
> > > > agree
> > > > > > we
> > > > > > > > > would
> > > > > > > > > > > not
> > > > > > > > > > > > need metadata. What I have in mind is that, for some
> > > > specific
> > > > > > > > > built-in
> > > > > > > > > > > > broker-assignors, e.g. rack-aware assignors, if it's
> > > > possible
> > > > > > that
> > > > > > > > > in a
> > > > > > > > > > > > newer version we would have a hierarchical rack ID
> > string
> > > > > > format,
> > > > > > > > > like
> > > > > > > > > > > > "tier1-tier2" etc, but if some client has not upgraded
> > > > their
> > > > > > rack
> > > > > > > > ID
> > > > > > > > > > > > would still be in old format. In this case, the broker
> > then
> > > > > > needs
> > > > > > > > to
> > > > > > > > > > > choose
> > > > > > > > > > > > the old versioned assignor. I'm probably making
> > something
> > > > up
> > > > > > here
> > > > > > > > for
> > > > > > > > > > > rack
> > > > > > > > > > > > aware assignors, but I'm wondering if in general such
> > an
> > > > > > > > > "auto-downgrade"
> > > > > > > > > > > > behavior would be needed still for broker-side
> > assignor,
> > > > and
> > > > > > if yes
> > > > > > > > > would
> > > > > > > > > > > > "version" still be useful.
> > > > > > > > > > > >
> > > > > > > > > > > > > Yeah, that's right. Within a rebalance,
> > `onAssignment` is
> > > > > > called
> > > > > > > > > once
> > > > > > > > > > > > when the member transitions to a new epoch. This one
> > > > contains
> > > > > > the
> > > > > > > > > full
> > > > > > > > > > > > metadata provided by the client side assignor. Then,
> > > > > > `onAssignment`
> > > > > > > > > > > > can be called max N times where N is the number of
> > > > partitions
> > > > > > > > pending
> > > > > > > > > > > > revocation by other members. Let me try to clarify
> > this in
> > > > the
> > > > > > KIP.
> > > > > > > > > > > >
> > > > > > > > > > > > Okay, my understanding is that the calling ordering of
> > > > these
> > > > > > > > > callbacks
> > > > > > > > > > > > would be like the following:
> > > > > > > > > > > >
> > > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > > onPartitionsRevoked(); // just once, since we do not
> > > > really
> > > > > > need
> > > > > > > > > > > > to revoke incrementally.
> > > > > > > > > > > >
> > > > > > > > > > > > onAssignment(); // the first call, with epoch
> > > > incremented
> > > > > > > > > > > > onPartitionsAssigned(); // paired with the
> > onAssignment
> > > > > > > > > > > >
> > > > > > > > > > > > onAssignment(); // the first onAssignment
> > > > would
> > > > > > bump
> > > > > > > > up
> > > > > > > > > the
> > > > > > > > > > > > epoch, and the metadata reflected.
> > > > > > > > > > > > onPartitionsAssigned(); // each time we get an
> > additional
> > > > > > > > > assignment,
> > > > > > > > > > > we
> > > > > > > > > > > > call onAssignment and then paired with an
> > > > onPartitionsAssigned
> > > > > > > > > > > > ...
> > > > > > > > > > > > onAssignment();
> > > > > > > > > > > > onPartitionsAssigned(); // on each of the
> > onAssignment
> > > > > > calls, the
> > > > > > > > > > > encoded
> > > > > > > > > > > > metadata would not change, only the incrementally added
> > > > > > partitions
> > > > > > > > be
> > > > > > > > > > > > reflected
> > > > > > > > > > > >
> > > > > > > > > > > > Is that the case?
> > > > > > > > > > > >
> > > > > > > > > > > > I'm wondering if we would still call onAssignment just
> > > > once,
> > > > > > that
> > > > > > > > > encodes
> > > > > > > > > > > > all the assignment for this rebalance, including all
> > the
> > > > > > partitions
> > > > > > > > > that
> > > > > > > > > > > > should be assigned to the member but not yet assigned
> > since
> > > > > > they
> > > > > > > > > have not
> > > > > > > > > > > > been revoked by others. In that case the call ordering
> > > > would
> > > > > > be:
> > > > > > > > > > > >
> > > > > > > > > > > > ----------------------------------------
> > > > > > > > > > > > onPartitionsRevoked(); // just once
> > > > > > > > > > > > onAssignment(); // just once, with epoch
> > incremented,
> > > > and
> > > > > > > > metadata
> > > > > > > > > > > > encoded changed, the "assignment" field also reflect
> > the
> > > > final
> > > > > > > > target
> > > > > > > > > > > > assignment
> > > > > > > > > > > > onPartitionsAssigned(); // multiple times, which
> > > > represent
> > > > > > > > > > > incrementally
> > > > > > > > > > > > added partitions
> > > > > > > > > > > > ...
> > > > > > > > > > > > onPartitionsAssigned();
> > > > > > > > > > > >
> > > > > > > > > > > > The motivation from this is that, most users would only
> > > > > > implement
> > > > > > > > the
> > > > > > > > > > > > rebalance callback listeners and hence we'd definitely
> > > > need to
> > > > > > make
> > > > > > > > > sure
> > > > > > > > > > > > the semantics of that does not change much, and the
> > time
> > > > > > > > > > > > `onPartitionsAssigned` indicate the time when the
> > > > partitions
> > > > > > are
> > > > > > > > > actually
> > > > > > > > > > > > assigned to it; while for assignors, the
> > `onAssignment` is
> > > > > > used to
> > > > > > > > > > > indicate
> > > > > > > > > > > > what decision is made regarding for this member, i.e.
> > when
> > > > the
> > > > > > > > > partitions
> > > > > > > > > > > > are decided to be given to it, but not necessarily
> > meant
> > > > that
> > > > > > it
> > > > > > > > has
> > > > > > > > > been
> > > > > > > > > > > > given, since that time should be determined by the
> > time of
> > > > > > > > > > > > `onPartitionsAssigned`. The benefits though, would be
> > that
> > > > > > assignor
> > > > > > > > > > > > implementers would not need to reason which
> > `onAssignment`
> > > > > > would be
> > > > > > > > > the
> > > > > > > > > > > > last one for this epoch.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Guozhang
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Guozhang
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Sep 22, 2022 at 2:20 AM David Jacot
> > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Guozhang,
> > > > > > > > > > > > >
> > > > > > > > > > > > > > 1) The client-side "PartitionAssignor#Assignment"
> > has
> > > > an
> > > > > > Error
> > > > > > > > > byte
> > > > > > > > > > > > > field,
> > > > > > > > > > > > > > while the broker-side
> > "PartitionAssignor#Assignment"
> > > > does
> > > > > > not.
> > > > > > > > > And
> > > > > > > > > > > the
> > > > > > > > > > > > > > rationale seems to be assuming that we should
> > always be
> > > > > > able to
> > > > > > > > > do
> > > > > > > > > > > the
> > > > > > > > > > > > > > assignment at the broker-side assignor without
> > errors.
> > > > > > > > > Personally I
> > > > > > > > > > > > think
> > > > > > > > > > > > > > it's still potentially beneficial to add the Error
> > > > field
> > > > > > even
> > > > > > > > for
> > > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> > where
> > > > some
> > > > > > > > > subscribed
> > > > > > > > > > > > > > topics are not recognized with the current broker's
> > > > > > metadata.
> > > > > > > > > What do
> > > > > > > > > > > > you
> > > > > > > > > > > > > > think?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yeah, that seems reasonable. However, I wonder if it
> > > > would be
> > > > > > > > > better
> > > > > > > > > > > > > to use an exception on the server side. This is what
> > we
> > > > > > usually
> > > > > > > > do
> > > > > > > > > for
> > > > > > > > > > > > > server side plugins. On the client side, we use a
> > field
> > > > > > because
> > > > > > > > the
> > > > > > > > > > > > > errors are not defined in advance.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Your comment also makes me think about what we
> > should do
> > > > > > when the
> > > > > > > > > > > > > server side assignor fails. I suppose that we have to
> > > > keep
> > > > > > the
> > > > > > > > > current
> > > > > > > > > > > > > assignment until a new event occurs. For instance, in
> > > > your
> > > > > > > > example,
> > > > > > > > > > > > > the coordinator would have to trigger a rebalance
> > when
> > > > > > > > unrecognized
> > > > > > > > > > > > > topics are available. This would be part of the
> > metadata
> > > > > > > > > monitoring.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> > additional
> > > > > > fields
> > > > > > > > > > > > > > reason/version/metadata compared with the
> > broker-side
> > > > > > > > > GroupMember. I
> > > > > > > > > > > > > agree
> > > > > > > > > > > > > > that broker-side assignor would not need
> > > > reason/metadata
> > > > > > since
> > > > > > > > > they
> > > > > > > > > > > are
> > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but what
> > about
> > > > > > version?
> > > > > > > > > E.g.
> > > > > > > > > > > is
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > possible that we evolve our broker-side built-in
> > > > assignor
> > > > > > but
> > > > > > > > > the old
> > > > > > > > > > > > > > versioned clients would not be able to work with
> > the
> > > > new
> > > > > > > > > version, in
> > > > > > > > > > > > > which
> > > > > > > > > > > > > > case we need to let the broker being aware of this
> > and
> > > > > > upgrade
> > > > > > > > > its
> > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't really see the benefits here because server
> > side
> > > > > > > > assignors
> > > > > > > > > > > > > don't have metadata at all. They only assign
> > > > > > topic-partitions.
> > > > > > > > They
> > > > > > > > > > > > > are not supposed to generate metadata nor to receive
> > > > metadata
> > > > > > > > from
> > > > > > > > > the
> > > > > > > > > > > > > members.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > 3) Also related to 2) above, for the client-side
> > > > > > "GroupMember",
> > > > > > > > > > > instead
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > including these three fields, what about just
> > adding
> > > > the
> > > > > > > > > "Metadata"
> > > > > > > > > > > > field
> > > > > > > > > > > > > > class which has these three fields? Also, there
> > are two
> > > > > > > > > "Metadata"
> > > > > > > > > > > > > > currently in the APIs, the first is a class that
> > > > encodes
> > > > > > > > > > > > > > reason/version/metadata, and the second is just the
> > > > encoded
> > > > > > > > > metadata
> > > > > > > > > > > > > bytes.
> > > > > > > > > > > > > > I'm wondering what about just naming the first as
> > > > > > > > memberMetadata,
> > > > > > > > > > > which
> > > > > > > > > > > > > > then has a bytebuffer field Metadata, or instead
> > > > naming the
> > > > > > > > > second
> > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > > >
> > > > > > > > > > > > > That's a good point. Let me try to rationalize this
> > > > interface
> > > > > > > > > based on
> > > > > > > > > > > > > your suggestions.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > David
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Sep 13, 2022 at 9:21 PM Guozhang Wang <
> > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Just had a few more nit questions about the public
> > > > APIs:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1) The client-side "PartitionAssignor#Assignment"
> > has
> > > > an
> > > > > > Error
> > > > > > > > > byte
> > > > > > > > > > > > > field,
> > > > > > > > > > > > > > while the broker-side
> > "PartitionAssignor#Assignment"
> > > > does
> > > > > > not.
> > > > > > > > > And
> > > > > > > > > > > the
> > > > > > > > > > > > > > rationale seems to be assuming that we should
> > always be
> > > > > > able to
> > > > > > > > > do
> > > > > > > > > > > the
> > > > > > > > > > > > > > assignment at the broker-side assignor without
> > errors.
> > > > > > > > > Personally I
> > > > > > > > > > > > think
> > > > > > > > > > > > > > it's still potentially beneficial to add the Error
> > > > field
> > > > > > even
> > > > > > > > for
> > > > > > > > > > > > > > broker-side assignors, e.g. for some edge cases
> > where
> > > > some
> > > > > > > > > subscribed
> > > > > > > > > > > > > > topics are not recognized with the current broker's
> > > > > > metadata.
> > > > > > > > > What do
> > > > > > > > > > > > you
> > > > > > > > > > > > > > think?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2) The client-side "GroupMember" has three
> > additional
> > > > > > fields
> > > > > > > > > > > > > > reason/version/metadata compared with the
> > broker-side
> > > > > > > > > GroupMember. I
> > > > > > > > > > > > > agree
> > > > > > > > > > > > > > that broker-side assignor would not need
> > > > reason/metadata
> > > > > > since
> > > > > > > > > they
> > > > > > > > > > > are
> > > > > > > > > > > > > > blackbox strings/bytes to the assignor, but what
> > about
> > > > > > version?
> > > > > > > > > E.g.
> > > > > > > > > > > is
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > possible that we evolve our broker-side built-in
> > > > assignor
> > > > > > but
> > > > > > > > > the old
> > > > > > > > > > > > > > versioned clients would not be able to work with
> > the
> > > > new
> > > > > > > > > version, in
> > > > > > > > > > > > > which
> > > > > > > > > > > > > > case we need to let the broker being aware of this
> > and
> > > > > > upgrade
> > > > > > > > > its
> > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > to cooperate with the clients?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 3) Also related to 2) above, for the client-side
> > > > > > "GroupMember",
> > > > > > > > > > > instead
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > including these three fields, what about just
> > adding
> > > > the
> > > > > > > > > "Metadata"
> > > > > > > > > > > > field
> > > > > > > > > > > > > > class which has these three fields? Also, there
> > are two
> > > > > > > > > "Metadata"
> > > > > > > > > > > > > > currently in the APIs, the first is a class that
> > > > encodes
> > > > > > > > > > > > > > reason/version/metadata, and the second is just the
> > > > encoded
> > > > > > > > > metadata
> > > > > > > > > > > > > bytes.
> > > > > > > > > > > > > > I'm wondering what about just naming the first as
> > > > > > > > memberMetadata,
> > > > > > > > > > > which
> > > > > > > > > > > > > > then has a bytebuffer field Metadata, or instead
> > > > naming the
> > > > > > > > > second
> > > > > > > > > > > > > > bytebuffer field as metadataBytes?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Sep 13, 2022 at 12:08 PM Guozhang Wang <
> > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hello David,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks for bringing this question up. I think the
> > > > main
> > > > > > > > > benefits as
> > > > > > > > > > > > you
> > > > > > > > > > > > > > > listed is 2) above if it stays; just to clarify,
> > we
> > > > would
> > > > > > > > only
> > > > > > > > > be
> > > > > > > > > > > > able
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > save one round trip if the rebalance is still
> > > > triggered
> > > > > > by
> > > > > > > > the
> > > > > > > > > > > > broker;
> > > > > > > > > > > > > if
> > > > > > > > > > > > > > > the rebalance is triggered by the client then the
> > > > > > > > > num.round.trips
> > > > > > > > > > > are
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > same:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 1) With GroupPrepareAssignment:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> > suppose
> > > > it has
> > > > > > > > > already
> > > > > > > > > > > > sent
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > > HB and hence has to wait for it to return first
> > since
> > > > > > only
> > > > > > > > one
> > > > > > > > > > > > request
> > > > > > > > > > > > > /
> > > > > > > > > > > > > > > response can be inflight with the coordinator's
> > > > socket.
> > > > > > > > > > > > > > > T1: client receives the HB response, and then
> > sends
> > > > the
> > > > > > > > > > > > > > > GroupPrepareAssignment request.
> > > > > > > > > > > > > > > T2: the GroupPrepareAssignment response is
> > returned.
> > > > > > > > > > > > > > > T3: it calculates the new assignment, and sends a
> > > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > > request.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In total, two round trips.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2) Without GroupPrepareAssignment:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > T0: client decides to do a new assignment,
> > suppose
> > > > it has
> > > > > > > > > already
> > > > > > > > > > > > sent
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > > HB and hence has to wait for it to return first
> > since
> > > > > > only
> > > > > > > > one
> > > > > > > > > > > > request
> > > > > > > > > > > > > /
> > > > > > > > > > > > > > > response can be inflight with the coordinator's
> > > > socket.
> > > > > > > > > > > > > > > T1: client receives the HB response, and then
> > sends
> > > > the
> > > > > > new
> > > > > > > > HB
> > > > > > > > > > > > request
> > > > > > > > > > > > > > > with the flag indicating a new rebalance needed..
> > > > > > > > > > > > > > > T2: the HB response with the optional member
> > metadata
> > > > > > map is
> > > > > > > > > > > > returned.
> > > > > > > > > > > > > > > T3: it calculates the new assignment, and sends a
> > > > > > > > > > > > > GroupInstallAssignment
> > > > > > > > > > > > > > > request.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In total, two round trips as well.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -----------------------------
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > So to complete the full picture here, we'd need
> > to
> > > > modify
> > > > > > > > both
> > > > > > > > > HB
> > > > > > > > > > > > > request
> > > > > > > > > > > > > > > and response so that the client can also
> > indicate a
> > > > new
> > > > > > > > > rebalance
> > > > > > > > > > > via
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > HB request as well, right?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Assuming all above is true, I think it's okay to
> > > > merge
> > > > > > the
> > > > > > > > > > > > > > > GroupPrepareAssignment into HB given that we can
> > > > make the
> > > > > > > > > > > additional
> > > > > > > > > > > > > fields
> > > > > > > > > > > > > > > encoding the full member (subscription) metadata
> > and
> > > > > > topic
> > > > > > > > > metadata
> > > > > > > > > > > > as
> > > > > > > > > > > > > > > optional fields.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Mon, Sep 12, 2022 at 5:22 AM David Jacot
> > > > > > > > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >> Hi all,
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> During an offline conversation, someone asked
> > why we
> > > > > > need
> > > > > > > > the
> > > > > > > > > > > > > > >> ConsumerGroupPrepareAssignment API and suggested
> > > > that we
> > > > > > > > could
> > > > > > > > > > > > instead
> > > > > > > > > > > > > > >> provide the group state in the heartbeat
> > response.
> > > > This
> > > > > > has
> > > > > > > > a
> > > > > > > > > few
> > > > > > > > > > > > > > >> advantages: 1) it does not require using a
> > special
> > > > error
> > > > > > > > code
> > > > > > > > > to
> > > > > > > > > > > > > > >> signal that a new assignment is required as the
> > > > signal
> > > > > > would
> > > > > > > > > be
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> provided group state; 2) it removes one round
> > trip
> > > > when
> > > > > > a
> > > > > > > > > client
> > > > > > > > > > > > side
> > > > > > > > > > > > > > >> assignor is used. The downside is that it makes
> > the
> > > > > > > > heartbeat
> > > > > > > > > > > > > > >> response's definition quite large. I recall
> > that I
> > > > went
> > > > > > with
> > > > > > > > > the
> > > > > > > > > > > > > > >> current approach due to this.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Providing the group state in the heartbeat
> > response
> > > > is
> > > > > > > > > appealing.
> > > > > > > > > > > > What
> > > > > > > > > > > > > > >> do you guys think?
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Best,
> > > > > > > > > > > > > > >> David
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On Mon, Sep 12, 2022 at 2:17 PM David Jacot <
> > > > > > > > > djacot@confluent.io>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Hi Guozhang,
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > 1. I have added a reference to the relevant
> > > > chapter
> > > > > > > > instead
> > > > > > > > > of
> > > > > > > > > > > > > > >> > repeating the whole thing. Does that work for
> > you?
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > 2. The "Rebalance Triggers" section you are
> > > > referring
> > > > > > to
> > > > > > > > is
> > > > > > > > > > > about
> > > > > > > > > > > > > when
> > > > > > > > > > > > > > >> > a rebalance should be triggered for the
> > > > non-upgraded
> > > > > > > > members
> > > > > > > > > > > using
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > old protocol. The section mentions that a
> > > > rebalance
> > > > > > must
> > > > > > > > be
> > > > > > > > > > > > > triggered
> > > > > > > > > > > > > > >> > when a new assignment is installed. This
> > implies
> > > > that
> > > > > > the
> > > > > > > > > group
> > > > > > > > > > > > > epoch
> > > > > > > > > > > > > > >> > was updated either by a native member or a
> > > > > > non-upgraded
> > > > > > > > > member.
> > > > > > > > > > > > For
> > > > > > > > > > > > > > >> > the latter, the JoinGroup request would be the
> > > > > > trigger. I
> > > > > > > > > have
> > > > > > > > > > > > > added a
> > > > > > > > > > > > > > >> > reference to the relevant chapter in the
> > > > "JoinGroup
> > > > > > > > > Handling"
> > > > > > > > > > > > > section
> > > > > > > > > > > > > > >> > as well. Does that make sense?
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Thanks,
> > > > > > > > > > > > > > >> > David
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > On Fri, Sep 9, 2022 at 10:35 PM Guozhang Wang
> > <
> > > > > > > > > > > wangguoz@gmail.com
> > > > > > > > > > > > >
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > Hello David,
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > Alright I think that's sufficient. Just to
> > make
> > > > that
> > > > > > > > > clear in
> > > > > > > > > > > > the
> > > > > > > > > > > > > doc,
> > > > > > > > > > > > > > >> > > could we update:
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > 1) the heartbeat request handling section,
> > > > stating
> > > > > > when
> > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > >> will
> > > > > > > > > > > > > > >> > > trigger rebalance based on the HB's member
> > > > metadata
> > > > > > /
> > > > > > > > > reason?
> > > > > > > > > > > > > > >> > > 2) the "Rebalance Triggers" section to
> > include
> > > > what
> > > > > > we
> > > > > > > > > > > described
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > >> "Group
> > > > > > > > > > > > > > >> > > Epoch - Trigger a rebalance" section as
> > well?
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > Guozhang
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > On Fri, Sep 9, 2022 at 1:28 AM David Jacot
> > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > Hi Guozhang,
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > I thought that the assignor will always be
> > > > > > consulted
> > > > > > > > > when
> > > > > > > > > > > the
> > > > > > > > > > > > > next
> > > > > > > > > > > > > > >> > > > heartbeat request is constructed. In other
> > > > words,
> > > > > > > > > > > > > > >> > > > `PartitionAssignor#metadata` will be
> > called
> > > > for
> > > > > > every
> > > > > > > > > > > > heartbeat.
> > > > > > > > > > > > > > >> This
> > > > > > > > > > > > > > >> > > > gives the opportunity for the assignor to
> > > > enforce
> > > > > > a
> > > > > > > > > > > rebalance
> > > > > > > > > > > > by
> > > > > > > > > > > > > > >> > > > setting the reason to a non-zero value or
> > by
> > > > > > changing
> > > > > > > > > the
> > > > > > > > > > > > > bytes. Do
> > > > > > > > > > > > > > >> > > > you think that this is not sufficient?
> > Are you
> > > > > > > > > concerned by
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> delay?
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > Best,
> > > > > > > > > > > > > > >> > > > David
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > On Fri, Sep 9, 2022 at 7:10 AM Guozhang
> > Wang <
> > > > > > > > > > > > > wangguoz@gmail.com>
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > Hello David,
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > One of Jun's comments make me thinking:
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > > >> > > > > In this case, a new assignment is
> > triggered
> > > > by
> > > > > > the
> > > > > > > > > client
> > > > > > > > > > > > side
> > > > > > > > > > > > > > >> > > > > assignor. When constructing the HB, the
> > > > consumer
> > > > > > > > will
> > > > > > > > > > > always
> > > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > > >> > > > > the client side assignor and propagate
> > the
> > > > > > > > > information to
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > coordinator. In other words, we don't
> > expect
> > > > > > users
> > > > > > > > to
> > > > > > > > > call
> > > > > > > > > > > > > > >> > > > > Consumer#enforceRebalance anymore.
> > > > > > > > > > > > > > >> > > > > ```
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > As I looked at the current
> > > > PartitionAssignor's
> > > > > > > > > interface,
> > > > > > > > > > > we
> > > > > > > > > > > > > > >> actually do
> > > > > > > > > > > > > > >> > > > > not have a way yet to instruct how to
> > > > construct
> > > > > > the
> > > > > > > > > next
> > > > > > > > > > > HB
> > > > > > > > > > > > > > >> request, e.g.
> > > > > > > > > > > > > > >> > > > > when the assignor wants to enforce a new
> > > > > > rebalance
> > > > > > > > > with a
> > > > > > > > > > > > new
> > > > > > > > > > > > > > >> assignment,
> > > > > > > > > > > > > > >> > > > > we'd need some customizable APIs inside
> > the
> > > > > > > > > > > > PartitionAssignor
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > >> indicate
> > > > > > > > > > > > > > >> > > > > the next HB telling broker about so.
> > WDYT
> > > > about
> > > > > > > > adding
> > > > > > > > > > > such
> > > > > > > > > > > > an
> > > > > > > > > > > > > > >> API on the
> > > > > > > > > > > > > > >> > > > > PartitionAssignor?
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > Guozhang
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > On Tue, Sep 6, 2022 at 6:09 AM David
> > Jacot
> > > > > > > > > > > > > > >> <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > Hi Jun,
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > I have updated the KIP to include your
> > > > > > feedback. I
> > > > > > > > > have
> > > > > > > > > > > > also
> > > > > > > > > > > > > > >> tried to
> > > > > > > > > > > > > > >> > > > > > clarify the parts which were not
> > cleared.
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > Best,
> > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > On Fri, Sep 2, 2022 at 4:18 PM David
> > > > Jacot <
> > > > > > > > > > > > > djacot@confluent.io
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Thanks for your feedback. Let me
> > start
> > > > by
> > > > > > > > > answering
> > > > > > > > > > > your
> > > > > > > > > > > > > > >> questions
> > > > > > > > > > > > > > >> > > > > > > inline and I will update the KIP
> > next
> > > > week.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall, the
> > main
> > > > > > benefits
> > > > > > > > > of
> > > > > > > > > > > the
> > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > > efficient
> > > > > > > > > support of
> > > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > I would also add that the KIP
> > removes
> > > > the
> > > > > > global
> > > > > > > > > sync
> > > > > > > > > > > > > barrier
> > > > > > > > > > > > > > >> in the
> > > > > > > > > > > > > > >> > > > > > > protocol which is essential to
> > improve
> > > > group
> > > > > > > > > stability
> > > > > > > > > > > > and
> > > > > > > > > > > > > > >> > > > > > > scalability, and the KIP also
> > > > simplifies the
> > > > > > > > > client by
> > > > > > > > > > > > > moving
> > > > > > > > > > > > > > >> most of
> > > > > > > > > > > > > > >> > > > > > > the logic to the server side.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 30. ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> > singleton.
> > > > Do we
> > > > > > plan
> > > > > > > > > to
> > > > > > > > > > > > > support
> > > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in the
> > > > > > consumers?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Definitely. The group coordinator
> > will
> > > > use
> > > > > > the
> > > > > > > > > > > assignor
> > > > > > > > > > > > > used
> > > > > > > > > > > > > > >> by a
> > > > > > > > > > > > > > >> > > > > > > majority of the members. This
> > allows the
> > > > > > group
> > > > > > > > to
> > > > > > > > > move
> > > > > > > > > > > > > from
> > > > > > > > > > > > > > >> one
> > > > > > > > > > > > > > >> > > > > > > assignor to another by a roll. This
> > is
> > > > > > explained
> > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> Assignor
> > > > > > > > > > > > > > >> > > > > > > Selection chapter.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> > explain
> > > > > > whether
> > > > > > > > > it's
> > > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > > >> > > > > > > > request or the scenarios when it
> > > > needs to
> > > > > > be
> > > > > > > > > filled?
> > > > > > > > > > > > For
> > > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > > >> > > > > > > > clear to me when TopicPartitions
> > > > needs to
> > > > > > be
> > > > > > > > > filled.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > The client is expected to set those
> > > > fields
> > > > > > in
> > > > > > > > > case of
> > > > > > > > > > > a
> > > > > > > > > > > > > > >> connection
> > > > > > > > > > > > > > >> > > > > > > issue (e.g. timeout) or when the
> > fields
> > > > have
> > > > > > > > > changed
> > > > > > > > > > > > since
> > > > > > > > > > > > > > >> the last
> > > > > > > > > > > > > > >> > > > > > > HB. The server populates those
> > fields as
> > > > > > long as
> > > > > > > > > the
> > > > > > > > > > > > > member
> > > > > > > > > > > > > > >> is not
> > > > > > > > > > > > > > >> > > > > > > fully reconciled - the member should
> > > > > > acknowledge
> > > > > > > > > that
> > > > > > > > > > > it
> > > > > > > > > > > > > has
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > expected epoch and assignment. I
> > will
> > > > > > clarify
> > > > > > > > > this in
> > > > > > > > > > > > the
> > > > > > > > > > > > > KIP.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> > protocol,
> > > > the
> > > > > > rack
> > > > > > > > > > > > affinity
> > > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > and the broker is only considered
> > > > during
> > > > > > > > > fetching,
> > > > > > > > > > > but
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > >> during
> > > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> > Sometimes,
> > > > once
> > > > > > the
> > > > > > > > > > > > assignment
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > > >> > > > > > > > no opportunity for read affinity
> > > > because
> > > > > > no
> > > > > > > > > replicas
> > > > > > > > > > > > of
> > > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > > wondering
> > > > > > if we
> > > > > > > > > should
> > > > > > > > > > > > use
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > > >> > > > > > > > to address this by including rack
> > in
> > > > > > > > > GroupMember.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > That's an interesting idea. I don't
> > see
> > > > any
> > > > > > > > issue
> > > > > > > > > with
> > > > > > > > > > > > > adding
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > rack
> > > > > > > > > > > > > > >> > > > > > > to the members. I will do so.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> > it's
> > > > > > useful to
> > > > > > > > > know
> > > > > > > > > > > how
> > > > > > > > > > > > > busy
> > > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> > event
> > > > loop
> > > > > > > > model,
> > > > > > > > > it
> > > > > > > > > > > > seems
> > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the fraction
> > of
> > > > the
> > > > > > time
> > > > > > > > > the
> > > > > > > > > > > > event
> > > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > That's a great idea. I will add it.
> > > > Thanks.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > > coordinator
> > > > > > > > > failover
> > > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the check
> > if
> > > > any
> > > > > > group
> > > > > > > > > with
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> > matching
> > > > topic?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Sure. When the new group coordinator
> > > > takes
> > > > > > over,
> > > > > > > > > it
> > > > > > > > > > > has
> > > > > > > > > > > > > to:
> > > > > > > > > > > > > > >> > > > > > > * Setup the session timeouts.
> > > > > > > > > > > > > > >> > > > > > > * Trigger a new assignment if a
> > client
> > > > side
> > > > > > > > > assignor
> > > > > > > > > > > is
> > > > > > > > > > > > > used.
> > > > > > > > > > > > > > >> We
> > > > > > > > > > > > > > >> > > > don't
> > > > > > > > > > > > > > >> > > > > > > store the information about the
> > member
> > > > > > selected
> > > > > > > > > to run
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > >> > > > > > > so we have to start a new one.
> > > > > > > > > > > > > > >> > > > > > > * Update the topics metadata,
> > verify the
> > > > > > > > wildcard
> > > > > > > > > > > > > > >> subscriptions, and
> > > > > > > > > > > > > > >> > > > > > > trigger a rebalance if needed.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 34. ConsumerGroupMetadataValue,
> > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > > >> > > > > > > > ConsumerGroupMemberMetadataValue:
> > > > Could we
> > > > > > > > > document
> > > > > > > > > > > > what
> > > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does the
> > epoch
> > > > in
> > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> > about the
> > > > > > one in
> > > > > > > > > > > > > > >> > > > > > > >
> > ConsumerGroupPartitionMetadataValue
> > > > and
> > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Sure. I will clarify that but it is
> > > > always
> > > > > > the
> > > > > > > > > latest
> > > > > > > > > > > > > group
> > > > > > > > > > > > > > >> epoch.
> > > > > > > > > > > > > > >> > > > > > > When the group state is updated, the
> > > > group
> > > > > > epoch
> > > > > > > > > is
> > > > > > > > > > > > > bumped so
> > > > > > > > > > > > > > >> we use
> > > > > > > > > > > > > > >> > > > > > > that one for all the change records
> > > > related
> > > > > > to
> > > > > > > > the
> > > > > > > > > > > > update.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator will
> > ensure
> > > > > > that
> > > > > > > > the
> > > > > > > > > > > > > following
> > > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > > >> > > > > > > > met: ... All members exists." It's
> > > > > > possible
> > > > > > > > for
> > > > > > > > > a
> > > > > > > > > > > > member
> > > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > That's right. Here I meant that the
> > > > members
> > > > > > > > > provided
> > > > > > > > > > > by
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > >> > > > > > > in the assignment must exist in the
> > > > group.
> > > > > > The
> > > > > > > > > > > assignor
> > > > > > > > > > > > > can
> > > > > > > > > > > > > > >> not make
> > > > > > > > > > > > > > >> > > > > > > up new member ids.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> > with a
> > > > > > member
> > > > > > > > > epoch
> > > > > > > > > > > > > equals to
> > > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what member
> > id
> > > > > > would be
> > > > > > > > > used?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > A member is expected to abandon all
> > its
> > > > > > > > > partitions and
> > > > > > > > > > > > > > >> rejoins when
> > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > >> > > > > > > receives the FENCED_MEMBER_EPOCH
> > error.
> > > > In
> > > > > > this
> > > > > > > > > case,
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > > coordinator will have removed the
> > member
> > > > > > from
> > > > > > > > the
> > > > > > > > > > > group.
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > >> member
> > > > > > > > > > > > > > >> > > > > > > can rejoin the group with the same
> > > > member
> > > > > > id but
> > > > > > > > > with
> > > > > > > > > > > 0
> > > > > > > > > > > > as
> > > > > > > > > > > > > > >> epoch. Let
> > > > > > > > > > > > > > >> > > > > > > me see if I can clarify this in the
> > KIP.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> > have
> > > > the
> > > > > > > > ability
> > > > > > > > > to
> > > > > > > > > > > > > trigger a
> > > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> > reason
> > > > or
> > > > > > by
> > > > > > > > > updating
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to be
> > > > > > conflicting
> > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > In this case, a new assignment is
> > > > triggered
> > > > > > by
> > > > > > > > the
> > > > > > > > > > > > client
> > > > > > > > > > > > > side
> > > > > > > > > > > > > > >> > > > > > > assignor. When constructing the HB,
> > the
> > > > > > consumer
> > > > > > > > > will
> > > > > > > > > > > > > always
> > > > > > > > > > > > > > >> consult
> > > > > > > > > > > > > > >> > > > > > > the client side assignor and
> > propagate
> > > > the
> > > > > > > > > information
> > > > > > > > > > > > to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > > coordinator. In other words, we
> > don't
> > > > expect
> > > > > > > > > users to
> > > > > > > > > > > > call
> > > > > > > > > > > > > > >> > > > > > > Consumer#enforceRebalance anymore.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples are
> > > > nice.
> > > > > > But
> > > > > > > > the
> > > > > > > > > > > > section
> > > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group transitions to
> > > > epoch
> > > > > > 2, B
> > > > > > > > > > > > > immediately
> > > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> > which
> > > > seems
> > > > > > > > > > > incorrect.
> > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group transitions to
> > > > epoch
> > > > > > 3, C
> > > > > > > > > seems
> > > > > > > > > > > to
> > > > > > > > > > > > > get
> > > > > > > > > > > > > > >> into
> > > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to epoch
> > 3, C
> > > > > > still
> > > > > > > > > has A -
> > > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Sorry for that! I will revise them.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of consumers:
> > Do
> > > > we
> > > > > > > > support
> > > > > > > > > the
> > > > > > > > > > > > > upgrade
> > > > > > > > > > > > > > >> from
> > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > We will support upgrading from the
> > > > consumer
> > > > > > > > > protocol
> > > > > > > > > > > > > version
> > > > > > > > > > > > > > >> 3,
> > > > > > > > > > > > > > >> > > > > > > introduced in KIP-792. KIP-792 is
> > not
> > > > > > > > implemented
> > > > > > > > > yet
> > > > > > > > > > > so
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> earliest
> > > > > > > > > > > > > > >> > > > > > > version is unknown at the moment.
> > This
> > > > is
> > > > > > > > > explained in
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> migration
> > > > > > > > > > > > > > >> > > > > > > plan chapter.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Thanks again for your feedback,
> > Jun. I
> > > > will
> > > > > > > > > update the
> > > > > > > > > > > > KIP
> > > > > > > > > > > > > > >> based on
> > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > >> > > > > > > next week.
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > Best,
> > > > > > > > > > > > > > >> > > > > > > David
> > > > > > > > > > > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > On Thu, Sep 1, 2022 at 9:07 PM Jun
> > Rao
> > > > > > > > > > > > > > >> <ju...@confluent.io.invalid>
> > > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > Hi, David,
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > Thanks for the KIP. Overall, the
> > main
> > > > > > benefits
> > > > > > > > > of
> > > > > > > > > > > the
> > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > >> seem to
> > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > >> > > > > > fewer
> > > > > > > > > > > > > > >> > > > > > > > RPCs during rebalance and more
> > > > efficient
> > > > > > > > > support of
> > > > > > > > > > > > > > >> wildcard. A few
> > > > > > > > > > > > > > >> > > > > > > > comments below.
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 30. ConsumerGroupHeartbeatRequest
> > > > > > > > > > > > > > >> > > > > > > > 30.1 ServerAssignor is a
> > singleton.
> > > > Do we
> > > > > > plan
> > > > > > > > > to
> > > > > > > > > > > > > support
> > > > > > > > > > > > > > >> rolling
> > > > > > > > > > > > > > >> > > > > > changing
> > > > > > > > > > > > > > >> > > > > > > > of the partition assignor in the
> > > > > > consumers?
> > > > > > > > > > > > > > >> > > > > > > > 30.2 For each field, could you
> > explain
> > > > > > whether
> > > > > > > > > it's
> > > > > > > > > > > > > > >> required in
> > > > > > > > > > > > > > >> > > > every
> > > > > > > > > > > > > > >> > > > > > > > request or the scenarios when it
> > > > needs to
> > > > > > be
> > > > > > > > > filled?
> > > > > > > > > > > > For
> > > > > > > > > > > > > > >> example,
> > > > > > > > > > > > > > >> > > > it's
> > > > > > > > > > > > > > >> > > > > > not
> > > > > > > > > > > > > > >> > > > > > > > clear to me when TopicPartitions
> > > > needs to
> > > > > > be
> > > > > > > > > filled.
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 31. In the current consumer
> > protocol,
> > > > the
> > > > > > rack
> > > > > > > > > > > > affinity
> > > > > > > > > > > > > > >> between the
> > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > and the broker is only considered
> > > > during
> > > > > > > > > fetching,
> > > > > > > > > > > but
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > >> during
> > > > > > > > > > > > > > >> > > > > > assigning
> > > > > > > > > > > > > > >> > > > > > > > partitions to consumers.
> > Sometimes,
> > > > once
> > > > > > the
> > > > > > > > > > > > assignment
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > >> made,
> > > > > > > > > > > > > > >> > > > there
> > > > > > > > > > > > > > >> > > > > > is
> > > > > > > > > > > > > > >> > > > > > > > no opportunity for read affinity
> > > > because
> > > > > > no
> > > > > > > > > replicas
> > > > > > > > > > > > of
> > > > > > > > > > > > > > >> assigned
> > > > > > > > > > > > > > >> > > > > > partitions
> > > > > > > > > > > > > > >> > > > > > > > are close to the member. I am
> > > > wondering
> > > > > > if we
> > > > > > > > > should
> > > > > > > > > > > > use
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > > > > opportunity
> > > > > > > > > > > > > > >> > > > > > > > to address this by including rack
> > in
> > > > > > > > > GroupMember.
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 32. On the metric side, often,
> > it's
> > > > > > useful to
> > > > > > > > > know
> > > > > > > > > > > how
> > > > > > > > > > > > > busy
> > > > > > > > > > > > > > >> a group
> > > > > > > > > > > > > > >> > > > > > > > coordinator is. By moving the
> > event
> > > > loop
> > > > > > > > model,
> > > > > > > > > it
> > > > > > > > > > > > seems
> > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > >> > > > > > add
> > > > > > > > > > > > > > >> > > > > > > > a metric that tracks the fraction
> > of
> > > > the
> > > > > > time
> > > > > > > > > the
> > > > > > > > > > > > event
> > > > > > > > > > > > > > >> loop is
> > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > actual work.
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 33. Could we add a section on
> > > > coordinator
> > > > > > > > > failover
> > > > > > > > > > > > > > >> handling? For
> > > > > > > > > > > > > > >> > > > > > example,
> > > > > > > > > > > > > > >> > > > > > > > does it need to trigger the check
> > if
> > > > any
> > > > > > group
> > > > > > > > > with
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> wildcard
> > > > > > > > > > > > > > >> > > > > > > > subscription now has a new
> > matching
> > > > topic?
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 34. ConsumerGroupMetadataValue,
> > > > > > > > > > > > > > >> > > > ConsumerGroupPartitionMetadataValue,
> > > > > > > > > > > > > > >> > > > > > > > ConsumerGroupMemberMetadataValue:
> > > > Could we
> > > > > > > > > document
> > > > > > > > > > > > what
> > > > > > > > > > > > > > >> the epoch
> > > > > > > > > > > > > > >> > > > > > field
> > > > > > > > > > > > > > >> > > > > > > > reflects? For example, does the
> > epoch
> > > > in
> > > > > > > > > > > > > > >> ConsumerGroupMetadataValue
> > > > > > > > > > > > > > >> > > > > > reflect
> > > > > > > > > > > > > > >> > > > > > > > the latest group epoch? What
> > about the
> > > > > > one in
> > > > > > > > > > > > > > >> > > > > > > >
> > ConsumerGroupPartitionMetadataValue
> > > > and
> > > > > > > > > > > > > > >> > > > > > ConsumerGroupMemberMetadataValue?
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 35. "the group coordinator will
> > ensure
> > > > > > that
> > > > > > > > the
> > > > > > > > > > > > > following
> > > > > > > > > > > > > > >> > > > invariants
> > > > > > > > > > > > > > >> > > > > > are
> > > > > > > > > > > > > > >> > > > > > > > met: ... All members exists." It's
> > > > > > possible
> > > > > > > > for
> > > > > > > > > a
> > > > > > > > > > > > member
> > > > > > > > > > > > > > >> not to
> > > > > > > > > > > > > > >> > > > get any
> > > > > > > > > > > > > > >> > > > > > > > assigned partitions, right?
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 36. "He can rejoins the group
> > with a
> > > > > > member
> > > > > > > > > epoch
> > > > > > > > > > > > > equals to
> > > > > > > > > > > > > > >> 0":
> > > > > > > > > > > > > > >> > > > When
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > a consumer rejoin and what member
> > id
> > > > > > would be
> > > > > > > > > used?
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 37. "Instead, power users will
> > have
> > > > the
> > > > > > > > ability
> > > > > > > > > to
> > > > > > > > > > > > > trigger a
> > > > > > > > > > > > > > >> > > > > > reassignment
> > > > > > > > > > > > > > >> > > > > > > > by either providing a non-zero
> > reason
> > > > or
> > > > > > by
> > > > > > > > > updating
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> assignor
> > > > > > > > > > > > > > >> > > > > > > > metadata." Hmm, this seems to be
> > > > > > conflicting
> > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> deprecation
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > > > Consumer#enforeRebalance.
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 38. The reassignment examples are
> > > > nice.
> > > > > > But
> > > > > > > > the
> > > > > > > > > > > > section
> > > > > > > > > > > > > > >> seems to
> > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > >> > > > > > > > multiple typos.
> > > > > > > > > > > > > > >> > > > > > > > 38.1 When the group transitions to
> > > > epoch
> > > > > > 2, B
> > > > > > > > > > > > > immediately
> > > > > > > > > > > > > > >> gets into
> > > > > > > > > > > > > > >> > > > > > > > "epoch=1, partitions=[foo-2]",
> > which
> > > > seems
> > > > > > > > > > > incorrect.
> > > > > > > > > > > > > > >> > > > > > > > 38.2 When the group transitions to
> > > > epoch
> > > > > > 3, C
> > > > > > > > > seems
> > > > > > > > > > > to
> > > > > > > > > > > > > get
> > > > > > > > > > > > > > >> into
> > > > > > > > > > > > > > >> > > > > > epoch=3,
> > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-1] too early.
> > > > > > > > > > > > > > >> > > > > > > > 38.3 After A transitions to epoch
> > 3, C
> > > > > > still
> > > > > > > > > has A -
> > > > > > > > > > > > > > >> epoch=2,
> > > > > > > > > > > > > > >> > > > > > > > partitions=[foo-0].
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > 39. Rolling upgrade of consumers:
> > Do
> > > > we
> > > > > > > > support
> > > > > > > > > the
> > > > > > > > > > > > > upgrade
> > > > > > > > > > > > > > >> from
> > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > >> > > > > > old
> > > > > > > > > > > > > > >> > > > > > > > version to new one?
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > Thanks,
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > Jun
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > On Mon, Aug 29, 2022 at 9:20 AM
> > David
> > > > > > Jacot
> > > > > > > > > > > > > > >> > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > >> > > > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > Hi all,
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > The KIP states that we will
> > > > > > re-implement the
> > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > >> in
> > > > > > > > > > > > > > >> > > > Java. I
> > > > > > > > > > > > > > >> > > > > > > > > discussed this offline with a
> > few
> > > > folks
> > > > > > and
> > > > > > > > > folks
> > > > > > > > > > > > are
> > > > > > > > > > > > > > >> concerned
> > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > >> > > > > > > > > we could introduce many
> > regressions
> > > > in
> > > > > > the
> > > > > > > > old
> > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > >> if we do
> > > > > > > > > > > > > > >> > > > so.
> > > > > > > > > > > > > > >> > > > > > > > > Therefore, I am going to remove
> > this
> > > > > > > > statement
> > > > > > > > > > > from
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> KIP. It
> > > > > > > > > > > > > > >> > > > is an
> > > > > > > > > > > > > > >> > > > > > > > > implementation detail after all
> > so
> > > > it
> > > > > > does
> > > > > > > > not
> > > > > > > > > > > have
> > > > > > > > > > > > > to be
> > > > > > > > > > > > > > >> > > > decided at
> > > > > > > > > > > > > > >> > > > > > > > > this stage. We will likely
> > start by
> > > > > > trying
> > > > > > > > to
> > > > > > > > > > > > > refactor the
> > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > >> > > > > > > > > implementation as a first step.
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > Cheers,
> > > > > > > > > > > > > > >> > > > > > > > > David
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > On Mon, Aug 29, 2022 at 3:52 PM
> > > > David
> > > > > > Jacot
> > > > > > > > <
> > > > > > > > > > > > > > >> djacot@confluent.io
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > Hi Luke,
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> > machine
> > > > are:
> > > > > > > > > "Empty,
> > > > > > > > > > > > > assigning,
> > > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in Consumer
> > > > Group
> > > > > > States
> > > > > > > > > > > > section,
> > > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > This sentence does not refer
> > to
> > > > those
> > > > > > > > group
> > > > > > > > > > > states
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > >> rather
> > > > > > > > > > > > > > >> > > > to a
> > > > > > > > > > > > > > >> > > > > > > > > > state machine replication
> > (SMR).
> > > > This
> > > > > > > > > refers to
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> entire
> > > > > > > > > > > > > > >> > > > state of
> > > > > > > > > > > > > > >> > > > > > > > > > group coordinator which is
> > > > replicated
> > > > > > via
> > > > > > > > > the
> > > > > > > > > > > log
> > > > > > > > > > > > > > >> layer. I will
> > > > > > > > > > > > > > >> > > > > > > > > > clarify this in the KIP.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean "each
> > > > state
> > > > > > > > machine
> > > > > > > > > is
> > > > > > > > > > > > > modelled
> > > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > The idea is to follow a model
> > > > similar
> > > > > > to
> > > > > > > > > the new
> > > > > > > > > > > > > quorum
> > > > > > > > > > > > > > >> > > > > > controller. We
> > > > > > > > > > > > > > >> > > > > > > > > > will have N threads to process
> > > > events.
> > > > > > > > Each
> > > > > > > > > > > > > > >> __consumer_offsets
> > > > > > > > > > > > > > >> > > > > > > > > > partition is assigned to a
> > unique
> > > > > > thread
> > > > > > > > > and all
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> events
> > > > > > > > > > > > > > >> > > > (e.g.
> > > > > > > > > > > > > > >> > > > > > > > > > requests, callbacks, etc.) are
> > > > > > processed
> > > > > > > > by
> > > > > > > > > this
> > > > > > > > > > > > > > >> thread. This
> > > > > > > > > > > > > > >> > > > > > simplify
> > > > > > > > > > > > > > >> > > > > > > > > > concurrency and will enable
> > us to
> > > > do
> > > > > > > > > simulation
> > > > > > > > > > > > > testing
> > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > > >> > > > > > > > > > coordinator.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a state
> > > > machine
> > > > > > per
> > > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > > consumer
> > > > > > group"
> > > > > > > > > owned
> > > > > > > > > > > > by
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > > coordinator
> > > > > > owns 2
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> > will we
> > > > > > have 1
> > > > > > > > > state
> > > > > > > > > > > > > machine
> > > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > See 1.1. The confusion comes
> > from
> > > > > > there, I
> > > > > > > > > > > think.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > > *is
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> number
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > > >> > > > > > > > > > > to run the state machines.
> > But
> > > > I'm
> > > > > > > > > wondering
> > > > > > > > > > > if
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the state of
> > > > each
> > > > > > > > consumer
> > > > > > > > > > > group
> > > > > > > > > > > > > (or
> > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no heavy
> > > > > > computation,
> > > > > > > > > why
> > > > > > > > > > > > > should we
> > > > > > > > > > > > > > >> need
> > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > See 1.2. The idea is to have
> > an
> > > > > > ability to
> > > > > > > > > shard
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > > > processing as
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > computation could be heavy.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer session
> > > > timeout,
> > > > > > why
> > > > > > > > > does
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > > >> > > > > > > > > > > locate between min (45s) and
> > > > > > max(60s)? I
> > > > > > > > > > > thought
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> > lower/upper
> > > > > > bound
> > > > > > > > of
> > > > > > > > > it,
> > > > > > > > > > > > no?
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > group.consumer.session.timeout.ms
> > > > > > int
> > > > > > > > > 30s The
> > > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> > consumer
> > > > > > group
> > > > > > > > > > > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > group.consumer.min.session.timeout.ms
> > > > > > > > > int 45s
> > > > > > > > > > > > The
> > > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > group.consumer.max.session.timeout.ms
> > > > > > > > > int 60s
> > > > > > > > > > > > The
> > > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > This is indeed a mistake. The
> > > > default
> > > > > > > > > session
> > > > > > > > > > > > > timeout
> > > > > > > > > > > > > > >> should
> > > > > > > > > > > > > > >> > > > be 45s
> > > > > > > > > > > > > > >> > > > > > > > > > (the current default).
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server side
> > > > > > assignor
> > > > > > > > are
> > > > > > > > > > > > [range,
> > > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > > assignor.
> > > > > > I'd
> > > > > > > > > like to
> > > > > > > > > > > > > know
> > > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users will
> > > > choose
> > > > > > > > uniform
> > > > > > > > > > > > assignor
> > > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> > distribution.
> > > > Any
> > > > > > > > other
> > > > > > > > > > > reason
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> > List
> > > > range,
> > > > > > > > > uniform
> > > > > > > > > > > The
> > > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > The order on the server side
> > has
> > > > no
> > > > > > > > > influence
> > > > > > > > > > > > > because
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > > >> > > > > > must
> > > > > > > > > > > > > > >> > > > > > > > > > chose the selector that he
> > wants
> > > > to
> > > > > > use.
> > > > > > > > > There
> > > > > > > > > > > is
> > > > > > > > > > > > no
> > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > >> > > > in the
> > > > > > > > > > > > > > >> > > > > > > > > > current proposal. If the
> > assignor
> > > > is
> > > > > > not
> > > > > > > > > > > specified
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > client,
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > request is rejected. The
> > default
> > > > > > client
> > > > > > > > > value
> > > > > > > > > > > for
> > > > > > > > > > > > > > >> > > > > > > > > > `group.remote.assignor` is
> > > > `uniform`
> > > > > > > > though.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > Thanks for your very good
> > > > comments,
> > > > > > Luke.
> > > > > > > > I
> > > > > > > > > hope
> > > > > > > > > > > > > that my
> > > > > > > > > > > > > > >> > > > answers
> > > > > > > > > > > > > > >> > > > > > help
> > > > > > > > > > > > > > >> > > > > > > > > > to clarify things. I will
> > update
> > > > the
> > > > > > KIP
> > > > > > > > as
> > > > > > > > > well
> > > > > > > > > > > > > based
> > > > > > > > > > > > > > >> on your
> > > > > > > > > > > > > > >> > > > > > > > > > feedback.
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > Cheers,
> > > > > > > > > > > > > > >> > > > > > > > > > David
> > > > > > > > > > > > > > >> > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > On Mon, Aug 22, 2022 at 9:29
> > AM
> > > > Luke
> > > > > > Chen
> > > > > > > > <
> > > > > > > > > > > > > > >> showuon@gmail.com>
> > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > Hi David,
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > Thanks for the update.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > Some more questions:
> > > > > > > > > > > > > > >> > > > > > > > > > > 1. In Group Coordinator
> > > > section, you
> > > > > > > > > > > mentioned:
> > > > > > > > > > > > > > >> > > > > > > > > > > > The new group coordinator
> > will
> > > > > > have a
> > > > > > > > > state
> > > > > > > > > > > > > machine
> > > > > > > > > > > > > > >> per
> > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets*
> > partitions,
> > > > > > where
> > > > > > > > > each
> > > > > > > > > > > > state
> > > > > > > > > > > > > > >> machine is
> > > > > > > > > > > > > > >> > > > > > modelled
> > > > > > > > > > > > > > >> > > > > > > > > as an
> > > > > > > > > > > > > > >> > > > > > > > > > > event loop. Those state
> > machines
> > > > > > will be
> > > > > > > > > > > > executed
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > >> > > > > > > > > > > *group.coordinator.threads*
> > > > threads.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.1. I think the state
> > machine
> > > > are:
> > > > > > > > > "Empty,
> > > > > > > > > > > > > assigning,
> > > > > > > > > > > > > > >> > > > > > reconciling,
> > > > > > > > > > > > > > >> > > > > > > > > stable,
> > > > > > > > > > > > > > >> > > > > > > > > > > dead" mentioned in Consumer
> > > > Group
> > > > > > States
> > > > > > > > > > > > section,
> > > > > > > > > > > > > > >> right?
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.2. What do you mean "each
> > > > state
> > > > > > > > machine
> > > > > > > > > is
> > > > > > > > > > > > > modelled
> > > > > > > > > > > > > > >> as an
> > > > > > > > > > > > > > >> > > > event
> > > > > > > > > > > > > > >> > > > > > > > > loop"?
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.3. Why do we need a state
> > > > machine
> > > > > > per
> > > > > > > > > > > > > > >> *__consumer_offsets*
> > > > > > > > > > > > > > >> > > > > > > > > partitions?
> > > > > > > > > > > > > > >> > > > > > > > > > > Not a state machine "per
> > > > consumer
> > > > > > group"
> > > > > > > > > owned
> > > > > > > > > > > > by
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > coordinator?
> > > > > > > > > > > > > > >> > > > > > > > > For
> > > > > > > > > > > > > > >> > > > > > > > > > > example, if one group
> > > > coordinator
> > > > > > owns 2
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> groups, and
> > > > > > > > > > > > > > >> > > > > > both
> > > > > > > > > > > > > > >> > > > > > > > > exist in
> > > > > > > > > > > > > > >> > > > > > > > > > > *__consumer_offsets-0*,
> > will we
> > > > > > have 1
> > > > > > > > > state
> > > > > > > > > > > > > machine
> > > > > > > > > > > > > > >> for it,
> > > > > > > > > > > > > > >> > > > or
> > > > > > > > > > > > > > >> > > > > > 2?
> > > > > > > > > > > > > > >> > > > > > > > > > > 1.4. I know the
> > > > > > > > > "*group.coordinator.threads"
> > > > > > > > > > > *is
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> number
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > threads
> > > > > > > > > > > > > > >> > > > > > > > > used
> > > > > > > > > > > > > > >> > > > > > > > > > > to run the state machines.
> > But
> > > > I'm
> > > > > > > > > wondering
> > > > > > > > > > > if
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> purpose
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > threads
> > > > > > > > > > > > > > >> > > > > > > > > > > is only to keep the state of
> > > > each
> > > > > > > > consumer
> > > > > > > > > > > group
> > > > > > > > > > > > > (or
> > > > > > > > > > > > > > >> > > > > > > > > *__consumer_offsets*
> > > > > > > > > > > > > > >> > > > > > > > > > > partitions?), and no heavy
> > > > > > computation,
> > > > > > > > > why
> > > > > > > > > > > > > should we
> > > > > > > > > > > > > > >> need
> > > > > > > > > > > > > > >> > > > > > > > > multi-threads
> > > > > > > > > > > > > > >> > > > > > > > > > > here?
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 2. For the default value in
> > the
> > > > new
> > > > > > > > > configs:
> > > > > > > > > > > > > > >> > > > > > > > > > > 2.1. The consumer session
> > > > timeout,
> > > > > > why
> > > > > > > > > does
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> default
> > > > > > > > > > > > > > >> > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout not
> > > > > > > > > > > > > > >> > > > > > > > > > > locate between min (45s) and
> > > > > > max(60s)? I
> > > > > > > > > > > thought
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> min/max
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > > > timeout is to define
> > lower/upper
> > > > > > bound
> > > > > > > > of
> > > > > > > > > it,
> > > > > > > > > > > > no?
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > group.consumer.session.timeout.ms
> > > > > > int
> > > > > > > > > 30s The
> > > > > > > > > > > > > > >> timeout to
> > > > > > > > > > > > > > >> > > > detect
> > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > > > > failures when using the
> > consumer
> > > > > > group
> > > > > > > > > > > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > group.consumer.min.session.timeout.ms
> > > > > > > > > int 45s
> > > > > > > > > > > > The
> > > > > > > > > > > > > > >> minimum
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > group.consumer.max.session.timeout.ms
> > > > > > > > > int 60s
> > > > > > > > > > > > The
> > > > > > > > > > > > > > >> maximum
> > > > > > > > > > > > > > >> > > > > > session
> > > > > > > > > > > > > > >> > > > > > > > > timeout.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > 2.2. The default server side
> > > > > > assignor
> > > > > > > > are
> > > > > > > > > > > > [range,
> > > > > > > > > > > > > > >> uniform],
> > > > > > > > > > > > > > >> > > > > > which means
> > > > > > > > > > > > > > >> > > > > > > > > > > we'll default to "range"
> > > > assignor.
> > > > > > I'd
> > > > > > > > > like to
> > > > > > > > > > > > > know
> > > > > > > > > > > > > > >> why not
> > > > > > > > > > > > > > >> > > > > > uniform
> > > > > > > > > > > > > > >> > > > > > > > > one? I
> > > > > > > > > > > > > > >> > > > > > > > > > > thought usually users will
> > > > choose
> > > > > > > > uniform
> > > > > > > > > > > > assignor
> > > > > > > > > > > > > > >> (former
> > > > > > > > > > > > > > >> > > > sticky
> > > > > > > > > > > > > > >> > > > > > > > > assinor)
> > > > > > > > > > > > > > >> > > > > > > > > > > for better evenly
> > distribution.
> > > > Any
> > > > > > > > other
> > > > > > > > > > > reason
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > >> choose
> > > > > > > > > > > > > > >> > > > range
> > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > > > as default?
> > > > > > > > > > > > > > >> > > > > > > > > > > group.consumer.assignors
> > List
> > > > range,
> > > > > > > > > uniform
> > > > > > > > > > > The
> > > > > > > > > > > > > > >> server side
> > > > > > > > > > > > > > >> > > > > > assignors.
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > Thank you.
> > > > > > > > > > > > > > >> > > > > > > > > > > Luke
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Aug 22, 2022 at
> > 2:10 PM
> > > > Luke
> > > > > > > > Chen
> > > > > > > > > <
> > > > > > > > > > > > > > >> showuon@gmail.com
> > > > > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > > Hi Sagar,
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > > I have some thoughts about
> > > > Kafka
> > > > > > > > Connect
> > > > > > > > > > > > > > >> integrating with
> > > > > > > > > > > > > > >> > > > > > KIP-848,
> > > > > > > > > > > > > > >> > > > > > > > > but I
> > > > > > > > > > > > > > >> > > > > > > > > > > > think we should have a
> > > > separate
> > > > > > > > > discussion
> > > > > > > > > > > > > thread
> > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > >> > > > Kafka
> > > > > > > > > > > > > > >> > > > > > > > > Connect
> > > > > > > > > > > > > > >> > > > > > > > > > > > KIP: Integrating Kafka
> > Connect
> > > > > > With
> > > > > > > > New
> > > > > > > > > > > > Consumer
> > > > > > > > > > > > > > >> Rebalance
> > > > > > > > > > > > > > >> > > > > > Protocol
> > > > > > > > > > > > > > >> > > > > > > > > [1],
> > > > > > > > > > > > > > >> > > > > > > > > > > > and let this discussion
> > thread
> > > > > > focus
> > > > > > > > on
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> rebalance
> > > > > > > > > > > > > > >> > > > > > protocol,
> > > > > > > > > > > > > > >> > > > > > > > > WDYT?
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > > [1]
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > > Thank you.
> > > > > > > > > > > > > > >> > > > > > > > > > > > Luke
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Aug 12, 2022 at
> > 9:31
> > > > PM
> > > > > > Sagar
> > > > > > > > <
> > > > > > > > > > > > > > >> > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Thank you Guozhang/David
> > for
> > > > the
> > > > > > > > > feedback.
> > > > > > > > > > > > > Looks
> > > > > > > > > > > > > > >> like
> > > > > > > > > > > > > > >> > > > there's
> > > > > > > > > > > > > > >> > > > > > > > > agreement on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> using separate APIs for
> > > > Connect.
> > > > > > I
> > > > > > > > > would
> > > > > > > > > > > > > revisit
> > > > > > > > > > > > > > >> the doc
> > > > > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > > >> > > > > > see
> > > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > > >> > > > > > > > > > > >> changes are to be made.
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Thanks!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Sagar.
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> On Tue, Aug 9, 2022 at
> > 7:11
> > > > PM
> > > > > > David
> > > > > > > > > Jacot
> > > > > > > > > > > > > > >> > > > > > > > > <dj...@confluent.io.invalid>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Hi Sagar,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Thanks for the feedback
> > > > and the
> > > > > > > > > document.
> > > > > > > > > > > > > That's
> > > > > > > > > > > > > > >> really
> > > > > > > > > > > > > > >> > > > > > helpful. I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > will take a look at it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Overall, it seems to me
> > > > that
> > > > > > both
> > > > > > > > > Connect
> > > > > > > > > > > > > and the
> > > > > > > > > > > > > > >> > > > Consumer
> > > > > > > > > > > > > > >> > > > > > could
> > > > > > > > > > > > > > >> > > > > > > > > share
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the same underlying
> > > > "engine".
> > > > > > The
> > > > > > > > > main
> > > > > > > > > > > > > > >> difference is
> > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > Consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigns
> > topic-partitions to
> > > > > > members
> > > > > > > > > > > whereas
> > > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > > >> > > > assigns
> > > > > > > > > > > > > > >> > > > > > tasks
> > > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > workers. I see two
> > ways to
> > > > move
> > > > > > > > > forward:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) We extend the new
> > > > proposed
> > > > > > APIs
> > > > > > > > to
> > > > > > > > > > > > support
> > > > > > > > > > > > > > >> different
> > > > > > > > > > > > > > >> > > > > > resource
> > > > > > > > > > > > > > >> > > > > > > > > types
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g. partitions,
> > tasks,
> > > > > > etc.); or
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > 2) We use new dedicated
> > > > APIs
> > > > > > for
> > > > > > > > > Connect.
> > > > > > > > > > > > The
> > > > > > > > > > > > > > >> dedicated
> > > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > > >> > > > > > > > > would be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > similar to the new
> > ones but
> > > > > > > > > different on
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > content/resources and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > they would rely on the
> > same
> > > > > > engine
> > > > > > > > > on the
> > > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > > >> > > > side.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > I personally lean
> > towards
> > > > 2)
> > > > > > > > because
> > > > > > > > > I am
> > > > > > > > > > > > > not a
> > > > > > > > > > > > > > >> fan of
> > > > > > > > > > > > > > >> > > > > > > > > overcharging
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > APIs to serve different
> > > > > > purposes.
> > > > > > > > > That
> > > > > > > > > > > > being
> > > > > > > > > > > > > > >> said, I am
> > > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > > >> > > > > > > > > opposed to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > 1) if we can find an
> > > > elegant
> > > > > > way to
> > > > > > > > > do
> > > > > > > > > > > it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > I think that we can
> > > > continue to
> > > > > > > > > discuss
> > > > > > > > > > > it
> > > > > > > > > > > > > here
> > > > > > > > > > > > > > >> for now
> > > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > > >> > > > > > order
> > > > > > > > > > > > > > >> > > > > > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > ensure that this KIP is
> > > > > > compatible
> > > > > > > > > with
> > > > > > > > > > > > what
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > >> will do
> > > > > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > > >> > > > > > > > > Connect in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the future.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Best,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > On Mon, Aug 8, 2022 at
> > > > 2:41 PM
> > > > > > > > David
> > > > > > > > > > > Jacot
> > > > > > > > > > > > <
> > > > > > > > > > > > > > >> > > > > > djacot@confluent.io>
> > > > > > > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Hi all,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > I am back from
> > vacation.
> > > > I
> > > > > > will
> > > > > > > > go
> > > > > > > > > > > > through
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > >> address
> > > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > > >> > > > > > > > > comments
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > in the coming days.
> > > > Thanks
> > > > > > for
> > > > > > > > your
> > > > > > > > > > > > > feedback.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > Cheers,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > On Wed, Aug 3, 2022
> > at
> > > > 10:05
> > > > > > PM
> > > > > > > > > Gregory
> > > > > > > > > > > > > Harris
> > > > > > > > > > > > > > >> <
> > > > > > > > > > > > > > >> > > > > > > > > gharris1727@gmail.com
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Hey All!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Thanks for the KIP,
> > > > it's
> > > > > > > > > wonderful to
> > > > > > > > > > > > see
> > > > > > > > > > > > > > >> > > > cooperative
> > > > > > > > > > > > > > >> > > > > > > > > rebalancing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > making it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > down the stack!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > I had a few
> > questions:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 1. The 'Rejected
> > > > > > Alternatives'
> > > > > > > > > > > section
> > > > > > > > > > > > > > >> describes how
> > > > > > > > > > > > > > >> > > > > > member
> > > > > > > > > > > > > > >> > > > > > > > > epoch
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > advance in step
> > with
> > > > the
> > > > > > group
> > > > > > > > > epoch
> > > > > > > > > > > > and
> > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > >> > > > > > epoch
> > > > > > > > > > > > > > >> > > > > > > > > values. I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > that this is a good
> > > > idea
> > > > > > for
> > > > > > > > the
> > > > > > > > > > > > reasons
> > > > > > > > > > > > > > >> described
> > > > > > > > > > > > > > >> > > > in
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > KIP. When
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > protocol is
> > > > incrementally
> > > > > > > > > assigning
> > > > > > > > > > > > > > >> partitions to a
> > > > > > > > > > > > > > >> > > > > > worker,
> > > > > > > > > > > > > > >> > > > > > > > > what
> > > > > > > > > > > > > > >> > > > > > > > > > > >> member
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > epoch does each
> > > > incremental
> > > > > > > > > > > assignment
> > > > > > > > > > > > > use?
> > > > > > > > > > > > > > >> Are
> > > > > > > > > > > > > > >> > > > member
> > > > > > > > > > > > > > >> > > > > > epochs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> re-used,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > a single member
> > epoch
> > > > can
> > > > > > > > > correspond
> > > > > > > > > > > to
> > > > > > > > > > > > > > >> multiple
> > > > > > > > > > > > > > >> > > > > > different
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > (monotonically
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > larger)
> > assignments?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 2. Is the
> > Assignor's
> > > > > > 'Reason'
> > > > > > > > > field
> > > > > > > > > > > > > opaque
> > > > > > > > > > > > > > >> to the
> > > > > > > > > > > > > > >> > > > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> coordinator? If
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > not, should custom
> > > > > > client-side
> > > > > > > > > > > assignor
> > > > > > > > > > > > > > >> > > > implementations
> > > > > > > > > > > > > > >> > > > > > > > > interact
> > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Reason field, and
> > how
> > > > is
> > > > > > its
> > > > > > > > > common
> > > > > > > > > > > > > meaning
> > > > > > > > > > > > > > >> agreed
> > > > > > > > > > > > > > >> > > > > > upon? If
> > > > > > > > > > > > > > >> > > > > > > > > so, what
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > is the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > benefit of a
> > distinct
> > > > > > Reason
> > > > > > > > > field
> > > > > > > > > > > over
> > > > > > > > > > > > > > >> including
> > > > > > > > > > > > > > >> > > > such
> > > > > > > > > > > > > > >> > > > > > > > > functionality
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > opaque metadata?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 3. The following is
> > > > > > included in
> > > > > > > > > the
> > > > > > > > > > > > KIP:
> > > > > > > > > > > > > > >> "Thanks to
> > > > > > > > > > > > > > >> > > > > > this, the
> > > > > > > > > > > > > > >> > > > > > > > > input
> > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client side
> > assignor is
> > > > > > > > entirely
> > > > > > > > > > > driven
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > > >> > > > > > > > > coordinator.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > consumer is no
> > longer
> > > > > > > > > responsible for
> > > > > > > > > > > > > > >> maintaining
> > > > > > > > > > > > > > >> > > > any
> > > > > > > > > > > > > > >> > > > > > state
> > > > > > > > > > > > > > >> > > > > > > > > besides
> > > > > > > > > > > > > > >> > > > > > > > > > > >> its
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assigned
> > partitions."
> > > > Does
> > > > > > this
> > > > > > > > > mean
> > > > > > > > > > > > > that the
> > > > > > > > > > > > > > >> > > > > > client-side
> > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> MAY
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > incorporate
> > additional
> > > > > > > > > non-Metadata
> > > > > > > > > > > > state
> > > > > > > > > > > > > > >> (such as
> > > > > > > > > > > > > > >> > > > > > partition
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > throughput,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > cpu/memory metrics,
> > > > config
> > > > > > > > > topics,
> > > > > > > > > > > > etc),
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > >> that
> > > > > > > > > > > > > > >> > > > > > additional
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > non-Metadata
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > state SHOULD NOT be
> > > > used?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > 4. I see that
> > there are
> > > > > > > > separate
> > > > > > > > > > > > classes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > for
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > org.apache.kafka.server.group.consumer.PartitionAssignor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and
> > > > > > > > > > > > > > >> > > >
> > > > > > org.apache.kafka.clients.consumer.PartitionAssignor
> > > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > seem to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > overlap
> > significantly.
> > > > Is
> > > > > > it
> > > > > > > > > possible
> > > > > > > > > > > > for
> > > > > > > > > > > > > > >> these two
> > > > > > > > > > > > > > >> > > > > > > > > implementations
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > unified? This would
> > > > serve
> > > > > > to
> > > > > > > > > promote
> > > > > > > > > > > > > feature
> > > > > > > > > > > > > > >> parity
> > > > > > > > > > > > > > >> > > > of
> > > > > > > > > > > > > > >> > > > > > > > > server-side
> > > > > > > > > > > > > > >> > > > > > > > > > > >> and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > client-side
> > assignors,
> > > > and
> > > > > > > > would
> > > > > > > > > also
> > > > > > > > > > > > > > >> facilitate
> > > > > > > > > > > > > > >> > > > > > operational
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > flexibility in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > certain
> > situations. For
> > > > > > > > example,
> > > > > > > > > if a
> > > > > > > > > > > > > > >> server-side
> > > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > has some
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > poor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > behavior and needs
> > a
> > > > patch,
> > > > > > > > > deploying
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> patched
> > > > > > > > > > > > > > >> > > > > > assignor to
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and switching one
> > > > consumer
> > > > > > > > group
> > > > > > > > > to a
> > > > > > > > > > > > > > >> client-side
> > > > > > > > > > > > > > >> > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > may be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > faster
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > and less risky than
> > > > > > patching
> > > > > > > > all
> > > > > > > > > of
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> brokers.
> > > > > > > > > > > > > > >> > > > With
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > currently
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > proposed distinct
> > > > APIs, a
> > > > > > > > > non-trivial
> > > > > > > > > > > > > > >> > > > reimplementation
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > assembled, and if
> > the
> > > > two
> > > > > > APIs
> > > > > > > > > have
> > > > > > > > > > > > > diverged
> > > > > > > > > > > > > > >> > > > > > significantly,
> > > > > > > > > > > > > > >> > > > > > > > > then it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > possible that a
> > > > > > > > reimplementation
> > > > > > > > > > > would
> > > > > > > > > > > > > not be
> > > > > > > > > > > > > > >> > > > possible.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > --
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > Greg Harris
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > gharris1727@gmail.com
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > github.com/gharris1727
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Aug 3,
> > 2022 at
> > > > > > 8:39 AM
> > > > > > > > > Sagar
> > > > > > > > > > > <
> > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Hi
> > Guozhang/David,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > I created a
> > > > confluence
> > > > > > page
> > > > > > > > to
> > > > > > > > > > > > discuss
> > > > > > > > > > > > > how
> > > > > > > > > > > > > > >> Connect
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > change
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > based on the new
> > > > > > rebalance
> > > > > > > > > > > protocol.
> > > > > > > > > > > > > > >> Here's the
> > > > > > > > > > > > > > >> > > > page:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/%5BDRAFT%5DIntegrating+Kafka+Connect+With+New+Consumer+Rebalance+Protocol
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > It's also pretty
> > > > longish
> > > > > > and
> > > > > > > > I
> > > > > > > > > have
> > > > > > > > > > > > > tried
> > > > > > > > > > > > > > >> to keep
> > > > > > > > > > > > > > >> > > > a
> > > > > > > > > > > > > > >> > > > > > format
> > > > > > > > > > > > > > >> > > > > > > > > > > >> similar to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > KIP-848. Let me
> > know
> > > > > > what you
> > > > > > > > > > > think.
> > > > > > > > > > > > > Also,
> > > > > > > > > > > > > > >> do you
> > > > > > > > > > > > > > >> > > > > > think this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > moved to a
> > separate
> > > > > > > > discussion
> > > > > > > > > > > thread
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > >> is this
> > > > > > > > > > > > > > >> > > > one
> > > > > > > > > > > > > > >> > > > > > fine?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Thanks!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Sagar.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul 26,
> > 2022
> > > > at
> > > > > > 7:37
> > > > > > > > AM
> > > > > > > > > > > > Sagar <
> > > > > > > > > > > > > > >> > > > > > > > > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Hello Guozhang,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thank you so
> > much
> > > > for
> > > > > > the
> > > > > > > > > doc on
> > > > > > > > > > > > > Kafka
> > > > > > > > > > > > > > >> Streams.
> > > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > > >> > > > > > > > > would do
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > analysis and
> > come
> > > > up
> > > > > > with
> > > > > > > > > such a
> > > > > > > > > > > > > > >> document.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Sagar.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Tue, Jul 26,
> > > > 2022 at
> > > > > > > > 4:47
> > > > > > > > > AM
> > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > >> Wang <
> > > > > > > > > > > > > > >> > > > > > > > > > > >> wangguoz@gmail.com>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Hello Sagar,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> It would be
> > great
> > > > if
> > > > > > you
> > > > > > > > > could
> > > > > > > > > > > > come
> > > > > > > > > > > > > > >> back with
> > > > > > > > > > > > > > >> > > > some
> > > > > > > > > > > > > > >> > > > > > > > > analysis on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > how to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> implement the
> > > > Connect
> > > > > > side
> > > > > > > > > > > > > integration
> > > > > > > > > > > > > > >> with
> > > > > > > > > > > > > > >> > > > the new
> > > > > > > > > > > > > > >> > > > > > > > > protocol;
> > > > > > > > > > > > > > >> > > > > > > > > > > >> so
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > far
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> besides
> > > > leveraging on
> > > > > > the
> > > > > > > > > new
> > > > > > > > > > > > > "protocol
> > > > > > > > > > > > > > >> type"
> > > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > > >> > > > > > did not
> > > > > > > > > > > > > > >> > > > > > > > > yet
> > > > > > > > > > > > > > >> > > > > > > > > > > >> think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > through
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the Connect
> > side
> > > > > > > > > > > implementations.
> > > > > > > > > > > > > For
> > > > > > > > > > > > > > >> Streams
> > > > > > > > > > > > > > >> > > > > > here's a
> > > > > > > > > > > > > > >> > > > > > > > > draft of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> integration
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> plan:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://docs.google.com/document/d/17PNz2sGoIvGyIzz8vLyJTJTU2rqnD_D9uHJnH9XARjU/edit#heading=h.pdgirmi57dvn
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> just FYI for
> > your
> > > > > > analysis
> > > > > > > > > on
> > > > > > > > > > > > > Connect.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> On Tue, Jul
> > 19,
> > > > 2022
> > > > > > at
> > > > > > > > > 10:48 PM
> > > > > > > > > > > > > Sagar <
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > sagarmeansocean@gmail.com
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Hi David,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thank you
> > for
> > > > your
> > > > > > > > > response.
> > > > > > > > > > > The
> > > > > > > > > > > > > > >> reason I
> > > > > > > > > > > > > > >> > > > thought
> > > > > > > > > > > > > > >> > > > > > > > > connect can
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > also fit
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> into
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > this new
> > scheme
> > > > is
> > > > > > that
> > > > > > > > > even
> > > > > > > > > > > > > today the
> > > > > > > > > > > > > > >> > > > connect
> > > > > > > > > > > > > > >> > > > > > uses a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > WorkerCoordinator
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > extending
> > from
> > > > > > > > > > > > > AbstractCoordinator to
> > > > > > > > > > > > > > >> empower
> > > > > > > > > > > > > > >> > > > > > > > > rebalances of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > tasks/connectors.
> > > > > > The
> > > > > > > > > > > > > > >> WorkerCoordinator sets
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> protocolType()
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> connect
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > and uses the
> > > > > > metadata()
> > > > > > > > > method
> > > > > > > > > > > > by
> > > > > > > > > > > > > > >> plumbing
> > > > > > > > > > > > > > >> > > > into
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > JoinGroupRequestProtocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > I think the
> > > > changes
> > > > > > to
> > > > > > > > > support
> > > > > > > > > > > > > > >> connect would
> > > > > > > > > > > > > > >> > > > be
> > > > > > > > > > > > > > >> > > > > > > > > similar at a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > high
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > level
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the changes
> > in
> > > > > > streams
> > > > > > > > > mainly
> > > > > > > > > > > > > because
> > > > > > > > > > > > > > >> of the
> > > > > > > > > > > > > > >> > > > > > Client
> > > > > > > > > > > > > > >> > > > > > > > > side
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > assignors
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > being
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > used in
> > both.
> > > > At an
> > > > > > > > > > > > implementation
> > > > > > > > > > > > > > >> level, we
> > > > > > > > > > > > > > >> > > > > > might
> > > > > > > > > > > > > > >> > > > > > > > > need to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> make
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > a lot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > changes to
> > get
> > > > onto
> > > > > > this
> > > > > > > > > new
> > > > > > > > > > > > > > >> assignment
> > > > > > > > > > > > > > >> > > > protocol
> > > > > > > > > > > > > > >> > > > > > like
> > > > > > > > > > > > > > >> > > > > > > > > > > >> enhancing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > JoinGroup
> > > > > > > > > request/response and
> > > > > > > > > > > > > > >> SyncGroup and
> > > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > ConsumerGroupHeartbeat
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > API etc
> > again on
> > > > > > similar
> > > > > > > > > lines
> > > > > > > > > > > > to
> > > > > > > > > > > > > > >> streams (or
> > > > > > > > > > > > > > >> > > > > > there
> > > > > > > > > > > > > > >> > > > > > > > > might be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> deviations). I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > would try to
> > > > > > perform a
> > > > > > > > > > > detailed
> > > > > > > > > > > > > > >> analysis of
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > same
> > > > > > > > > > > > > > >> > > > > > > > > and we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > can have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > separate
> > > > discussion
> > > > > > > > > thread for
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > >> as that
> > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > >> > > > > > > > > derail this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > discussion
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > thread. Let
> > me
> > > > know
> > > > > > if
> > > > > > > > > that
> > > > > > > > > > > > sounds
> > > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > > >> > > > you.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Thanks!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > Sagar.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > On Fri, Jul
> > 15,
> > > > > > 2022 at
> > > > > > > > > 5:47
> > > > > > > > > > > PM
> > > > > > > > > > > > > David
> > > > > > > > > > > > > > >> Jacot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Hi Sagar,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Thanks for
> > > > your
> > > > > > > > > comments.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 1) Yes.
> > That
> > > > > > refers to
> > > > > > > > > > > > > > >> `Assignment#error`.
> > > > > > > > > > > > > > >> > > > > > Sure, I
> > > > > > > > > > > > > > >> > > > > > > > > can
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > mention it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 2) The
> > idea
> > > > is to
> > > > > > > > > > > transition C
> > > > > > > > > > > > > from
> > > > > > > > > > > > > > >> his
> > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > >> > > > > > > > > assignment
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > his
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > target
> > > > assignment
> > > > > > when
> > > > > > > > > he
> > > > > > > > > > > can
> > > > > > > > > > > > > move
> > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > epoch 3.
> > > > > > > > > > > > > > >> > > > > > When
> > > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > happens,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > > assignment
> > > > > > is
> > > > > > > > > updated
> > > > > > > > > > > > and
> > > > > > > > > > > > > > >> persisted
> > > > > > > > > > > > > > >> > > > > > with all
> > > > > > > > > > > > > > >> > > > > > > > > its
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > assigned
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > partitions
> > > > even if
> > > > > > > > they
> > > > > > > > > are
> > > > > > > > > > > > not
> > > > > > > > > > > > > all
> > > > > > > > > > > > > > >> revoked
> > > > > > > > > > > > > > >> > > > > > yet. In
> > > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > words, the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > member
> > > > assignment
> > > > > > > > > becomes
> > > > > > > > > > > the
> > > > > > > > > > > > > target
> > > > > > > > > > > > > > >> > > > > > assignment.
> > > > > > > > > > > > > > >> > > > > > > > > This is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > basically
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > optimization
> > > > to
> > > > > > avoid
> > > > > > > > > having
> > > > > > > > > > > > to
> > > > > > > > > > > > > > >> write all
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > changes to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > log.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > examples
> > are
> > > > > > based on
> > > > > > > > > the
> > > > > > > > > > > > > persisted
> > > > > > > > > > > > > > >> state
> > > > > > > > > > > > > > >> > > > so I
> > > > > > > > > > > > > > >> > > > > > > > > understand
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > confusion.
> > > > Let me
> > > > > > see
> > > > > > > > > if I
> > > > > > > > > > > can
> > > > > > > > > > > > > > >> improve
> > > > > > > > > > > > > > >> > > > this in
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > description.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > 3)
> > Regarding
> > > > > > Connect,
> > > > > > > > it
> > > > > > > > > > > could
> > > > > > > > > > > > > > >> reuse the
> > > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > > >> > > > > > > > > with a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > client side
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > assignor
> > if it
> > > > > > fits in
> > > > > > > > > the
> > > > > > > > > > > > > > >> protocol. The
> > > > > > > > > > > > > > >> > > > > > assignment
> > > > > > > > > > > > > > >> > > > > > > > > is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> about
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > topicid-partitions +
> > > > > > > > > > > metadata,
> > > > > > > > > > > > > could
> > > > > > > > > > > > > > >> > > > Connect
> > > > > > > > > > > > > > >> > > > > > fit
> > > > > > > > > > > > > > >> > > > > > > > > into this?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Best,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > On Fri,
> > Jul
> > > > 15,
> > > > > > 2022
> > > > > > > > at
> > > > > > > > > 1:55
> > > > > > > > > > > > PM
> > > > > > > > > > > > > > >> Sagar <
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > sagarmeansocean@gmail.com>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Hi
> > David,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Thanks
> > for
> > > > the
> > > > > > KIP.
> > > > > > > > I
> > > > > > > > > just
> > > > > > > > > > > > had
> > > > > > > > > > > > > > >> minor
> > > > > > > > > > > > > > >> > > > > > observations:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 1) In
> > the
> > > > > > Assignment
> > > > > > > > > Error
> > > > > > > > > > > > > > >> section in
> > > > > > > > > > > > > > >> > > > Client
> > > > > > > > > > > > > > >> > > > > > Side
> > > > > > > > > > > > > > >> > > > > > > > > mode
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Assignment
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > process,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > you
> > > > mentioned
> > > > > > => `In
> > > > > > > > > this
> > > > > > > > > > > > > case,
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > client
> > > > > > > > > > > > > > >> > > > > > side
> > > > > > > > > > > > > > >> > > > > > > > > assignor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> can
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > return
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > error
> > to the
> > > > > > group
> > > > > > > > > > > > > coordinator`.
> > > > > > > > > > > > > > >> In this
> > > > > > > > > > > > > > >> > > > > > case are
> > > > > > > > > > > > > > >> > > > > > > > > you
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > referring to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Assignor
> > > > > > returning
> > > > > > > > an
> > > > > > > > > > > > > > >> AssignmentError
> > > > > > > > > > > > > > >> > > > that's
> > > > > > > > > > > > > > >> > > > > > > > > listed down
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > towards
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > end?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > If yes,
> > do
> > > > you
> > > > > > think
> > > > > > > > > it
> > > > > > > > > > > > would
> > > > > > > > > > > > > > >> make sense
> > > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > > >> > > > > > > > > mention this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > explicitly
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > here?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > 2) In
> > the
> > > > Case
> > > > > > > > Studies
> > > > > > > > > > > > > section, I
> > > > > > > > > > > > > > >> have a
> > > > > > > > > > > > > > >> > > > > > slight
> > > > > > > > > > > > > > >> > > > > > > > > > > >> confusion,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > sure
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> if
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > others
> > have
> > > > the
> > > > > > > > same.
> > > > > > > > > > > > Consider
> > > > > > > > > > > > > > >> this step:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > When B
> > > > > > heartbeats,
> > > > > > > > the
> > > > > > > > > > > group
> > > > > > > > > > > > > > >> coordinator
> > > > > > > > > > > > > > >> > > > > > > > > transitions him
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > epoch
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 3
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > because
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > B has no
> > > > > > partitions
> > > > > > > > to
> > > > > > > > > > > > > revoke. It
> > > > > > > > > > > > > > >> > > > persists
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > change and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > reply.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > Group
> > > > > > (epoch=3)
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > A
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > B
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > C
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > Target
> > > > > > > > Assignment
> > > > > > > > > > > > > (epoch=3)
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > A -
> > > > > > > > > > > > partitions=[foo-0]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > B -
> > > > > > > > > > > > partitions=[foo-2]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > C -
> > > > > > > > > > > > partitions=[foo-1]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > Member
> > > > > > > > Assignment
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > A -
> > > > > > epoch=2,
> > > > > > > > > > > > > > >> partitions=[foo-0,
> > > > > > > > > > > > > > >> > > > > > foo-1]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > B -
> > > > > > epoch=3,
> > > > > > > > > > > > > > >> partitions=[foo-2]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > -
> > C -
> > > > > > epoch=3,
> > > > > > > > > > > > > > >> partitions=[foo-1]
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > When C
> > > > > > heartbeats,
> > > > > > > > it
> > > > > > > > > > > > > transitions
> > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > epoch 3
> > > > > > > > > > > > > > >> > > > > > but
> > > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> get
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > foo-1
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> yet.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > Here,it's
> > > > > > mentioned
> > > > > > > > > that
> > > > > > > > > > > > > member C
> > > > > > > > > > > > > > >> can't
> > > > > > > > > > > > > > >> > > > get
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > foo-1
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > partition
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > yet,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > based
> > on the
> > > > > > > > > description
> > > > > > > > > > > > > above,
> > > > > > > > > > > > > > >> it seems
> > > > > > > > > > > > > > >> > > > it
> > > > > > > > > > > > > > >> > > > > > > > > already has
> > > > > > > > > > > > > > >> > > > > > > > > > > >> it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Do you
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > would be
> > > > better
> > > > > > to
> > > > > > > > > remove
> > > > > > > > > > > it
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > >> > > > populate it
> > > > > > > > > > > > > > >> > > > > > only
> > > > > > > > > > > > > > >> > > > > > > > > when it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > actually
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> gets
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > I see
> > this
> > > > in a
> > > > > > lot
> > > > > > > > of
> > > > > > > > > > > other
> > > > > > > > > > > > > > >> places, so
> > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > >> > > > > > I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> understood it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > incorrectly
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > Regarding
> > > > > > connect ,
> > > > > > > > it
> > > > > > > > > > > might
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > >> out of
> > > > > > > > > > > > > > >> > > > scope
> > > > > > > > > > > > > > >> > > > > > of
> > > > > > > > > > > > > > >> > > > > > > > > this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > discussion,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > from
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > what I
> > > > > > understood it
> > > > > > > > > would
> > > > > > > > > > > > > > >> probably be
> > > > > > > > > > > > > > >> > > > > > running in
> > > > > > > > > > > > > > >> > > > > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> assignor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > mode
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > even on
> > the
> > > > new
> > > > > > > > > rebalance
> > > > > > > > > > > > > > >> protocol as it
> > > > > > > > > > > > > > >> > > > has
> > > > > > > > > > > > > > >> > > > > > its
> > > > > > > > > > > > > > >> > > > > > > > > own
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Custom
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > Assignors(Eager
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > and
> > > > > > > > > > > IncrementalCooperative).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Thanks!
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > Sagar.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > On Fri,
> > Jul
> > > > 15,
> > > > > > 2022
> > > > > > > > > at
> > > > > > > > > > > 5:00
> > > > > > > > > > > > > PM
> > > > > > > > > > > > > > >> David
> > > > > > > > > > > > > > >> > > > Jacot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Thanks
> > > > > > Hector! Our
> > > > > > > > > goal
> > > > > > > > > > > is
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > >> move
> > > > > > > > > > > > > > >> > > > forward
> > > > > > > > > > > > > > >> > > > > > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > specialized API
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > instead of
> > > > > > relying
> > > > > > > > > on
> > > > > > > > > > > one
> > > > > > > > > > > > > > >> generic API.
> > > > > > > > > > > > > > >> > > > For
> > > > > > > > > > > > > > >> > > > > > > > > Connect, we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > can apply
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > exact
> > same
> > > > > > pattern
> > > > > > > > > and
> > > > > > > > > > > > > > >> reuse/share the
> > > > > > > > > > > > > > >> > > > core
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > implementation on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > server
> > > > side.
> > > > > > For
> > > > > > > > the
> > > > > > > > > > > > schema
> > > > > > > > > > > > > > >> registry, I
> > > > > > > > > > > > > > >> > > > > > think
> > > > > > > > > > > > > > >> > > > > > > > > that we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consider
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > having a
> > > > > > tailored
> > > > > > > > > API to
> > > > > > > > > > > > do
> > > > > > > > > > > > > > >> simple
> > > > > > > > > > > > > > >> > > > > > > > > membership/leader
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > election.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Best,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > On
> > Fri,
> > > > Jul
> > > > > > 15,
> > > > > > > > > 2022 at
> > > > > > > > > > > > > 10:22
> > > > > > > > > > > > > > >> AM Ismael
> > > > > > > > > > > > > > >> > > > > > Juma <
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > ismael@juma.me.uk
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > Three
> > > > quick
> > > > > > > > > comments:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 1.
> > > > Regarding
> > > > > > > > > > > > > > >> java.util.regex.Pattern
> > > > > > > > > > > > > > >> > > > vs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >>
> > > > > > com.google.re2j.Pattern,
> > > > > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > document the
> > > > > > > > > > > differences
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > >> > > > detail
> > > > > > > > > > > > > > >> > > > > > before
> > > > > > > > > > > > > > >> > > > > > > > > > > >> deciding
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > way
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> or
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > another.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > That
> > > > said,
> > > > > > if
> > > > > > > > > people
> > > > > > > > > > > > pass
> > > > > > > > > > > > > > >> > > > > > > > > java.util.regex.Pattern,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > expect
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > their
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > semantics
> > > > > > to be
> > > > > > > > > > > honored.
> > > > > > > > > > > > > If
> > > > > > > > > > > > > > >> we are
> > > > > > > > > > > > > > >> > > > doing
> > > > > > > > > > > > > > >> > > > > > > > > something
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > different,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> then
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > should
> > > > > > consider
> > > > > > > > > adding
> > > > > > > > > > > > an
> > > > > > > > > > > > > > >> overload
> > > > > > > > > > > > > > >> > > > with
> > > > > > > > > > > > > > >> > > > > > our own
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Pattern
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > class
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > (I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > don't
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > think
> > > > we'd
> > > > > > want
> > > > > > > > to
> > > > > > > > > > > > expose
> > > > > > > > > > > > > > >> re2j's at
> > > > > > > > > > > > > > >> > > > this
> > > > > > > > > > > > > > >> > > > > > > > > point).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 2.
> > > > Regarding
> > > > > > > > topic
> > > > > > > > > > > ids,
> > > > > > > > > > > > > any
> > > > > > > > > > > > > > >> major new
> > > > > > > > > > > > > > >> > > > > > protocol
> > > > > > > > > > > > > > >> > > > > > > > > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > integrate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > it
> > and
> > > > > > should
> > > > > > > > > handle
> > > > > > > > > > > the
> > > > > > > > > > > > > topic
> > > > > > > > > > > > > > >> > > > > > recreation case
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > correctly.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That's
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > main
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > part we
> > > > > > need to
> > > > > > > > > > > handle.
> > > > > > > > > > > > I
> > > > > > > > > > > > > > >> agree with
> > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > >> > > > > > > > > that we'd
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > want to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > add
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > to
> > the
> > > > > > relevant
> > > > > > > > > > > > protocols
> > > > > > > > > > > > > > >> that don't
> > > > > > > > > > > > > > >> > > > > > have it
> > > > > > > > > > > > > > >> > > > > > > > > yet and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > that we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > can
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > probably
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > focus
> > > > on the
> > > > > > > > > internals
> > > > > > > > > > > > > versus
> > > > > > > > > > > > > > >> adding
> > > > > > > > > > > > > > >> > > > new
> > > > > > > > > > > > > > >> > > > > > APIs
> > > > > > > > > > > > > > >> > > > > > > > > to the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > (unless
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > we
> > find
> > > > that
> > > > > > > > > adding
> > > > > > > > > > > new
> > > > > > > > > > > > > APIs
> > > > > > > > > > > > > > >> is
> > > > > > > > > > > > > > >> > > > required
> > > > > > > > > > > > > > >> > > > > > for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> reasonable
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> semantics).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > 3.
> > I am
> > > > > > still
> > > > > > > > not
> > > > > > > > > sure
> > > > > > > > > > > > > about
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > storing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> configs.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It's
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > powerful for
> > > > > > > > > configs
> > > > > > > > > > > to
> > > > > > > > > > > > be
> > > > > > > > > > > > > > >> > > > centralized
> > > > > > > > > > > > > > >> > > > > > in the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> metadata
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > log for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > various
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > reasons
> > > > > > > > > (auditability,
> > > > > > > > > > > > > > >> visibility,
> > > > > > > > > > > > > > >> > > > > > consistency,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> etc.).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Similarly, I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > am
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > sure
> > > > about
> > > > > > > > > > > automatically
> > > > > > > > > > > > > > >> deleting
> > > > > > > > > > > > > > >> > > > > > configs in a
> > > > > > > > > > > > > > >> > > > > > > > > way
> > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > they
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cannot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > recovered. A
> > > > > > > > good
> > > > > > > > > > > > property
> > > > > > > > > > > > > > >> for modern
> > > > > > > > > > > > > > >> > > > > > systems
> > > > > > > > > > > > > > >> > > > > > > > > is to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > minimize
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > number
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > unrecoverable
> > > > > > > > data
> > > > > > > > > > > loss
> > > > > > > > > > > > > > >> scenarios.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > Ismael
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > On
> > Wed,
> > > > Jul
> > > > > > 13,
> > > > > > > > > 2022
> > > > > > > > > > > at
> > > > > > > > > > > > > 3:47
> > > > > > > > > > > > > > >> PM David
> > > > > > > > > > > > > > >> > > > > > Jacot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > Thanks
> > > > > > > > > Guozhang. My
> > > > > > > > > > > > > answers
> > > > > > > > > > > > > > >> are
> > > > > > > > > > > > > > >> > > > below:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > 1)
> > > > the
> > > > > > > > > migration
> > > > > > > > > > > > path,
> > > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > flag
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > enable
> > > > > > > > the
> > > > > > > > > new
> > > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > protocols /
> > > > > > > > > rpcs
> > > > > > > > > > > and
> > > > > > > > > > > > > old
> > > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > group.
> > > > > > How
> > > > > > > > the
> > > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > using
> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > new
> > > > > > protocol
> > > > > > > > > is
> > > > > > > > > > > > > something
> > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > Noted. I
> > > > > > just
> > > > > > > > > > > > published
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > >> new
> > > > > > > > > > > > > > >> > > > version
> > > > > > > > > > > > > > >> > > > > > of KIP
> > > > > > > > > > > > > > >> > > > > > > > > which
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > includes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> more
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > details
> > > > > > about
> > > > > > > > > this.
> > > > > > > > > > > > See
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > > > "Supporting
> > > > > > > > > > > > > > >> > > > > > > > > Online
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Upgrade"
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > and
> > > > the
> > > > > > > > > > > > "Compatibility,
> > > > > > > > > > > > > > >> > > > Deprecation,
> > > > > > > > > > > > > > >> > > > > > and
> > > > > > > > > > > > > > >> > > > > > > > > Migration
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Plan". I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > I
> > > > have to
> > > > > > > > think
> > > > > > > > > > > > through
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > >> few cases
> > > > > > > > > > > > > > >> > > > > > now but
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > overall idea
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > mechanism
> > > > > > > > > should be
> > > > > > > > > > > > > > >> understandable.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > 2)
> > > > the
> > > > > > usage
> > > > > > > > > of
> > > > > > > > > > > > topic
> > > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > used
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > as
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > part of
> > > > > > RPCs
> > > > > > > > > and
> > > > > > > > > > > > admin
> > > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > public
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > APIs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > consumers
> > > > > > > > > yet.
> > > > > > > > > > > I
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > client
> > > > > > to be
> > > > > > > > > > > > > maintaining
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > leverage
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > on
> > > > all
> > > > > > the
> > > > > > > > > > > augmented
> > > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids;
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > secondly,
> > > > > > > > > should
> > > > > > > > > > > we
> > > > > > > > > > > > > ever
> > > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > public
> > > > > > APIs
> > > > > > > > as
> > > > > > > > > > > well
> > > > > > > > > > > > > (both
> > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > rebalance
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > listener for
> > > > > > > > > cases
> > > > > > > > > > > > > like
> > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > deletion-and-recreation).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > a)
> > > > > > Assuming
> > > > > > > > > that we
> > > > > > > > > > > > > would
> > > > > > > > > > > > > > >> include
> > > > > > > > > > > > > > >> > > > > > converting
> > > > > > > > > > > > > > >> > > > > > > > > all
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > offsets
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > related
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > RPCs
> > > > to
> > > > > > using
> > > > > > > > > topic
> > > > > > > > > > > > ids
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > > KIP,
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > would be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> able
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > fully
> > > > > > operate
> > > > > > > > > with
> > > > > > > > > > > > topic
> > > > > > > > > > > > > > >> ids. That
> > > > > > > > > > > > > > >> > > > > > being
> > > > > > > > > > > > > > >> > > > > > > > > said, it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > still has
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > provide
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > the
> > > > topics
> > > > > > > > > names in
> > > > > > > > > > > > > various
> > > > > > > > > > > > > > >> APIs so
> > > > > > > > > > > > > > >> > > > > > having a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> mapping
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > seems
> > > > > > > > > inevitable to
> > > > > > > > > > > > me.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > b) I
> > > > don't
> > > > > > > > have
> > > > > > > > > a
> > > > > > > > > > > > strong
> > > > > > > > > > > > > > >> opinion on
> > > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > > >> > > > > > > > > Here I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > wonder if
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > goes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > beyond the
> > > > > > > > > scope of
> > > > > > > > > > > > this
> > > > > > > > > > > > > > >> KIP. I
> > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > >> > > > > > rather
> > > > > > > > > > > > > > >> > > > > > > > > focus
> > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > internals
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > here
> > > > and
> > > > > > we
> > > > > > > > can
> > > > > > > > > > > > consider
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > > > > separately if
> > > > > > > > > > > > > > >> > > > > > > > > we see
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > value in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> doing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > Coming
> > > > > > back to
> > > > > > > > > > > > Ismael's
> > > > > > > > > > > > > > >> point about
> > > > > > > > > > > > > > >> > > > > > using
> > > > > > > > > > > > > > >> > > > > > > > > topic ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > in the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> ConsumerGroupHeartbeatRequest, I
> > > > > > > > > > > > > > >> > > > think
> > > > > > > > > > > > > > >> > > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > there
> > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > one
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > advantage
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > favour of
> > > > > > it.
> > > > > > > > > The
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> will
> > > > > > > > > > > > > > >> > > > have
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> opportunity
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > the
> > > > topics
> > > > > > > > > exists
> > > > > > > > > > > > before
> > > > > > > > > > > > > > >> passing
> > > > > > > > > > > > > > >> > > > them
> > > > > > > > > > > > > > >> > > > > > into
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > rebalance
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > protocol.
> > > > > > > > > Obviously,
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > > > coordinator
> > > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > > >> > > > > > > > > also
> > > > > > > > > > > > > > >> > > > > > > > > > > >> notice
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > it but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > does
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > not
> > > > really
> > > > > > > > have
> > > > > > > > > a
> > > > > > > > > > > way
> > > > > > > > > > > > to
> > > > > > > > > > > > > > >> reject an
> > > > > > > > > > > > > > >> > > > > > invalid
> > > > > > > > > > > > > > >> > > > > > > > > topic in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> response.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I'm
> > > > > > agreeing
> > > > > > > > > with
> > > > > > > > > > > > > David
> > > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > necessary
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > deprecate
> > > > > > > > the
> > > > > > > > > > > > > subscribe
> > > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > Pattern
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > while
> > > > > > just
> > > > > > > > > > > > documenting
> > > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected by
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > server.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > Since
> > > > > > the
> > > > > > > > > > > > incompatible
> > > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > overloaded
> > > > > > > > > > > `String`
> > > > > > > > > > > > > based
> > > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > various
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > invalid
> > > > > > > > > regexes.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > That
> > > > could
> > > > > > > > > work. I
> > > > > > > > > > > > have
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > >> look at
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > differences
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > between
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > two
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > engines to
> > > > > > > > > better
> > > > > > > > > > > > > > >> understand the
> > > > > > > > > > > > > > >> > > > > > potential
> > > > > > > > > > > > > > >> > > > > > > > > issues.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> My
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > understanding
> > > > is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > that
> > > > would
> > > > > > > > work
> > > > > > > > > for
> > > > > > > > > > > > all
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> basic
> > > > > > > > > > > > > > >> > > > > > regular
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > expressions. The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > differences
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > between
> > > > > > the
> > > > > > > > two
> > > > > > > > > are
> > > > > > > > > > > > > mainly
> > > > > > > > > > > > > > >> about
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > various
> > > > > > > > > > > > > > >> > > > > > > > > > > >> character
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> classes. I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > wonder
> > > > > > what
> > > > > > > > > other
> > > > > > > > > > > > people
> > > > > > > > > > > > > > >> think
> > > > > > > > > > > > > > >> > > > about
> > > > > > > > > > > > > > >> > > > > > this.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > Best,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > On
> > > > Tue,
> > > > > > Jul
> > > > > > > > 12,
> > > > > > > > > 2022
> > > > > > > > > > > > at
> > > > > > > > > > > > > > >> 11:28 PM
> > > > > > > > > > > > > > >> > > > > > Guozhang
> > > > > > > > > > > > > > >> > > > > > > > > Wang <
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> >
> > > > wangguoz@gmail.com
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > Thanks
> > > > > > > > David!
> > > > > > > > > I
> > > > > > > > > > > > think
> > > > > > > > > > > > > on
> > > > > > > > > > > > > > >> the high
> > > > > > > > > > > > > > >> > > > > > level
> > > > > > > > > > > > > > >> > > > > > > > > there are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > two meta
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > points we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > need
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > > > concretize
> > > > > > > > > a
> > > > > > > > > > > bit
> > > > > > > > > > > > > more:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > 1)
> > > > the
> > > > > > > > > migration
> > > > > > > > > > > > path,
> > > > > > > > > > > > > > >> especially
> > > > > > > > > > > > > > >> > > > > > the last
> > > > > > > > > > > > > > >> > > > > > > > > step
> > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > clients
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > flip the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > flag
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > enable
> > > > > > > > the
> > > > > > > > > new
> > > > > > > > > > > > > > >> protocol, in
> > > > > > > > > > > > > > >> > > > which
> > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> have a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > window
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > where
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > protocols /
> > > > > > > > > rpcs
> > > > > > > > > > > and
> > > > > > > > > > > > > old
> > > > > > > > > > > > > > >> > > > protocols /
> > > > > > > > > > > > > > >> > > > > > rpcs
> > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> used
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > by
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> members
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > group.
> > > > > > How
> > > > > > > > the
> > > > > > > > > > > > > > >> coordinator could
> > > > > > > > > > > > > > >> > > > > > "mimic"
> > > > > > > > > > > > > > >> > > > > > > > > the old
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > behavior
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> while
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > using
> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > new
> > > > > > protocol
> > > > > > > > > is
> > > > > > > > > > > > > something
> > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > >> > > > need to
> > > > > > > > > > > > > > >> > > > > > > > > present
> > > > > > > > > > > > > > >> > > > > > > > > > > >> about.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > 2)
> > > > the
> > > > > > usage
> > > > > > > > > of
> > > > > > > > > > > > topic
> > > > > > > > > > > > > > >> ids. So
> > > > > > > > > > > > > > >> > > > far as
> > > > > > > > > > > > > > >> > > > > > > > > KIP-516 the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > only
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > used
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > as
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > part of
> > > > > > RPCs
> > > > > > > > > and
> > > > > > > > > > > > admin
> > > > > > > > > > > > > > >> client,
> > > > > > > > > > > > > > >> > > > but
> > > > > > > > > > > > > > >> > > > > > they
> > > > > > > > > > > > > > >> > > > > > > > > are not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > exposed
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > via
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> any
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > public
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > APIs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > to
> > > > > > consumers
> > > > > > > > > yet.
> > > > > > > > > > > I
> > > > > > > > > > > > > think
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > question is,
> > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > should we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> let
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > client
> > > > > > to be
> > > > > > > > > > > > > maintaining
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > >> > > > > > -> ids
> > > > > > > > > > > > > > >> > > > > > > > > mapping
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > itself
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > fully
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > leverage
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > on
> > > > all
> > > > > > the
> > > > > > > > > > > augmented
> > > > > > > > > > > > > > >> existing
> > > > > > > > > > > > > > >> > > > RPCs
> > > > > > > > > > > > > > >> > > > > > and the
> > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> RPCs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > ids;
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > secondly,
> > > > > > > > > should
> > > > > > > > > > > we
> > > > > > > > > > > > > ever
> > > > > > > > > > > > > > >> consider
> > > > > > > > > > > > > > >> > > > > > exposing
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > public
> > > > > > APIs
> > > > > > > > as
> > > > > > > > > > > well
> > > > > > > > > > > > > (both
> > > > > > > > > > > > > > >> > > > > > > > > subscribe/assign, as
> > > > > > > > > > > > > > >> > > > > > > > > > > >> well
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > as in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > rebalance
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > listener for
> > > > > > > > > cases
> > > > > > > > > > > > > like
> > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > deletion-and-recreation).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > I'm
> > > > > > agreeing
> > > > > > > > > with
> > > > > > > > > > > > > David
> > > > > > > > > > > > > > >> on all
> > > > > > > > > > > > > > >> > > > other
> > > > > > > > > > > > > > >> > > > > > minor
> > > > > > > > > > > > > > >> > > > > > > > > > > >> questions
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > except
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > `subscribe(Pattern)`
> > > > > > > > > > > > > > >> question:
> > > > > > > > > > > > > > >> > > > > > personally
> > > > > > > > > > > > > > >> > > > > > > > > I think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > it's not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > necessary
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > deprecate
> > > > > > > > the
> > > > > > > > > > > > > subscribe
> > > > > > > > > > > > > > >> API with
> > > > > > > > > > > > > > >> > > > > > Pattern,
> > > > > > > > > > > > > > >> > > > > > > > > but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> still
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > use
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > Pattern
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > while
> > > > > > just
> > > > > > > > > > > > documenting
> > > > > > > > > > > > > > >> that our
> > > > > > > > > > > > > > >> > > > > > > > > subscription may
> > > > > > > > > > > > > > >> > > > > > > > > > > >> be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rejected by
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > server.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > Since
> > > > > > the
> > > > > > > > > > > > incompatible
> > > > > > > > > > > > > > >> case is a
> > > > > > > > > > > > > > >> > > > > > very rare
> > > > > > > > > > > > > > >> > > > > > > > > > > >> scenario
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > I felt
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > overloaded
> > > > > > > > > > > `String`
> > > > > > > > > > > > > based
> > > > > > > > > > > > > > >> > > > > > subscription may
> > > > > > > > > > > > > > >> > > > > > > > > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> more
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> vulnerable
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > various
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > invalid
> > > > > > > > > regexes.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > Guozhang
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > On
> > > > Tue,
> > > > > > Jul
> > > > > > > > > 12,
> > > > > > > > > > > 2022
> > > > > > > > > > > > > at
> > > > > > > > > > > > > > >> 5:23 AM
> > > > > > > > > > > > > > >> > > > > > David Jacot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > > > > > > > > > > > <djacot@confluent.io.invalid
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > Hi
> > > > > > Ismael,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > Thanks for
> > > > > > > > > your
> > > > > > > > > > > > > > >> feedback. Let
> > > > > > > > > > > > > > >> > > > me
> > > > > > > > > > > > > > >> > > > > > answer
> > > > > > > > > > > > > > >> > > > > > > > > your
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > questions
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > inline.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 1. I
> > > > > > > > think
> > > > > > > > > > > it's
> > > > > > > > > > > > > > >> premature to
> > > > > > > > > > > > > > >> > > > > > talk about
> > > > > > > > > > > > > > >> > > > > > > > > > > >> target
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > versions
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > deprecation
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > removal
> > > > > > > > > of the
> > > > > > > > > > > > > > >> existing group
> > > > > > > > > > > > > > >> > > > > > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Unlike
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > KRaft,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > affects a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > core
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > client
> > > > > > > > > > > protocol
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > >> hence
> > > > > > > > > > > > > > >> > > > > > > > > deprecation/removal
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > heavily
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > dependent
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > how
> > > > > > > > > quickly
> > > > > > > > > > > > > > >> applications
> > > > > > > > > > > > > > >> > > > migrate
> > > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > That
> > > > > > makes
> > > > > > > > > > > sense.
> > > > > > > > > > > > I
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > >> > > > remove it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 2.
> > > > > > The
> > > > > > > > KIP
> > > > > > > > > > > says
> > > > > > > > > > > > we
> > > > > > > > > > > > > > >> intend to
> > > > > > > > > > > > > > >> > > > > > release
> > > > > > > > > > > > > > >> > > > > > > > > this in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > 4.x, but
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > wasn't
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > made
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > clear
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > why. If
> > > > > > > > we
> > > > > > > > > > > added
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > >> as a
> > > > > > > > > > > > > > >> > > > way to
> > > > > > > > > > > > > > >> > > > > > > > > estimate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > we'd
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > deprecate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > remove
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > the
> > > > > > > > group
> > > > > > > > > > > > > protocol, I
> > > > > > > > > > > > > > >> also
> > > > > > > > > > > > > > >> > > > > > suggest
> > > > > > > > > > > > > > >> > > > > > > > > removing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > this part.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Let me
> > > > > > > > > explain
> > > > > > > > > > > my
> > > > > > > > > > > > > > >> reasoning. As
> > > > > > > > > > > > > > >> > > > > > > > > explained, I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> plan
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> rewrite
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > coordinator
> > > > > > > > > in
> > > > > > > > > > > > Java
> > > > > > > > > > > > > > >> while we
> > > > > > > > > > > > > > >> > > > > > implement
> > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> This
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > that
> > > > > > the
> > > > > > > > > > > internals
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > >> be
> > > > > > > > > > > > > > >> > > > slightly
> > > > > > > > > > > > > > >> > > > > > > > > different
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > (e.g.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> threading
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > model).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > Therefore, I
> > > > > > > > > > > > wanted
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > >> tighten
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > switch from
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the old
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > coordinator
> > > > > > > > > to
> > > > > > > > > > > the
> > > > > > > > > > > > > new
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > to a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > major
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> release.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > alternative
> > > > > > > > > > > would
> > > > > > > > > > > > > be to
> > > > > > > > > > > > > > >> use a
> > > > > > > > > > > > > > >> > > > flag
> > > > > > > > > > > > > > >> > > > > > to do
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instead
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > relying
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > on the
> > > > > > > > > software
> > > > > > > > > > > > > upgrade.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 3.
> > > > > > We
> > > > > > > > > need to
> > > > > > > > > > > > > flesh
> > > > > > > > > > > > > > >> out the
> > > > > > > > > > > > > > >> > > > > > details of
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > migration
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> story.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > It
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > sounds
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > like
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > we're
> > > > > > > > > saying
> > > > > > > > > > > we
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > >> support
> > > > > > > > > > > > > > >> > > > > > online
> > > > > > > > > > > > > > >> > > > > > > > > > > >> migrations.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > correct?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > We
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > explain
> > > > > > > > > this
> > > > > > > > > > > in
> > > > > > > > > > > > > > >> detail. It
> > > > > > > > > > > > > > >> > > > could
> > > > > > > > > > > > > > >> > > > > > also
> > > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > as a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> separate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > KIP,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > if
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > it's
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > easier.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Yes,
> > > > > > we
> > > > > > > > will
> > > > > > > > > > > > support
> > > > > > > > > > > > > > >> online
> > > > > > > > > > > > > > >> > > > > > migrations
> > > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > group.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > That
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > means
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > a
> > > > > > group
> > > > > > > > > using
> > > > > > > > > > > the
> > > > > > > > > > > > > old
> > > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > > >> > > > > > > > > able to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > switch to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Let me
> > > > > > > > > briefly
> > > > > > > > > > > > > explain
> > > > > > > > > > > > > > >> how that
> > > > > > > > > > > > > > >> > > > > > will work
> > > > > > > > > > > > > > >> > > > > > > > > > > >> though.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > It is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > basically
> > a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > four
> > > > > > step
> > > > > > > > > > > process:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > 1. The
> > > > > > > > > cluster
> > > > > > > > > > > > must
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > >> > > > upgraded or
> > > > > > > > > > > > > > >> > > > > > > > > rolled to a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > software
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > supporting
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > new
> > > > > > group
> > > > > > > > > > > > > coordinator.
> > > > > > > > > > > > > > >> Both the
> > > > > > > > > > > > > > >> > > > > > old and
> > > > > > > > > > > > > > >> > > > > > > > > the new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> coordinator
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > will
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > support
> > > > > > > > the
> > > > > > > > > old
> > > > > > > > > > > > > > >> protocol and
> > > > > > > > > > > > > > >> > > > rely
> > > > > > > > > > > > > > >> > > > > > on the
> > > > > > > > > > > > > > >> > > > > > > > > same
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > persisted
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > metadata
> > so
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > they
> > > > > > can
> > > > > > > > > work
> > > > > > > > > > > > > together.
> > > > > > > > > > > > > > >> This
> > > > > > > > > > > > > > >> > > > point
> > > > > > > > > > > > > > >> > > > > > is an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> offline
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> migration.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > We
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > cannot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > do
> > > > > > this
> > > > > > > > one
> > > > > > > > > live
> > > > > > > > > > > > > > >> because it
> > > > > > > > > > > > > > >> > > > would
> > > > > > > > > > > > > > >> > > > > > require
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > shutting down
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > current
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > coordinator
> > > > > > > > > and
> > > > > > > > > > > > > > >> starting up the
> > > > > > > > > > > > > > >> > > > > > new one
> > > > > > > > > > > > > > >> > > > > > > > > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> cause
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > unavailabilities.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > 2. The
> > > > > > > > > cluster's
> > > > > > > > > > > > > > >> metadata
> > > > > > > > > > > > > > >> > > > > > version/IBP
> > > > > > > > > > > > > > >> > > > > > > > > must be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> X
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > order
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > to
> > > > > > enable
> > > > > > > > > the
> > > > > > > > > > > new
> > > > > > > > > > > > > > >> protocol.
> > > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > > >> > > > > > cannot
> > > > > > > > > > > > > > >> > > > > > > > > be done
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > 1)
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > terminated
> > > > > > > > > > > because
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> old
> > > > > > > > > > > > > > >> > > > > > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > doesn't
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > support
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > protocol.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > 3. The
> > > > > > > > > consumers
> > > > > > > > > > > > > must be
> > > > > > > > > > > > > > >> > > > upgraded
> > > > > > > > > > > > > > >> > > > > > to a
> > > > > > > > > > > > > > >> > > > > > > > > version
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > supporting
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > online
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > migration
> > > > > > > > > (must
> > > > > > > > > > > > have
> > > > > > > > > > > > > > >> KIP-792).
> > > > > > > > > > > > > > >> > > > If
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > already
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > there.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > Nothing
> > > > > > > > > must be
> > > > > > > > > > > > > done at
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > > point.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > 4. The
> > > > > > > > > consumers
> > > > > > > > > > > > > must
> > > > > > > > > > > > > > >> be rolled
> > > > > > > > > > > > > > >> > > > > > with the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> turned
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > on.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > consumer
> > > > > > > > > group
> > > > > > > > > > > is
> > > > > > > > > > > > > > >> automatically
> > > > > > > > > > > > > > >> > > > > > > > > converted when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > using
> > > > > > the
> > > > > > > > > new
> > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > >> joins
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > group.
> > > > > > > > > > > > > > >> > > > > > > > > While
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > members
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > using the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > old
> > > > > > > > > protocol are
> > > > > > > > > > > > > being
> > > > > > > > > > > > > > >> > > > upgraded,
> > > > > > > > > > > > > > >> > > > > > the old
> > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> proxied
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > into
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > new
> > > > > > one.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Let me
> > > > > > > > > clarify
> > > > > > > > > > > all
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > >> this in
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > KIP.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 4.
> > > > > > I am
> > > > > > > > > happy
> > > > > > > > > > > > > that we
> > > > > > > > > > > > > > >> are
> > > > > > > > > > > > > > >> > > > > > pushing the
> > > > > > > > > > > > > > >> > > > > > > > > pattern
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> subscriptions
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > server,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > but
> > > > > > it
> > > > > > > > > seems
> > > > > > > > > > > > like
> > > > > > > > > > > > > > >> there
> > > > > > > > > > > > > > >> > > > could be
> > > > > > > > > > > > > > >> > > > > > some
> > > > > > > > > > > > > > >> > > > > > > > > tricky
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> compatibility
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > issues.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > Will
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > have a
> > > > > > > > > > > mechanism
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > >> users to
> > > > > > > > > > > > > > >> > > > > > detect
> > > > > > > > > > > > > > >> > > > > > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> they
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > update
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > their
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > regex
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > before
> > > > > > > > > > > switching
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > >> the new
> > > > > > > > > > > > > > >> > > > > > protocol?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > I
> > > > > > think
> > > > > > > > > that I
> > > > > > > > > > > am
> > > > > > > > > > > > a
> > > > > > > > > > > > > bit
> > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > >> > > > > > optimistic
> > > > > > > > > > > > > > >> > > > > > > > > than
> > > > > > > > > > > > > > >> > > > > > > > > > > >> you
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > on this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > point. I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > believe
> > > > > > > > > that the
> > > > > > > > > > > > > > >> majority of
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > cases
> > > > > > > > > > > > > > >> > > > > > > > > are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> simple
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > regexes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > which
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > work
> > > > > > with
> > > > > > > > > the
> > > > > > > > > > > new
> > > > > > > > > > > > > > >> engine. The
> > > > > > > > > > > > > > >> > > > > > > > > coordinator will
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > verify
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > regex
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > anyway
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > and
> > > > > > reject
> > > > > > > > > the
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> if the
> > > > > > > > > > > > > > >> > > > > > regex is
> > > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> valid.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > Coming
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > back
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > migration
> > > > > > > > > path,
> > > > > > > > > > > in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> worst
> > > > > > > > > > > > > > >> > > > case,
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > upgraded
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > joining
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > the
> > > > > > group
> > > > > > > > > will
> > > > > > > > > > > be
> > > > > > > > > > > > > > >> rejected.
> > > > > > > > > > > > > > >> > > > This
> > > > > > > > > > > > > > >> > > > > > should
> > > > > > > > > > > > > > >> > > > > > > > > be used
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > as the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> last
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > defence, I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > would
> > > > > > say.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > One
> > > > > > way
> > > > > > > > for
> > > > > > > > > > > > > customers to
> > > > > > > > > > > > > > >> > > > validate
> > > > > > > > > > > > > > >> > > > > > their
> > > > > > > > > > > > > > >> > > > > > > > > regex
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > before
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > upgrading
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > their
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > prod
> > > > > > would
> > > > > > > > > be to
> > > > > > > > > > > > > test
> > > > > > > > > > > > > > >> them with
> > > > > > > > > > > > > > >> > > > > > another
> > > > > > > > > > > > > > >> > > > > > > > > group.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> For
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> instance,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > could
> > > > > > be
> > > > > > > > > done
> > > > > > > > > > > in a
> > > > > > > > > > > > > > >> pre-prod
> > > > > > > > > > > > > > >> > > > > > environment.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Another
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > way
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> would be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > extend the
> > > > > > > > > > > > > > >> consumer-group tool
> > > > > > > > > > > > > > >> > > > to
> > > > > > > > > > > > > > >> > > > > > > > > provide a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> regex
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> validation
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > mechanism.
> > > > > > > > > Would
> > > > > > > > > > > > > this be
> > > > > > > > > > > > > > >> > > > enough in
> > > > > > > > > > > > > > >> > > > > > your
> > > > > > > > > > > > > > >> > > > > > > > > > > >> opinion?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 5.
> > > > > > > > > Related to
> > > > > > > > > > > > the
> > > > > > > > > > > > > last
> > > > > > > > > > > > > > >> > > > question,
> > > > > > > > > > > > > > >> > > > > > will
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> Java
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > client
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> allow
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > users
> > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > stick
> > > > > > > > > with the
> > > > > > > > > > > > > > >> current regex
> > > > > > > > > > > > > > >> > > > > > engine for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > compatibility
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > reasons?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > For
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > example,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > it
> > > > > > may
> > > > > > > > be
> > > > > > > > > > > handy
> > > > > > > > > > > > to
> > > > > > > > > > > > > > >> keep using
> > > > > > > > > > > > > > >> > > > > > client
> > > > > > > > > > > > > > >> > > > > > > > > based
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > regex at
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > keep
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > migrations
> > > > > > > > > > > > simple
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > >> then
> > > > > > > > > > > > > > >> > > > > > migrate to
> > > > > > > > > > > > > > >> > > > > > > > > server
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > based
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regexes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > as a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > second
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > step.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > I
> > > > > > > > understand
> > > > > > > > > > > your
> > > > > > > > > > > > > point
> > > > > > > > > > > > > > >> but I
> > > > > > > > > > > > > > >> > > > am
> > > > > > > > > > > > > > >> > > > > > > > > concerned that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > allow
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > users
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > to
> > > > > > > > actually
> > > > > > > > > stay
> > > > > > > > > > > > in
> > > > > > > > > > > > > > >> this mode.
> > > > > > > > > > > > > > >> > > > That
> > > > > > > > > > > > > > >> > > > > > > > > would go
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > against our
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> goal
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > simplifying
> > > > > > > > > the
> > > > > > > > > > > > > client
> > > > > > > > > > > > > > >> because
> > > > > > > > > > > > > > >> > > > we
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > continue
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > monitoring
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > the
> > > > > > > > > metadata on
> > > > > > > > > > > > the
> > > > > > > > > > > > > > >> client
> > > > > > > > > > > > > > >> > > > side. I
> > > > > > > > > > > > > > >> > > > > > would
> > > > > > > > > > > > > > >> > > > > > > > > rather
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > not do
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> this.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 6.
> > > > > > When
> > > > > > > > > we say
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > >> the group
> > > > > > > > > > > > > > >> > > > > > > > > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > > > >> will
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > responsible
> > > > for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > storing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > the
> > > > > > > > > > > > configurations
> > > > > > > > > > > > > > >> and that
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> configurations
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > will be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > deleted
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > group is
> > > > > > > > > > > > deleted.
> > > > > > > > > > > > > > >> Will a
> > > > > > > > > > > > > > >> > > > > > transition to
> > > > > > > > > > > > > > >> > > > > > > > > DEAD
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > trigger
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > deletion
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > configurations?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > That's
> > > > > > > > > right.
> > > > > > > > > > > The
> > > > > > > > > > > > > > >> > > > configurations
> > > > > > > > > > > > > > >> > > > > > will be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> deleted
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > deleted.
> > > > > > > > > They go
> > > > > > > > > > > > > > >> together.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 7.
> > > > > > Will
> > > > > > > > > the
> > > > > > > > > > > > > choice to
> > > > > > > > > > > > > > >> store
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > configs in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > make
> > > > it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > harder
> > > > > > > > to
> > > > > > > > > list
> > > > > > > > > > > > all
> > > > > > > > > > > > > > >> cluster
> > > > > > > > > > > > > > >> > > > > > configs and
> > > > > > > > > > > > > > >> > > > > > > > > their
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > values?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > I
> > > > > > don't
> > > > > > > > > think
> > > > > > > > > > > so.
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > >> group
> > > > > > > > > > > > > > >> > > > > > > > > configurations are
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > overrides
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > cluster
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > configs.
> > > > > > > > If
> > > > > > > > > you
> > > > > > > > > > > > > want to
> > > > > > > > > > > > > > >> know
> > > > > > > > > > > > > > >> > > > all
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > overrides
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > though,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > you
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > to
> > > > > > ask all
> > > > > > > > > the
> > > > > > > > > > > > group
> > > > > > > > > > > > > > >> > > > coordinators.
> > > > > > > > > > > > > > >> > > > > > You
> > > > > > > > > > > > > > >> > > > > > > > > cannot
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > rely on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > metadata
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > log
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > for
> > > > > > > > > instance.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 8.
> > > > > > How
> > > > > > > > > would
> > > > > > > > > > > > > someone
> > > > > > > > > > > > > > >> > > > configure a
> > > > > > > > > > > > > > >> > > > > > group
> > > > > > > > > > > > > > >> > > > > > > > > before
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > starting
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > consumers?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > we
> > > > > > > > > considered
> > > > > > > > > > > > > > >> allowing the
> > > > > > > > > > > > > > >> > > > > > explicit
> > > > > > > > > > > > > > >> > > > > > > > > creation
> > > > > > > > > > > > > > >> > > > > > > > > > > >> of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > groups?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > > > > > Alternatively,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > configs
> > > > > > > > > could
> > > > > > > > > > > be
> > > > > > > > > > > > > > >> decoupled
> > > > > > > > > > > > > > >> > > > from
> > > > > > > > > > > > > > >> > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > lifecycle.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Yes.
> > > > > > The
> > > > > > > > > group
> > > > > > > > > > > > will
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > >> > > > > > automatically
> > > > > > > > > > > > > > >> > > > > > > > > created in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > However,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > the
> > > > > > > > configs
> > > > > > > > > will
> > > > > > > > > > > > be
> > > > > > > > > > > > > > >> lost after
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > retention
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > period of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > passes.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 9.
> > > > > > Will
> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > Consumer.subscribe
> > > > > > > > > > > > > > >> > > > > > method
> > > > > > > > > > > > > > >> > > > > > > > > for the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > Java
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> client
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > still
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > take
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > > > > >> `java.util.regex.Pattern` of
> > > > > > > > > > > > > > >> > > > do
> > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > have to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > introduce
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > overload?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > That's a
> > > > > > > > > very
> > > > > > > > > > > > group
> > > > > > > > > > > > > > >> question. I
> > > > > > > > > > > > > > >> > > > > > forgot
> > > > > > > > > > > > > > >> > > > > > > > > about
> > > > > > > > > > > > > > >> > > > > > > > > > > >> that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > one.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > As
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> `java.util.regex.Pattern` is
> > > > > > > > > > > > > > >> > > > not
> > > > > > > > > > > > > > >> > > > > > fully
> > > > > > > > > > > > > > >> > > > > > > > > > > >> compatible
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > engine
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > we
> > > > > > plan to
> > > > > > > > > use,
> > > > > > > > > > > it
> > > > > > > > > > > > > > >> might be
> > > > > > > > > > > > > > >> > > > better
> > > > > > > > > > > > > > >> > > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > deprecate
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > it and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> use an
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > overload
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > which
> > > > > > > > takes
> > > > > > > > > a
> > > > > > > > > > > > > string.
> > > > > > > > > > > > > > >> We would
> > > > > > > > > > > > > > >> > > > > > rely on
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> server
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > side
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > validation.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > During the
> > > > > > > > > > > > > migration, I
> > > > > > > > > > > > > > >> think
> > > > > > > > > > > > > > >> > > > that
> > > > > > > > > > > > > > >> > > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > > >> > > > > > > > > > > >> still
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > try to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > toString
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > regex
> > > > > > and
> > > > > > > > > use
> > > > > > > > > > > it.
> > > > > > > > > > > > > That
> > > > > > > > > > > > > > >> should
> > > > > > > > > > > > > > >> > > > > > work, I
> > > > > > > > > > > > > > >> > > > > > > > > think, in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> majority
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > of the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > cases.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 10.
> > > > > > I
> > > > > > > > > agree
> > > > > > > > > > > with
> > > > > > > > > > > > > > >> Justine
> > > > > > > > > > > > > > >> > > > that we
> > > > > > > > > > > > > > >> > > > > > > > > should be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > clearer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > about
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > reason
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > switch
> > > > > > > > to
> > > > > > > > > > > > > > >> > > > IBP/metadata.version
> > > > > > > > > > > > > > >> > > > > > from the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> feature
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > flag.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Maybe
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > mean
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > can
> > > > > > > > > switch the
> > > > > > > > > > > > > > >> default for
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > feature
> > > > > > > > > > > > > > >> > > > > > > > > flag
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > true
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> based
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > metadata.version
> > > > > > > > > > > > > once
> > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > >> > > > want to
> > > > > > > > > > > > > > >> > > > > > make
> > > > > > > > > > > > > > >> > > > > > > > > it the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > default.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > My
> > > > > > plan
> > > > > > > > was
> > > > > > > > > to
> > > > > > > > > > > use
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > >> feature
> > > > > > > > > > > > > > >> > > > > > flag
> > > > > > > > > > > > > > >> > > > > > > > > mainly
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > during the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > development
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > phase. I
> > > > > > > > > should
> > > > > > > > > > > > not
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > >> > > > mentioned
> > > > > > > > > > > > > > >> > > > > > it, I
> > > > > > > > > > > > > > >> > > > > > > > > think,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > because
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > could
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > use
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > an
> > > > > > > > internal
> > > > > > > > > > > config
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > >> it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 11.
> > > > > > Some
> > > > > > > > > of
> > > > > > > > > > > the
> > > > > > > > > > > > > > >> protocol APIs
> > > > > > > > > > > > > > >> > > > > > don't
> > > > > > > > > > > > > > >> > > > > > > > > mention
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > required
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > ACLs, it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > would
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > good to
> > > > > > > > > add
> > > > > > > > > > > that
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > >> > > > consistency.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Noted.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 12.
> > > > > > It
> > > > > > > > is
> > > > > > > > > a
> > > > > > > > > > > bit
> > > > > > > > > > > > > odd
> > > > > > > > > > > > > > >> that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> ConsumerGroupHeartbeat
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > requires
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > "Read
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Group"
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > even
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > though
> > > > > > > > it
> > > > > > > > > > > seems
> > > > > > > > > > > > > to do
> > > > > > > > > > > > > > >> more
> > > > > > > > > > > > > > >> > > > than
> > > > > > > > > > > > > > >> > > > > > > > > reading.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > I
> > > > > > agree.
> > > > > > > > > This is
> > > > > > > > > > > > > how the
> > > > > > > > > > > > > > >> > > > current
> > > > > > > > > > > > > > >> > > > > > > > > protocol works
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > though.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > We
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > only
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > require
> > > > > > > > > "Read
> > > > > > > > > > > > > Group" to
> > > > > > > > > > > > > > >> join a
> > > > > > > > > > > > > > >> > > > > > group. We
> > > > > > > > > > > > > > >> > > > > > > > > could
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > consider
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > changing
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > this
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > but I
> > > > > > am
> > > > > > > > not
> > > > > > > > > > > sure
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > >> it is
> > > > > > > > > > > > > > >> > > > worth
> > > > > > > > > > > > > > >> > > > > > it.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 13.
> > > > > > How
> > > > > > > > is
> > > > > > > > > > > topic
> > > > > > > > > > > > > > >> recreation
> > > > > > > > > > > > > > >> > > > > > handled by
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > consumer
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > group
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > protocol?
> > > > > > > > > It
> > > > > > > > > > > > > would be
> > > > > > > > > > > > > > >> good to
> > > > > > > > > > > > > > >> > > > > > have a
> > > > > > > > > > > > > > >> > > > > > > > > section
> > > > > > > > > > > > > > >> > > > > > > > > > > >> on
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > this.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Noted.
> > > > > > > > From
> > > > > > > > > a
> > > > > > > > > > > > > protocol
> > > > > > > > > > > > > > >> > > > > > perspective, the
> > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > will
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> have a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > topic
> > > > > > id
> > > > > > > > so
> > > > > > > > > it
> > > > > > > > > > > > will
> > > > > > > > > > > > > > >> treat it
> > > > > > > > > > > > > > >> > > > like a
> > > > > > > > > > > > > > >> > > > > > > > > topic with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > different
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > name.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > only
> > > > > > issue
> > > > > > > > > is
> > > > > > > > > > > that
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> > > > fetch/commit
> > > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > > >> > > > > > > > > > > >> APIs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > do not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > support
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > IDs
> > > > > > so the
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> would
> > > > > > > > > > > > > > >> > > > reuse the
> > > > > > > > > > > > > > >> > > > > > > > > offsets
> > > > > > > > > > > > > > >> > > > > > > > > > > >> based
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > same.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > that
> > > > > > we
> > > > > > > > > should
> > > > > > > > > > > > > update
> > > > > > > > > > > > > > >> those
> > > > > > > > > > > > > > >> > > > APIs
> > > > > > > > > > > > > > >> > > > > > as well
> > > > > > > > > > > > > > >> > > > > > > > > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> order
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > to be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > consistent
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > end
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > to
> > > > > > end.
> > > > > > > > That
> > > > > > > > > > > would
> > > > > > > > > > > > > > >> strengthen
> > > > > > > > > > > > > > >> > > > the
> > > > > > > > > > > > > > >> > > > > > > > > semantics of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> consumer.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 14.
> > > > > > The
> > > > > > > > > KIP
> > > > > > > > > > > > > mentions
> > > > > > > > > > > > > > >> we will
> > > > > > > > > > > > > > >> > > > > > write the
> > > > > > > > > > > > > > >> > > > > > > > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > coordinator
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Java.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > Even
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > though
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > this is
> > > > > > > > an
> > > > > > > > > > > > > > >> implementation
> > > > > > > > > > > > > > >> > > > > > detail, do
> > > > > > > > > > > > > > >> > > > > > > > > we plan
> > > > > > > > > > > > > > >> > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > have a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > gradle
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > module
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > for
> > > > > > it?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Yes.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 15.
> > > > > > Do
> > > > > > > > we
> > > > > > > > > > > have a
> > > > > > > > > > > > > > >> scalability
> > > > > > > > > > > > > > >> > > > > > goal when
> > > > > > > > > > > > > > >> > > > > > > > > it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> comes
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > to how
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> many
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > >
> > members
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > new
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > group
> > > > > > > > > protocol
> > > > > > > > > > > > can
> > > > > > > > > > > > > > >> support?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > We
> > > > > > don't
> > > > > > > > > have
> > > > > > > > > > > > > numbers
> > > > > > > > > > > > > > >> at the
> > > > > > > > > > > > > > >> > > > > > moment. The
> > > > > > > > > > > > > > >> > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > should
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > support
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > 1000s
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > of
> > > > > > members
> > > > > > > > > per
> > > > > > > > > > > > > group.
> > > > > > > > > > > > > > >> We will
> > > > > > > > > > > > > > >> > > > > > measure
> > > > > > > > > > > > > > >> > > > > > > > > this when
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > we have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > first
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > implementation.
> > > > > > > > > > > > Note
> > > > > > > > > > > > > > >> that we
> > > > > > > > > > > > > > >> > > > might
> > > > > > > > > > > > > > >> > > > > > have
> > > > > > > > > > > > > > >> > > > > > > > > other
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > bottlenecks
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > down
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > road
> > > > > > (e.g.
> > > > > > > > > > > offset
> > > > > > > > > > > > > > >> commits).
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > 16.
> > > > > > Did
> > > > > > > > we
> > > > > > > > > > > > > consider
> > > > > > > > > > > > > > >> having
> > > > > > > > > > > > > > >> > > > > > > > > SubscribedTopidIds
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > instead
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > of
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > > > SubscribedTopicNames
> > > > > > > > > > > > > > >> in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> >
> > > > ConsumerGroupHeartbeatRequest?
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> Is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > idea
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > since we
> > > > > > > > > have
> > > > > > > > > > > to
> > > > > > > > > > > > > > >> resolve the
> > > > > > > > > > > > > > >> > > > > > regex on
> > > > > > > > > > > > > > >> > > > > > > > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > server, we
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> can do
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > same
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > the
> > > > > > > > topic
> > > > > > > > > > > name?
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > >> > > > difference
> > > > > > > > > > > > > > >> > > > > > is that
> > > > > > > > > > > > > > >> > > > > > > > > > > >> sending
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> regex is
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > more
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > efficient
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > whereas
> > > > > > > > > > > sending
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> topic
> > > > > > > > > > > > > > >> > > > names
> > > > > > > > > > > > > > >> > > > > > is less
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > efficient.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > Furthermore,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > delete
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > recreation is
> > > > > > > > > > > > > easier
> > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > > handle
> > > > > > > > > > > > > > >> > > > > > if we
> > > > > > > > > > > > > > >> > > > > > > > > have
> > > > > > > > > > > > > > >> > > > > > > > > > > >> topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > ids.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > The
> > > > > > idea
> > > > > > > > > was to
> > > > > > > > > > > > > > >> consolidate the
> > > > > > > > > > > > > > >> > > > > > metadata
> > > > > > > > > > > > > > >> > > > > > > > > lookup
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > on the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> server
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > for
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > both
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > paths
> > > > > > but
> > > > > > > > I
> > > > > > > > > do
> > > > > > > > > > > > agree
> > > > > > > > > > > > > > >> with your
> > > > > > > > > > > > > > >> > > > > > point. As
> > > > > > > > > > > > > > >> > > > > > > > > a
> > > > > > > > > > > > > > >> > > > > > > > > > > >> second
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > though,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > using
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > ids
> > > > > > may be
> > > > > > > > > > > better
> > > > > > > > > > > > > here
> > > > > > > > > > > > > > >> for the
> > > > > > > > > > > > > > >> > > > > > delete and
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > recreation
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > case.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > Also, I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > suppose
> > > > > > > > > that we
> > > > > > > > > > > > may
> > > > > > > > > > > > > > >> allow
> > > > > > > > > > > > > > >> > > > users to
> > > > > > > > > > > > > > >> > > > > > > > > subscribe
> > > > > > > > > > > > > > >> > > > > > > > > > > >> with
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> ids
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > in
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > future
> > > > > > > > > because
> > > > > > > > > > > > that
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > >> the only
> > > > > > > > > > > > > > >> > > > > > way to be
> > > > > > > > > > > > > > >> > > > > > > > > > > >> really
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > robust
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > topic
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > re-creation.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > Best,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > David
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > On
> > > > > > Tue,
> > > > > > > > Jul
> > > > > > > > > 12,
> > > > > > > > > > > > > 2022 at
> > > > > > > > > > > > > > >> 1:38 PM
> > > > > > > > > > > > > > >> > > > > > David
> > > > > > > > > > > > > > >> > > > > > > > > Jacot <
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > >
> > > > > > djacot@confluent.io>
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > >
> > wrote:
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > Hi
> > > > > > > > > Justine,
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > Thanks
> > > > > > > > for
> > > > > > > > > > > your
> > > > > > > > > > > > > > >> comments.
> > > > > > > > > > > > > > >> > > > Please
> > > > > > > > > > > > > > >> > > > > > find
> > > > > > > > > > > > > > >> > > > > > > > > my
> > > > > > > > > > > > > > >> > > > > > > > > > > >> answers
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > below.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > -
> > > > > > Yes,
> > > > > > > > > the new
> > > > > > > > > > > > > > >> protocol
> > > > > > > > > > > > > > >> > > > relies on
> > > > > > > > > > > > > > >> > > > > > > > > topic IDs
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > with the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > exception
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > of the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > > > topic
> > > > > > > > > names
> > > > > > > > > > > > based
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > >
> > > > > > > > ConsumerGroupHeartbeatRequest.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > am
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > sure
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > if
> > > > > > using
> > > > > > > > > topic
> > > > > > > > > > > > > names
> > > > > > > > > > > > > > >> is the
> > > > > > > > > > > > > > >> > > > > > right call
> > > > > > > > > > > > > > >> > > > > > > > > here.
> > > > > > > > > > > > > > >> > > > > > > > > > > >> I
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > need to
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > think
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > about
> > it
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > >> > > > > > > >
> > > >
> > > > a
> > > > > > little
> > > > > > > > > more.
> > > > > > > > > > > > > > >> Obviously, the
> > > > > > > > > > > > > > >> > > > > > KIP does
> > > > > > > > > > > > > > >> > > > > > > > > not
> > > > > > > > > > > > > > >> > > > > > > > > > > >> > change the
> > > > > > > > > > > > > > >> > > > > > > > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > >
> > >
> > >
> > > --
> > > -- Guozhang
> >
>
>
> --
> -- Guozhang